Recursive Enshittification

I’m not a math person, but when I was in business school, I was quite taken with a phenomenon in statistics called “regression to the mean”. Extreme values, when measured again, tend to move toward the average. Seems obvious, and as a fundamental concept of statistical analysis, it is. The weird thing is that it’s behaviorally counterintuitive. We think, “Shohei struck out 10 and then went 3 for 3 at the plate” and when we tune into the next game, we’re disappointed when there’s no repeat performance. Skill and luck aligned to give him an outlier game, but luck was lost in the next game and the performance was back to average (which obviously in Sho’s case is still pretty good.)

Lately I’ve been thinking about the averaging itself, as a function of AI. Large language models are, by definition, engines of averaging. When queried, they find the most probable, the most representative, the most average response across the distribution of everything they’ve been trained on. This is part of why models hallucinate, a big reason why they are inherently biased, and why I believe the architecture is fundamentally flawed, though that’s another post.

As I have experimented with AI to assist in my own work, it performs well on certain tasks (organization, summarization, recipes) and poorly on others (regulatory investigation, creative ideation and evaluation, writing creatively.) In the creative writing use case it is startlingly bad, I am almost always disappointed with the outcome, and I’ve found that the time saved becomes time wasted as everything needs to be re-written. As Eve Fairbanks articulated, “nothing is quite right” in the creative output. It all comes across to me as sort of bland.

A common discussion thread as to the reason for this is that the averaging [see above: regression] that makes LLMs so good at coming up with an acceptable response most of the time is what keeps them from coming up with an exceptional response. . .ever. This is one component of a well known phenomena in machine learning known as “model collapse”, caused by something resembling the following:

1. An LLM is trained on all of human knowledge and governed with a response architecture telling it to return the statistically most relevant response to any question.

2. A human prompts an LLM for an output. Multiply this by a jillion to represent all of human usage.

3. The LLM returns a response - which on the average, is average [see above again: regression]. Multiply this by a jillion to account for all the new IG posts, LinkedIn updates, High School spanish homeworks, and college entrance exams created by these queries.

4. Update the training set with all this new “content”.

5. Human requests a new output from LLM, however this time, the response is even more average, because the training set contains it’s original data + all of the incrementally generated average responses.

6. Repeat steps 1-5 until the responses are entirely useless.

This is a known issue, in which “data quality” is a key AI workstream that many consider a significant technical blocker to improvement in output quality. Will we run out of verifiably human created training data? Probably at some point. I mean, take a look at this mind blowing chart from a ProfG Media report:

There are many ways of band-aiding this output degradation (synthetic data verification, ensemble models, etc) but that’s not the point of this post.

There’s a second-order problem that doesn’t get discussed enough. It’s not just that the outputs get blander. It’s that the thinking gets blander that’s prompting the outputs.

After enough of the above loops to develop a working relationship with consumer AI, add these steps to the sequence:

1. Human requests a new output from LLM, however this time, the request itself is blander because the human has gotten a little lazy and lost some of their critical thinking capability due to cognitive outsourcing. (Don’t take my word for it.)

2. A bland request gets an even blander output. (Bland x bland = bland²)

3. This (plus all of the AI generated “books” on Amazon) get used as the input for the next training run.

4. And so on, ad infinitum.

This is problematic, as it accelerates the already existing issue of averaging at the expense of our own critical thinking ability into a kind of self-fulfilling enfeeblement cascade. I call this accelerated phenomena, “recursive enshittification”, and while not a perfect analog to the original “enshittifcation”, it was too hard a coinage to resist.

The brain muscle that produces original thought doesn’t get used. Over time, it gets harder to use. When something is hard, we tend to avoid it, and what started as apathy becomes inability. This didn’t start with LLMs, it started maybe with rock music, then TV, then probably with video games, then definitely with cell phones and social media. When I moved to LA, I frequently got lost if I didn’t have my Thomas Guide, but now we don’t go anywhere without a GPS. LLMs are a horizontal tech that introduces information ease across many domains simultaneously and they are on loudspeaker. I mean, I’m concerned about my child’s ability to navigate the world - thank the gods he reads!

Understand, this isn’t a diatribe against LLMs. On the contrary, their existence, influence, and impact is fascinating to me - fascinating enough to get me to take classes and try to write about it. But I do have concerns about our overall ability to contend with distraction, focus critically on a task, produce original thought response, and grow our own understanding as a result. It’s called learning (dummy) and it takes actual work - work which is being relinquished very quickly to an easier faster average.

What does this work look like? For me, in this instance you’re reading, it looked like this:

1. Have an idea - in this instance a phrase (recursive enshittifcation) - and consult with an LLM about how to articulate it.

2. Be disappointed in the result and feel lazy and shitty about beginning with step 1 at all.

3. Stew on it for a few days, read some relevant research, essays, blog posts, and take various stabs at writing about it - away from my LLMs.

4. Spend time to finally flesh out my own text and feel like I actually accomplished something. Bonus: I’ve also internalized the thought stream now through my own reading, thinking and writing, and could recall and defend it at will. This learning thing is magical!

If you eschew my process, here are a few focused approaches that may work for you:

1. Use your LLM for a sparring partner

2. Engage in chain-of-thought prompting to build your skepticism muscle with regard to LLM outputs

3. Use an LLM to gather resources for you, not as the resource itself, and then actually read them rather than getting AI summaries.

4. Come up with your own approach - remember. . .it takes work!