I don't claim these are novel thoughts, but they are things I want to jot down mostly so I can revisit them later.

Smarter or Dumber?

There is a lot of debate about whether AI, LLMs in particular, will make people smarter or dumber. The case for dumber is that LLMs can be used as a substitute for learning. Rather than struggling through a subject and working your brain around understanding it, you can just have an LLM produce the desired output. The case for smarter is that LLMs can be used like a very patient tutor who answers your questions and thereby helps you understand and learn things.

Generally speaking, I think the answer is obviously both. This is true even for a particular individual who may use AI to help them understand certain things better while also using it to help them avoid learning other things. This tracks my own experience. I have used AI to help me understand certain aspects of web programming better than before, while also using it to avoid learning R/ggplot2, which I now use extensively to produce graphs.

Across society, I think the answer varies, which could create some troubling educational disequalization. There are some people who, for internal reasons, just really like to understand and learn about things. These are the people who teach themselves things with audio, video, books, and other written materials, even outside of formal education. For people with that kind of curiosity and internal drive, AI-as-tutor will allow them to get smarter. But there are other people who don't care at all to understand or learn things and have to be coaxed into it in various ways: parental/social/peer pressure, economic incentives, and so on. For those people, AI-as-replacement will probably result in them being dumber.

Collapse of Main Merit Indicator

For at least some part of human history, productivity had a big physical component. In such an environment, people who were more physically capable were presumably able to produce more and therefore had more "merit" in the way we currently define it. But industrialization converted a lot of physical labor into machine labor, reducing the relative economic value of being strong or fast or similar. Now intelligence reigns.

But what we can see from this little historical story is that what sorts of characteristics make someone have "merit" depends in large part on totally contingent technological environments. In one technological environment, physicality may be the primary indicator of merit. In another, intelligence may be the primary indicator of merit. In yet another, it is conceivable that neither individual physicality nor intelligence would matter that much for individual productivity and thus merit.

Our current society seems to have built a huge ideological edifice around the fact that certain mental traits fetch the highest labor incomes right now. We don't just see that as a quirk of our current technological environment, but instead read huge meaning into it. We believe that the economic inequalities that result from this contingent reality are even good, that they reflect that good people are getting paid well and bad people are getting paid badly, and so on. You can see this at the most extreme end in Silicon Valley where it seems a lot of people who had certain mathematical and programming skills that happened to result in high pay starting about 30 years ago also imagine themselves to be world-historical geniuses.

I don't know how far AI will actually go, but I find the idea that it could destroy our current model of merit quite thrilling. Already we have seen that AI can do for a few dollars coding tasks that used to fetch programmers tremendous sums of money. If this were to actually spread across a lot of highly-paid, high-intelligence type tasks and jobs, that would be a pretty amusing state of affairs. What if these mental traits we put so much meaning into, in large part because of the labor incomes they attract, become as economically valuable as being able to quickly multiply four-digit numbers in your head? How long would it take for our current intelligence-based hierarchy of merit to fall apart?

I like the egalitarian possibilities of this. People believing that they are owed 20x the income of someone else because they are smarter than them is pretty insane on its face but does seem to be compatible to some extent with social stability. If AI makes it so that this extra intelligence doesn't really have much economic value, then these sorts of ideas become much less plausible.

Of course, we might replace intelligence with some other merit indicator and wind up in the exact same inegalitarian situation as before, with nobody ever stopping to ask themselves whether these temporary, technologically contingent hierarchies of merit are fundamentally silly and ridiculous. That would be a pretty bleak outcome but darkly funny as well. Maybe we just deep down love inequality and will find some other thing to base it on even if intelligence no longer works.

One Dark Possibility

If the AI maximalists are right, AI and much-improved robotics will make most human labor not really necessary. I have no idea if this is true. Right now, I don't see anything like that, but the premise of this argument is that the technology is going to get a lot better than it currently is.

But what if (most) people really are hardwired by evolution or whatever to want/need to get up and go work for a boss and have this particular kind of structure and all that? That by itself is not so dark. In such a world, you could create all manner of essentially fake jobs for people who like jobs to go to. In that world, a job is basically like a hobby, and already some people have hobbies like this, where they play flight simulators or trucking simulators. So that's fine.

What would not be fine is if most people had this preference for the job hobby while also preserving our current notion of deservingness such that they make everyone engage in the job hobby, even those who are totally fine doing other things with their time. So right as we finally win this great struggle against scarcity, we create a simulation of it because that scratches certain evolutionary itches we can't shake.