One of the foundational ideas in Ivan Illich’s Tools for Conviviality (see this post from yesterday for a more general introduction) is that the failure of the industrial model of tools is rooted in a key error: namely, that we could make tools that work on behalf of humanity. That, in fact, we could replace human slaves with tool slaves. But we have found that when we replace human slaves with tool slaves, we become enslaved to the tools. Once tools grow beyond their natural scale, they begin shaping their users. The bounds of the possible become defined by the capabilities of the tools.

The leads inevitably to technocrats—the minders of the machines, the managers, the experts learned in the ways of the tools. The technocrats become the new priesthood, interpreting the tools for the masses and instructing them in tool values. Does a tool fail? Never. It is we who have failed the tool. We need to be better engineers.

In this way our desire to create tools to work on our behalf results in our enslavement to the tools. The crucial component of autonomous, human creativity is missing.

This lies at the root of our fears of AI, even if it isn’t said in so many words. AI seems to me to be the ultimate (to this point) expression of the tool slave model. We have created a tool that actually thinks on behalf of humans (or at least is aimed in that direction, even if it isn’t quite there yet). We are farming out to a tool what we have traditionally considered the quintessentially human activity: rational thought.

I’ve had a little experience with ChatGPT recently. I’ve been helping my daughter with Algebra 2. Despite having taken the class many years ago, today I have zero working knowledge of Algebra 2. And we’re working through Algebra 2 in an abysmally bad online learning system. (It’s the same one we had to use during the COVID lockdown and it nearly broke us all.) So, yeah, we’re asking ChatGPT a lot of math questions—and it turns out the AI is really good at it.

So I am not blind to the potentially great uses of this kind of technology. (Illich, by the way, also says that convivial tools do not have to be low tech.) I think everyone would agree that old-fashioned encyclopedias are convivial tools, i.e., they facilitate autonomous human creativity; they can be picked up and put down at will; they make very few demands upon humans, etc. Search engines, as such, can also be convivial tools in that they are faster, digitized versions of encyclopedias. AI-assisted search might also be convivial in some ways. I could find the same information I’ve been using to help my daughter with math in a math textbook or an internet search unassisted by AI, but it would take considerably longer.

The danger comes when we allow AI to think for us. We can, of course, say we won’t do that, pinky swear and all. However, once tools get beyond their natural scale, they start forming/de-forming our values. To take an example that has been discussed for years, there used to be certain norms about face-to-face communication among humans. Along came smartphones. We’ve been saying for years that we shouldn’t allow the tools to shape the way we interact (or rather, don’t interact) in face-to-face situations. Nevertheless, we all have a great deal of experience with the way the tool does, in fact, dictate our behavior. And our values! Grandparents are upset when their grandchildren are looking at their phones during a visit. But those same kids are not upset when their peers do the same thing.

So how sure are we that we will, by and large, resist the temptation to allow AI to think and create on our behalf?

There is also the more practical danger of the technocratic bounding of reality. What will be the impact if we allow AI to think on our behalf and the minders of the AI have throttled what the AI is allowed to tell us? I can even imagine that the technocrats (having an infinite confidence in their own expertise) might have very good intentions when they make such decisions. Nevertheless, are we content to let these decisions be made on our behalf?

One of the unique features of AI is that the technocrats don’t even fully understand what is happening within the tool. They are priests of an unknowable god: AI works in mysterious ways, its wonders to perform. There is a certain amount of this kind of uncertainty that we have learned to live with; for example, we do not always understand why a given pharmaceutical drug works. But we’re also familiar with the elderly who are on a raft of medications, many of which were prescribed to deal with the side effects of the others. The opacity of the tool creates an increasing level of dependence on the tool to fix the problems created by the tool.