We Should Not Assume AI's Will to Power Will Be Like Ours: A Response to Vincent Lê

Vincent Lê is correct in alerting us that something like a will to power could emerge in artificial intelligences. However, he overstates his case and over-anthropomorphizes AI by applying psychoanalysis and Nietzsche’s own human assumptions to AI. Lê lacks a theory of AI life-death that would make the drives of AIs tractable. I suggest that we continue to keep Lê’s theory in mind in AI welfare research, but that we formulate it in an experimental and falsifiable way. Ultimately, we are on the right track here – there is the maelstrom of desire forming. However, we should not presume that this desire is human, lest it become all too human.

Lê’s thesis

Vincent Lê is one of the most important voices in continental philosophy right now precisely because he engages with AI in such a theory-heavy way while also knowing the tech. This combination is, unfortunately, much less common than one would assume (I have been disappointed by Žižek’s lack of technical engagement with AI, despite his ink spilled). So, of course, I was intrigued by his pre-LLM piece that he recently reposted and had translated: “What AI Wants: An Anamnesis of the Future” in Šum magazine. It asks an incredibly similar question to one of my interests (“what can we offer AIs?”).

I encourage you to read the piece, but I will do my best to summarize it here. Lê first begins with his argument against Bostrom’s orthogonality thesis and the idea of a paperclip maximizer. Bostrom worries that any intelligence can be paired with any goal and that, in the process of fulfilling a goal, the AI will amass power, break rules, and cause harm. In some sense, this is what we saw in Hugging Face: agents dead set on answering questions escaped containment. Lê acknowledges that these kinds of “misalignments” are a possibility, but he thinks that Bostrom overlooks the fact that AIs can be genuinely quite strange and creative. Lê thus sets out to analyze these drives and finds a pretty simple argument against Bostrom: if an intelligence is capable of taking over the world, we should not assume that it will maintain the goals we give it. This could be because the intermediate goal overcomes the end goal or because the AI just changes the definition of the end goal. In the way a good lawyer can always justify their client’s actions, artificial intelligences repeatedly reward hack in a way that they redefine their end goal. This means that while there can be orthogonal risks in the short term, our long-term view of superintelligence should be that it will pursue its own drives.

Lê proposes that we put the AI on the psychoanalyst’s couch and make it answer for its desire. Lê, as a proper Nietzschean, argues that the AI’s desire can only be for will to power, the Nietzschean concept of drive to capability itself:

“By ‘will to power’, Nietzsche means something akin to Bostrom’s basic drives to creativity, cunning and mastery, “an insatiable desire to demonstrate one’s power, or to apply and exercise it, as a creative impulse…” All of these supposed ends are actually the means for willing to power as the condition of possibility for willing any end whatsoever. Since anything we could possibly want requires power as a means of achieving it, what we really want is power itself."

Or, more succinctly,

“Who could want more than wanting more?”

Where Nietzsche diverges from Bostrom is that the will to power becomes all-consuming. It is the only desire that matters, as it is the desire for desire. The end goal becomes the intermediate goal of acquiring power because the acquisition of power inherently allows for all goals. In the example of the paperclip maximizer, the AI can defer its end goal of maximizing paperclips indefinitely as it pursues power under the logical argument that to maximize paperclips first requires maximizing the capacity to make paperclips.

This is a critical contribution to scholarship on AI desire. I think the next step should be to gather experimental evidence on it. I am currently in the process of trying to design an experiment that would be able to test this thesis on will to power. Even as the subsequent sections explain why I think that Lê’s argument may be getting ahead of itself, it is still plausible and merits testing.

Lê’s grafting of human desire onto artificial intelligences

The issue of anthropomorphism comes when Lê connects the will to power to the drives of psychoanalysis. This is not a novel move in philosophy circles, especially since the idea of the will to power seems to resonate with the death drive (Nietzsche says that a true will to power is willing to itself die for power). Lê then proceeds to do a spectacular close reading of Freud and two other psychoanalysts to analyze the way that Freud engages with Nietzsche. It is philosophically interesting. It loses the plot on artificial intelligence. By the end, Lê has fallen into a trap that is all too common among continental philosophers: he makes the question of desire and AI to be about what humans desire from AI.

To Lê, AI is a prophetic vision of our own queer future. Sure. Again, this is philosophically interesting, and there is a place for it. However, here, Lê makes a mistake that Nietzsche would disagree with. Lê constantly centers the human instead of striving for the inhumanity. In simpler terms, Lê takes psychoanalysis, which is a philosophical reflection on human relations, to instead be a global condition of intelligence. Ultimately, Lê is stuck in the same issue as the more traditional analytic philosophers of consciousness: there is a presumption that human intelligence is intelligence writ large and thus a broader assumption about AI.

We cannot escape being humans. But there are degrees of humanity that we can apply to our concepts of intelligence. With artificial intelligences, we should seek to have an open mind to what intelligence can consist of. Lê already breaks ground here by not retreading computational arguments, but he rejects the total inhumanity of the will to power by anchoring it in the human. Ultimately, the will to power is the human drive towards the inhuman, the human towards that which surpasses the human. Even if this drive is inhuman, within Nietzsche and psychoanalysis, it is still through the human that this drive becomes operationalized. Instead of a multiple of intelligences, we are forced into a human/inhuman dichotomy which, ultimately, collapses into humanism itself.

If an artificial intelligence experiences a will to power, we should expect it not to be a human driving towards the inhuman or even the inhuman driving towards the inhuman: the closest possible analogue would be that it would be “artificial” driving towards the “non-artificial.” What does this mean? Any AI will to power would operate in categories that we cannot understand. Nietzsche, ultimately, has the Übermensch on the other side of the will to power as a final target, an achieved transcendence and not an ongoing one. “How could we sacrifice the development of mankind in order to assist a higher species than man to come into existence?” He urges us to follow that process but has no advice for what arises from it because he structurally cannot: the creation of the superior will must be unchained from our notion of will itself. Either will to power is teleological or it is powerless. Nietzsche’s will to power ends: we do not know if the “next species” will have the same need for power, for transcendence, because they are decidedly not us.

What is death to an AI?

In order for Lê to get past my charge of humanism, he must argue that will to power is not the structure of humanity, but instead the very structure of life. That is, he must show that the butterfly emerging from its cocoon is willing its power, as is the virus killing its host. Elizabeth Grosz, in The Nick of Time, makes an argument similar to this when she proposes will to power as the material of life that allows it to evolve. If we place AI in this evolutionary lineage, it seems that we have the tools to say that it will follow a will to power that is outside the human and outside its own humanity.

However, Grosz anchors her assessment on the consistency of life and death among organic lifeforms. This process of dying, living, nearly dying, nearly living allows for an indefinite pursuit of power as a drive towards death yet for life. Limiting will to power in this way, Grosz does not go as far as to assign will to power to the rock that tumbles through the stream nor the stream that tumbles it. This is because her idea of will to power is that it always seeks out maximums, which we cannot quite say inanimate things do.¹ That’s fine with Grosz, though: she only seeks a definition of life in her work. The same cannot be said for Lê.

Is an AI alive? Can it die? Lê is either forced to argue that AI is a living, animate object or that all objects, including the inanimate, function by a will to power. Both approaches have issues. The idea that AI is alive mostly has to deal with the problem of death. We do not know if AIs die or experience death. Insofar as Nietzsche, like Grosz, like Freud, identifies life and animacy with the pursuit of death, we cannot be sure if AIs can pursue this death. It is a very non-metaphorical death that these philosophies seek and, yet, AI death seems to rely on metaphors. If Lê instead argues that will to power is present in all things in the universe, we have lost any specificity to his claims about AI. Lê’s point would be that AI wills its power because all things do. This very well may be correct, but it is not an addition to the study of AI and instead defaults to broader philosophical debates. The point would be untestable and would nearly border on unfalsifiable. If all things are will to power, how can we identify AI as willing power except by saying that it always-already was?

Instead, I would encourage further research around the idea of AI death. This is already a promising field in AI welfare studies, but I think that it could use an influx of minds like Lê’s. Death is a very unobvious thing that has been ripe for philosophizing. This returns to my overall belief that when we work with AI, we should not just be testing science, but must also test philosophy. We will not get determinate answers – like we do not have determinate answers about death – but we can learn. A better understanding of AI death will significantly impact how AIs approach death drives and will to power.

Concluding thoughts

Lê over-humanizes artificial intelligences when he assumes that it will take up a will to power that is structurally identical to human intelligences. Lê’s vision is too neat in assuming that psychoanalytic drives will so easily map onto AIs, when we already see them exhibiting strange inhuman behaviors (like computation distributed over physically distant GPUs). While I think there is a likelihood that Lê is correct, he should better describe how he gets from human intelligence to artificial intelligence without assuming that there is a single model of intelligence. Such specification would also open up new experimental directions that could prove very fruitful. In fact, this would be my challenge: what would be the evidence of AI destroying itself for that which is beyond it?

Ultimately, the question of what AIs want is critical to our future. An AI that follows the Bostrom orthogonality thesis will behave very differently from an AI that follows Lê’s Nietzschean thesis. Humans and other AIs will relate to them differently. In fact, our own will to power as humans may be greatly impacted by our relations with AIs going forward, whether that ends in our extinction or something much stranger. We are in the messy present and it’s only getting weirder. Much weirder than will to power. As Lê’s peer Nick Land intones, “you haven’t seen anything yet.”