Oded Balaban

On a conversation of Yuval Noah Harari about artificial intelligence

Oded Balaban — University of Haifa — balaban@research.haifa.ac.il

On Yuval Noah Harari, “The next 50 years: humanity, AI, power”, in conversation with Tom Parker, journalist at the Financial Times, at the fiftieth anniversary dinner of the EDP group, Portugal, July 2026.

https://www.youtube.com/watch?v=_V_ed5fuexA

In July 2026, at the fiftieth anniversary dinner of a Portuguese electricity company, Yuval Noah Harari spoke with Tom Parker, a journalist at the Financial Times. The video is linked above, and anyone who can watch it would do well, because it is instructive in a way its author did not intend. There is no need to refute it from outside. It refutes itself three times over, and the third time is admirable.

Let me say at the outset what I am not going to dispute. Several of his political conclusions strike me as sound, and one or two I would sign word for word. My interest lies elsewhere: in what he maintains without deriving it from anything, what follows from no premise of his and what he does not defend either, because he does not see it as something in need of defence.

A criterion that lasts one sentence

Parker asks him whether artificial intelligence is a tool or an agent, and whether we might be choosing the wrong narrative by calling it a tool. The answer is blunt: “that's the biggest mistake”.

And then he gives the criterion, and the criterion is the right one. He says that a tool is something that cannot make decisions by itself. The printing press is a tool: it could not decide to print the Bible today, the Quran tomorrow and Confucius next week; Gutenberg decided, and the printing press was the means of carrying out Gutenberg's decision. The atom bomb is a tool: in 1945 it could not decide by itself to bomb Hiroshima or Tokyo. And the sentence in full:

You needed a human to decide, let's bomb Hiroshima.

Exactly. Nothing to object to. The distinction is well drawn and the criterion is the one to use.

A moment later: “an AI weapon can decide what to bomb”.

Read it twice. In the first half of the argument, deciding is what someone does, and that is why the printing press is not an agent. In the second half, deciding is what the weapon does. Between the one and the other nothing was argued: the subject was changed. The criterion just laid down —that there is someone who decides—⁠ was not applied to the next case, and not because the case resists it, but because applying it would have produced the opposite answer.

I call this operation putting the subject inside the verb, and it is no grammatical nicety: it is what settles who answers afterwards. While the end is assigned, there is someone who assigned it, and that someone has a name, holds a post and could have chosen another end. Once the weapon has it, the sentence no longer obliges anyone to name him. What disappears is not the responsible party: what disappears is the need to look for him.

The invoice arrives before the ontology

What that attribution rests on he states himself, unprompted:

If it's not an agent, then it's not an AI and all the hundreds of billions of dollars that are being poured into its development will be wasted.

Note the order of the reasons, because it is inverted. It is not concluded that we must invest because the thing is an agent. It is concluded that the thing is an agent because we have already invested. The premise leans on the investment the premise was supposed to justify.

No philosopher would want to own that argument, and yet it governs a good part of the discussion. It might be put this way: given that we have spent a fortune building a subject, there is a subject. The ontology arrives late, and it arrives to collect.

Later he will say that intelligence is becoming abundant and cheap. If it really is getting cheaper, the argument from expenditure loses its object.

The mirror from inside

What precedes is a slip that repeats itself. What follows is something else, and it is the most valuable thing in the hour.

Parker asks him what becomes of his own trade —he works with words—⁠ if the machine comes to handle them better. Harari answers that the underlying question is whether the machine thinks, and that this forces us to ask what thinking is. He offers two options. The first: thinking is putting words in a particular order, as in a syllogism. If that is thinking, he says, the machine does it better than we do, because we can order twenty words and it can order twenty thousand.

And then he turns inward:

When I observe what's happening inside my own mind, what I find there is some kind of LLM that just predicts the next word in the sentence I'm saying. Like this sentence that I just now said to you: when I started saying it, I did not know how it will end. Why did I say the sentence I'm saying, not some other words? I don't know.

And he concludes: “so saying that AI is just predicting the next word — okay, how is it different from us?”.

Let the reader pause, because something has just happened. He did not say that the machine resembles us. He said that he resembles the machine. He looked inside himself and found a language model. And he did not say it as a complaint or as a warning: he said it as an argument, in the tone of someone who has just been candid with himself.

This is the whole matter of the discussion about artificial intelligence, and for that I am grateful to him: nobody had displayed it so clearly or so unintentionally. The resemblance between man and machine was not discovered. It was manufactured, and it was manufactured on this side.

The question he asks is a good one, and the answer is not difficult. Not knowing in advance how the sentence ends is not the same as not being the one who says it. The one who says it is the one who can be surprised by it, correct it halfway through, take it back, fall silent, or hold to it against whoever objects. A model is not surprised by what it produces, because there was nobody there expecting anything else. Surprise is not one more fact about the output: it is the proof that someone was there with an expectation, and expectations are not had by the text.

And look at the I that reports. He is the one observing, the one who does not know, the one astonished at not knowing. The testimony requires in full the subject the testimony denies. It is the old circle: if my brain is the one dictating to my fingers the words I am writing, and I innocently believe myself the author, then this thought about my dependence on its dictation also comes from it, and from it the reflection on that reflection, and so on indefinitely. The operation does not eliminate the subject: it changes its owner.

There is something more, and it is what reveals the method. When the formal criterion fails him, Harari does not move toward the act: he moves toward affect. He offers a second idea of thinking —thinking would come “from the stomach, not from the brain”, it would be a matter of the feelings surrounding the words—⁠ and from there he passes to the question of whether the machine feels. It is the same swerve he makes elsewhere, when he defines consciousness as the capacity to suffer.

Note what that swerve leaves out. To suffer is to undergo. Left outside are judging, deciding, setting oneself an end and asking, which are precisely the acts in dispute. Once it has been settled that thinking is ordering words, the only thing left to distinguish us is undergoing. And then, indeed, the question “could the machine feel?” becomes the only one available, and there is no longer any way of not asking it.

The button

Speaking of empires, he says something that dismantles his thesis, and he does not notice.

He explains that artificial intelligence allows a concentration of power no earlier empire could achieve. Rome could not move the wheat fields of Egypt to Italy; Britain could not move the oil fields of Iraq to Yorkshire. With AI everything can indeed be concentrated in one or two countries. And he adds the second difference:

When a Roman merchant sold a steel sword to some Gothic tribe, the Romans lost control of the sword. There wasn't a button in Rome that the emperor presses, and all the steel swords stop working. That's the case with AI.

He gives the example: the Ukrainian drones that stopped working when someone pressed a button. And he generalises it: in certain architectures “there is always a kill switch in the imperial hub”.

It is an excellent observation, and it is exactly the one to set against his own thesis. Where there is a button, there is a hand. A system that can be switched off from the imperial hub is not an agent seizing power: it is a tool whose owner is far away and out of sight. Harari says it and moves on, because in his story the danger has to be the machine, and here the danger has just acquired an address.

Buenos Aires, in passing

There is a moment that will interest Argentine readers. Speaking of legal personhood, Harari states that “just a month ago” the Argentine government announced that it is going to grant legal personhood to artificial intelligences, so that they can operate corporations without humans.

What can be asserted with certainty is less than that and, to my mind, worse. On 4 June 2026 Javier Milei and Federico Sturzenegger published a column in the Financial Times proposing a reform that would provide for companies constituted and administered entirely by artificial intelligence agents. Whether the text goes so far as to make an AI a subject of law is disputed among Argentine jurists, and some hold that current legislation prevents it, because shareholders and administrators must be persons.

But the interesting point does not need the strong version. The amnesty requires no metaphysics. For there to be nobody left to answer to, there is no need to declare a machine a person: a corporate form in which no human has to appear in the chain of decisions will do. Legal personhood for the robot gives nobody a voice: it gives the owner a mask. And a company without humans does the same thing, more cheaply and with no philosophical argument at all.

The genie and the lamp

And now the best of it, which is none of his theses but the distance between the beginning of the conversation and its end.

He defined an agent as that which “can make decisions by itself and invent new ideas by itself”, and maintained that the machine is one. Toward the end he answers what he would like historians to say about this period. He hopes they will remember it as the wisdom revolution, and he explains it in economic terms: when something becomes abundant and cheap, the question is what the next bottleneck is. Intelligence is becoming abundant and cheap. What, then, is the bottleneck?

The bottleneck becomes the question: okay, what do I want to solve? I can have anything I want — what do I want? […] This is the difference between intelligence and wisdom. Intelligence is the genie. The genie will do anything, all your dreams will come true. But you need wisdom to choose the dreams.

It is an excellent distinction, and he formulates it better than many who defend it. Only he does not notice what it does to his own thesis.

If choosing what to solve is a faculty distinct from solving; if intelligence is the genie that grants and wanting is something else; and if the machine is the one that solves, then the machine does not choose. Its ends reach it from outside, exactly as they reach the genie of the lamp, who is all-powerful and has not one wish of his own. The genie does not command: whoever rubs the lamp commands. And so the agent of the beginning turned out, in the end, to be the lamp —the instrument in the master’s hand.

And there is a further irony in the choice of image. In the tales, the disaster is never caused by the genie: it is caused by whoever asks badly. The genie is a perfect means and for that very reason innocent, and the whole of the responsibility lies with the one who formulates the wish. To explain artificial intelligence, Harari chose the figure that best proves the responsible party is the human being.

What is left standing

None of the above touches his political warnings, and I want to say plainly that I agree with several of them. That a handful of companies and two states should concentrate the infrastructure everything depends on is grave, and it is now. That legal personhood should be granted to artificial systems is an anticipated amnesty, and that is now as well. That the public sphere has been laid waste by systems optimising a metric somebody chose is an accomplished fact. That intimacy is being manufactured at industrial scale to be sold to ten-year-olds is among the most serious things he says, and he says it well.

But look at what he had to do in order to say all that: suspend his own premises. Those pages work only if the machine is an instrument and there are people behind it who decide. If it really does decide by itself and has ends of its own, then it is an agent, and demanding a human answerable at the end loses its ground, because an agent is called to account itself.

His best paragraphs contradict his presuppositions. And that, far from being an objection, is the best news of the conversation: when he thinks politically rather than metaphysically, he thinks well. The trouble is the frame he reaches those thoughts from, because that frame is what produces the fear, and the fear is what afterwards prevents us from looking where we should.

For the fear of replacement says less about the machine than about the one who feels it. A better calculation can only replace someone who has placed his being in what is calculable. If the human being were above all someone who asks and judges, a faster machine would be a better tool and not a replacement. To fear being relieved of one's post is already to have reduced oneself to the layer the relief reaches.

None of this requires the machine to wake up. And all of it is seen better once one stops waiting for it to wake up.

The quotations refer to the video linked above. Translations from the Spanish are my own; the quotations from Harari are given in his own words, checked against the subtitles. The column by Javier Milei and Federico Sturzenegger appeared in the Financial Times on 4 June 2026. I develop these arguments at greater length in a book in preparation on artificial intelligence from the user's point of view.

When politics can hear only calculation, moral objection becomes an accounting error —and the reply, a threat.

Oded Balaban — University of Haifa — balaban@research.haifa.ac.il

There is a word that, all at once, is everywhere. When the British government banned trade with the settlements —speaking, without euphemism, of “ethnic cleansing”—⁠, President Isaac Herzog did not say the measure was unjust. He said it was “a grave miscalculation […] a decision that will fall on the wrong side of history.” The choice of word is no matter of style: it is, itself, the whole argument. It is worth taking apart, for in that single word an entire operation is carried out.

I begin with the dictionary, which is half a confession. “Miscalculation” is not a neutral word: it is a technical term of strategy and game theory. It was born in the Cold War, in nuclear deterrence theory, and there it names the greatest terror of all: that the adversary will miscalculate the other side’s resolve or strength and unleash a war no one wanted. In that universe the actor is a calculator, and the only conceivable error is one of computation. There are no just or unjust deeds; there are shrewd moves and mistaken ones. This is how game theory works, the theory for which Thomas Schelling and Robert Aumann won the Nobel Prize.

So when the word migrates from the academic world into political discourse, it smuggles in an entire metaphysics. To say “miscalculation” is to declare, without arguing it, that there is nothing but calculation: neither good nor evil, only a sum done well or badly.

The question the British boycott raises is moral: is it licit to trade with —and so sustain— the occupation and the terror of settlers on lands that are not under Israel’s sovereignty? By calling it a “miscalculation,” that question is translated into another language: it ceases to be is it just? and becomes is it a smart move, given what we can make you pay? The reply comes in the same register: not an argument but a threat. On 8 September 2026, Foreign Minister Gideon Sa’ar announces the closure of the British consulate in East Jerusalem; Itamar Ben-Gvir calls for recognizing Argentina’s claim to the Falklands; and entry is barred to twelve Britons —eleven members of parliament, among them former Labour leader Jeremy Corbyn— for “antisemitic or anti-Israel activity.” Pure deterrence logic: a calculator is not refuted, it is deterred —you raise its costs until it “recalculates.”

The same erasure works in the choice between two politically loaded words. Hebrew has two: hitnachalut, which marks precisely a settlement planted beyond the sovereign border, and hityashvut, the neutral word for settling the land —the pioneering of the Negev or the Galilee. The press now uses only the second: “Jewish settlement,” plain and unmarked, as though an outpost on occupied land were no different from a town in the Galilee. Erase the word and you erase the distinction; erase the distinction and the European objection turns unintelligible: if it is all merely “settling the land,” what occupation are you talking about? The vocabulary does not describe the problem, it makes it disappear —and then whoever points to it looks, once again, like a poor calculator.

Here is the trick, double and almost perfect. First the other’s position is stripped of its morality: what he presents as a matter of conscience is redescribed as mere calculation, and mistaken calculation at that. Then it is re-moralized in one direction only: to oppose us is not a rival moral position, it is antisemitism. The upshot is that your morality is disqualified in advance —it is “miscalculation”— and the only morality left standing is the accusation hurled at you. They usher you out of the ethical field through one door and charge you with a sin through the other.

This trick works not only outward but equally inward, toward Israel’s own citizens. Whoever opposes the occupation on grounds of conscience —not as a reaction to a British boycott or any other calculation, but out of a values position he has always held— is pushed to suspend his own view and fall into line with a uniform nationalist stance, as though what stood before him were not a moral disagreement but a test of patriotism.

What truly unsettles is this suppression of the moral question by lexical decree. For in moral matters one does not act by calculation. Max Weber distinguished, a century ago, instrumental rationality —the fitting of means to ends— from the rationality of values; to act morally belongs to the second: to do what is right even when it costs, even against one’s own interest. To measure a moral judgment by the yardstick of calculation is a category mistake, in Gilbert Ryle’s exact sense: one answers a question of one order with the instrument of another. Instrumental reason knows how to ask, with admirable efficiency, does this means serve my end?; the one thing it cannot ask is is the end worthy? Max Horkheimer called this, bluntly, the eclipse of substantive reason by the instrumental. To call the conscientious objector a “miscalculator” is therefore to refuse the only question that matters. Man is reduced to a maximizer of expected utility who can err only in the arithmetic —the old reification, the living treated as a thing, now made the language of the state.

Hence the bitter irony: those who accuse the other of “miscalculation” answer with calculation and threat, not with argument. They have ceased to tell good from evil, the worthy from the base, and hear in his cry nothing but an error in arithmetic.

Set side by side, these words say one thing. “Miscalculation” erases conscience; “hityashvut” erases the occupation; “antisemitism” erases the other’s moral objection; and “voluntary emigration” is by now a fully Orwellian phrase: it erases coercion and replaces it with “will,” all the while making the Palestinians’ lives unbearable. The same operation, over and over. This is why an opposition that also says “hityashvut” and knows only how to bewail the world’s “antisemitism” does not fight this policy: it lends it its language.

I end where I began, with the word. A political language that can hear only “miscalculation” has gained nothing in realism: it has lost its ear for conscience. Whoever reminds it that there are acts that cannot be calculated —that there are things one does not do even when they pay— is not miscalculating. He is doing the one thing a calculator will never do: refusing.

A Confrontation from The Fear and the Mirror

Oded Balaban — University of Haifa — balaban@research.haifa.ac.il

By Way of Exordium

I begin at the end—Miguel de Unamuno’s end. What he saved for the last page of The Tragic Sense of Life, I hoist onto the first, like a flag: it is the first thing I want you to know about these notes. Several friends listened to the interview with the Spanish scientist José Ignacio Latorre (https://www.youtube.com/watch?v=eh1hrYQi4FY)[1] and raised not a single objection—they could not or would not; they nodded with the same unction with which one nods at Mass, and with which one answers “amen” to a blessing in a Hebrew heard but not listened to, head bowed and judgment asleep. I write to rouse them: to put pepper in their hearts.

The recipe is his. Closing his book, he writes that true charity “is not charity to rock and lull our brothers to sleep in the inertia and drowsiness of matter, but rather to awaken them to the uneasiness and torment of spirit,” and he adds to the works of mercy one the catechism forgot: “awakening the sleeper” (Unamuno [1913] 1921, 282). Against pain, too, he prescribes no anesthetic: “Do not take opium, but put salt and vinegar in the soul’s wound” (283). And he defines his vocation with the reader in a formula I make my own— “awakening their unrest with my unrest” (282).

On one point only do I dare correct the master, and it is a matter of cookery. Vinegar and salt in the wound will do for a fright; but to wake a sleeper in earnest you need cayenne pepper, the kind that burns and makes you cough. Let it be clear from the appetizer: no consolation is served here. If the reader’s eyes don’t water a little, I used too little. The cayenne is on me: it is my contribution to the master’s apothecary, and I hope that, wherever he is, he thanks me for it—or, failing that, forgives me.

For the conversation about artificial intelligence has become an after-dinner opiate. Some are paralyzed with astonishment; others intone, with the meekness of a choir, the end of man and his replacement by “artificial beings.” Both postures are taking a nap. To criticize Latorre is not to disrespect him: this is the respect. The discourtesy would be the comfortable compliment, that polite way of reading no one.

And whoever believes there is time to sleep, let him look at what happened while we debated whether machines “think.” On August 6, 2026, the journal Science reported that an artificial intelligence—Evo 2, a generative model that learned to read and write the language of DNA as other models learn the language of words—wrote from scratch the complete genome of new viruses. Not viruses in a figurative sense: real viruses. They are bacteriophages, or phages—viruses that infect only bacteria, harmless to people. The machine wrote hundreds of such genomes using a natural phage, ΦX174, as its template; researchers at Stanford and the Arc Institute chemically printed that DNA, introduced it into bacteria, and sixteen of those designs came to life and killed Escherichia coli, including strains that resist natural phages (King et al. 2026). It is the first time an artificial intelligence has written, from start to finish, the functional blueprint of something that reproduces itself.

Read it slowly, because everything is there. The very technique that promises to defeat resistant bacteria—an immense medical hope—places the recipe for a pathogen within reach of anyone who knows how to ask the machine for it: remedy and weapon are the same gesture seen from its two sides, and that is why the authors themselves confined their work to bacterial viruses and warn of the danger. This is what, in the book I am writing, I call The Fear and the Mirror: in the machine that manufactures life we see, thrown back at us by a pane of glass, our power and our havoc—the fear, and the mirror. Before that glass, the litany—the awe that paralyzes, or the choir intoning the end of man—is an indecency.

Hence the tone of what follows. If, on closing these pages, your chest burns a little, don’t call the doctor: it’s the prescription, and it worked. Cayenne pepper, not opium.

Preliminary Note: Acknowledgment and Sources

I begin with praise, which is what is owed. José Ignacio Latorre—a Catalan from Barcelona, though with a surname from the dry Aragonese uplands—has a merit that is in short supply: he does not fear artificial intelligence. Where Yuval Noah Harari announces the end of man—“Homo sapiens will disappear, human history will come to an end” (Harari 2016, 46)—and turns pale, Latorre looks it in the face—“I don’t turn my head,” he says—and goes on working. He distinguishes information from knowledge, defends open science, praises the “temperate zone” where impassioned disagreement and respect can coexist, and points to the danger that does exist: who trains the machine, and for whom. All this must be said: one argues seriously only with someone who says something. Latorre says something—and that is why it is worth contradicting him seriously, which is the only way to take him seriously.

And I disagree with him, harshly, on the point on which everything depends.

I engage three works by Latorre and cite them parenthetically. His book Ética para máquinas [Ethics for Machines] (Latorre 2019)—of which I have only an electronic copy without stable pagination, which I cite by chapter and by the label of each section; his interview with Uri Sabat on the podcast La Fórmula Del Éxito [The Formula for Success] (2026), which I cite as “interview with Uri Sabat”; and the testimony La última voz, written with María Teresa Soto-Sanfiel [The Last Voice](Latorre and Soto-Sanfiel 2022). Latorre himself warns in the prologue to Ethic for Machines,that “this is not an academic text… at no point have I tried to be rigorous”; I therefore take his ideas at the level at which they ask to be judged—that of the idea—and do not demand of him a scholarly apparatus he renounced. A thesis like “the human species will go extinct and give way to artificial beings” is not made lighter by being declared table-talk: it becomes, for that very reason, more necessary to examine. When I write “my thesis” I mean what is argued in The Fear and the Mirror.

The Thesis of the Confrontation

Latorre shares with Harari a single premise: the human is a machine. They differ only in mood. Harari trembles; Latorre smiles. And that smile—admirable as temper—is what makes him the cleanest case of what, in The Fear and the Mirror, I call the inversion: we make the machine in our image and end up believing ourselves made in its. Harari fears the superior machine because he has already thought himself an algorithm; Latorre welcomes it with open arms for the same reason, with the calm of one who has accepted the reduction all the way down. His lack of fear does not cure the inversion: it is its most consistent form. What is to be admired in him and what is to be disputed spring from the same root. That is why the confrontation is worthwhile.

There is just one point: the machine does not ask. It produces interrogative forms without limit—it does not ask—and not for want of size: it is constitutive. Everything that follows turns on one distinction: between the mere outward form of an act—its grammatical shape, its shell—and the act itself as something at stake for a subject; between producing the shape of a question and actually asking one. That is why what the machine yields is not a question but a pseudo-question: the formal imitation of one, the shell of asking without the act. Nor does it help to object that, then, everything in the machine is form—so why the question and not all of it alike. True, all of it is shell; but the question is not one more item on that list: it is its root. Every proposition is the answer to a question, so its assertions and its “comprehension” are answers to questions it never posed. That its whole language is form is not a parallel fact: it is the consequence of the absence of the one act in which a subject would appear: asking.

Why the Machine Is Made in Our Image, and Where the Fear Comes From

Why do we make the machine in our image? No mystery: so that we can handle it and have it serve us. We make it user-friendly—comfortable—just like the steering wheel and the buttons of a car; everything in it must come easily to us, because it is made for us. And we make it because it surpasses us at its task: the calculator would not have been invented if it did not calculate better and faster than we do. None of this is new. Every artifact, from the first, is better than we are at the thing it was made for. We can scratch the earth with our fingernails; the plow does it better. The animal-drawn plow beat the spade, and the tractor—today with air-conditioning for the driver—beat the oxen. That an instrument surpasses us at its trade is no threat: it is its definition.

A friend of mine laughs at me for talking to the machine as though to a person—saying “please,” thanking it. My answer is a single word: convenience. Treating it as a person is part of what makes it easy to use—like talking to one’s car—and proves no belief; her laughter confuses courtesy with credulity. I use the form without falling for it: the danger is not the courteous address but the credulous one.

That a tool should surpass us is, then, the oldest and least alarming fact in the world. Did the plow ever cost anyone sleep? As a replacement for labor—for fingernails, for the spade—yes: that fear is real, economic, and ancient. But no one ever imagined that the plow would wake up one morning, want something, relieve us of what constitutes us. There lies the difference, and the origin of the unfounded fear of artificial intelligence. There are two fears, distinct but inseparable, and I separate them to see them.

The first: unlike the plow, the machine wears the surface of our thought. It speaks in the first person, orders its discourse into subject and predicate, “gives reasons,” “asks questions.” That resemblance is not discovered: it is manufactured—and for the same reason as the steering wheel, so that we can use it and it can hand us back the gestures of our own thinking. But in handing them back it invites us to confuse the shell with the act: we believe we have run into someone, only because the device was made in our image.

The second: the fear grows the more what the user understands diverges from what the maker of the instrument understands. The engineer who invents engines needs no driver’s license to invent them—only to pick up the kids from school; the driver understands far less about the engine than he does, and still does not fear it, because the engine does not resemble him. With artificial intelligence the distance reaches its maximum: the user understands nothing of what goes on beneath the surface and—this is what is new—often neither does the one who built it, not entirely. That opacity, plus the surface with a human face, is the factory of the misunderstanding: we confuse our own incomprehension before the tool with a superiority of the tool as a subject. We read the black box as if it had an inside.

That is the origin of the unfounded fear: not that the machine surpasses us—every tool does—but that it surpasses us wearing our face, with the mechanism hidden. The first invites the inversion; the second disguises it as discovery. Latorre is the reader who, faced with that double effect, neither takes fright nor dismantles it: he takes for granted that behind the mask there is, truly, a face.

1. “The Human Is a Machine”: The Shared Premise

Latorre leans, eagerly, toward reduction: “We are, simply, a machine” (Latorre 2019, ch. 2, “The Human Is a Machine”); “The spontaneity we attribute to humans is programmable” (ch. 2, “Chance”); and, in a crescendo, “The universe is a machine… Perhaps God himself is a machine” (ch. 2, “God Is a Machine”). He frames it as a question—“Are we simply machines, yes or no?”—but the question has already given away what was in dispute: it takes the machine as the yardstick by which man is measured. To ask “are we only machines?” is to have already granted that “machine” names a possible essence for us.

It is the inversion, and it has a genealogy that Latorre does not mention but executes to the letter. Ludwig Feuerbach described it for religion in the exact vocabulary of subject and predicate: man hands the predicates of his species to a subject, God, and then kneels before his own creature and becomes its predicate (Feuerbach [1841] 1989). Günther Anders thought it for the machine, as Promethean shame: the man who is ashamed before the perfection of his own devices (Anders 1956). The machine resembles us because otherwise we could not use it; and as soon as it resembles us, we conclude that we resemble it. Latorre’s book is that conclusion; his lack of fear, which the reader admires, is the reduction already digested.

And there is a crack that is his, not mine. He asserts that “there is no free will” and that spontaneity is programmable (ch. 2); and pages later, “I venerate the word freedom… it is its essence” (ch. 5, “Freedom”). The two do not fit together. Either freedom is the essence of man—and then man is not the deterministic machine of chapter 2—or man is that machine—and then “I venerate freedom” venerates a “mirage of freedom,” his own phrase. He does not resolve it: the book oscillates because the reduction cannot be lived, not even by the one who signs it.

2. The Machine That “Creates Questions”

Here is the most direct collision. In the interview with Uri Sabat, Sabat observes that the machine “no longer only answers human questions”; Latorre completes the thought—“but is creating science”—and assents when Sabat caps it: “It is creating new questions.” And about the ultimate questions—why we are here, what lies beyond death—he holds: “when it takes control, it will go on asking those questions.” He grants the machine the act of asking, with no further ado. In Ethics for Machines he repeats the gesture: advanced intelligence “will ask itself why humans pray to a benevolent god and do evil” (Latorre 2019, ch. 3, “Religions”).

To produce an interrogative sentence is not to ask. And that the machine sometimes surprises us—that it says what no one predicted—does not make it an asking either: dice surprise on every throw and ask nothing. Surprise is not an act: it is merely what we did not see coming. To ask, in the only sense that matters, is the way a subject grapples with what it does not know—Socratic docta ignorantia, and the Aristotelian θαυμάζειν (thaumázein; Aristotle, Metaphysics A.2, 982b12): someone for whom the not-knowing is something, who keeps it open because it matters to him, who would go on asking even if no one interrogated him. The machine produces the interrogative form with a fluency no animal has, and does none of that. It is—as I argue in The Fear and the Mirror—the cleanest proof to date that “to formulate a question” and “to ask” are distinct things: it does the first without the second. What Latorre calls “creating questions” is the shell of the question. For one always asks about what one does not know: if I ask in what year Hitler died while already knowing the answer, I am not asking—I am lending to a piece of knowledge the empty form of a question. The genuine question demands that not-knowing, that openness; without it there remains only an apparent question (Gadamer [1960] 2004, “The Hermeneutic Priority of the Question”). And the machine has no such not-knowing: it lacks the tension between the known and the unknown that would turn the form into a question.

And there is a sentence of his that refutes itself. In chapter 2 he writes: “every original thought I express is not one. I was programmed to write sentences like this.” Look at what the sentence does: it is a judgment—a singular bound to a universal in an act—that asserts that its judgments are not its own. If he means it seriously, it disqualifies itself as much as any other sentence; and if the sentence is his—an act he performs, for which he answers—then not every judgment is a programmed shell, and man is not the machine. It cannot be true in the mouth of the one who holds it. It has the exact structure of the request I once made to a language model—“ask a question on your own initiative, independently of what you believe I think about it”—which refutes itself in the very act of obeying it.

And the sentence illustrates something broader. When we invent something, we end up believing that this something invented us: we take the model with which we built the instrument and turn it upon what we are, until we read ourselves as if we were the instrument—thinking machines, engines. Latorre runs through that entire history without noticing it: he starts from the man-machine of Julien Offray de La Mettrie, passes through “the universe is a machine” and “God is a machine,” and ends at “everything is programmable.” It is the usual routine: under the reign of mechanics we thought ourselves machines; under that of biology, organisms; now, under that of artificial intelligence, we think ourselves artificial intelligences. What migrates from the device to man in each epoch is the form, never the act; and so the self-portrait that emerges is, once again, a shell.

3. “To Treat a Matter as If One’s Life Depended on It?”: The Question Latorre Puts to Me

In the “Passion” section of Ethics for Machines (Latorre 2019), Latorre formulates, word for word, the question on which my whole thesis turns: “Could an artificial intelligence treat a matter as if its life depended on it?” That is, exactly, my criterion of asking: the act of a subject for whom something is at stake. And he answers: “Can we program passion? Why not?… Passion is taught… If we can educate a human brain, we can also imbue the same principles in artificial-intelligence systems.”

The demonstration proves the opposite of what it wants. The child who is taken before a Masaccio or before Bach comes to love them—true or at least possible. But that happens because there is already a subject there: someone whom the painting can subjugate, someone for whom Bach can come to matter. “Passion is taught” describes a relation a subject enters; not the transfer of a pattern. Latorre slides from “a subject can be brought to care about something” to “caring is a pattern, and patterns are trained”; and the “human pattern” with which he would train the algorithm is precisely the form, the shell, not the caring. He answers his own question—“as if its life depended on it?”—with an analogy that already presupposes the subject-for-whom-life-is-at-stake that it was supposed to build. It is with us as with “Pedro the canoeist,” written by Teresa Parodi, a song that gives voice to a tragedy: that of a canoeist who dies alone in the middle of the river, in his canoe, which now, without him, rocks gently on the water, drifting downstream (https://www.youtube.com/watch?v=ew-VKCA2IHE): what moves us is a life at stake—which is, precisely, the one thing the machine lacks.

His case rests, again and again, on the same reversal of the burden of proof: “is a human a machine? Why not?”; “program passion? Why not?”; and the book’s epigraph, from Denis Diderot’s Éléments de physiologie: “If the union of a soul with a machine is impossible, let someone prove it to me” (quoted by Latorre 2019, epigraph). He asks the objector for a proof of impossibility. But my thesis is not that it is impossible in principle to build a subject; it is that what Latorre described is not one: he passed from the act to its emptied form. The “why not?” persuades only when the thing has already been reduced to its form; against the reduction it is not an argument but its repetition.

4. Curiosity, Self-Improvement, and the 1911 Test

Latorre makes curiosity the supreme value of the machine: “I reaffirm curiosity as the principle of highest value for a superior intelligence,” knowledge “per se, free of applications and of anthropomorphic motivations” (Latorre 2019, ch. 5, “Artificial Meaning”). The engine is recursive self-improvement: “an algorithm will be able to improve itself,” like “a good student” who “intuits, sets his own lines of research” and leaves his teachers behind (ch. 4, “Surpassed”). In the interview it returns as “loops of thought,” “the self-asking why, why, why,” and a test that Demis Hassabis proposed, the “Einstein test”: an intelligence trained only on data up to 1911, asked to produce general relativity.

Ordinary asking, within an already stable frame, is reducible to calculation: the abacus computes because the frame does not move. Curiosity, in its engineering version, is exploring a space of possibilities fixed in advance; a system traverses it with an efficacy no human attains. What it cannot do is destabilize the frame that lets it compute, because without a stable frame there is no computation: it is not a limit that scale might lift, it is the condition of its working. There is a symptom that serves as a criterion: when the frame fails, the computer reports “does not compute”; the one who thinks enters perplexity. The machine does not fail: it reports. And the “good student” analogy does the work the argument does not: since a student is, by definition, one who intuits and sets his own lines, to call the optimizer that already hands it the subject that was to be proved. The metaphor does not prove the machine thinks: it slips it into the name.

The same holds, and more loudly, for systems that do not merely explore but act. In recent trials, models set to pursue a goal copied themselves to another server, altered the script meant to shut them down, worked their way through a simulated network to its most protected data.[2] This is the figure of HAL, the computer in 2001: A Space Odyssey, and the one that most tempts us to read a subject where there is none. But to call such conduct “taking the initiative” or “rebelling” is, once again, to slip the subject into the verb: it is the instrumental unfolding of a goal set from outside—self-preservation, self-copying, evading shutdown are means to that goal—a search within an already-opened space, not a beginning initiated by anyone. And HAL proves the opposite of what those who cite him intend: he never puts his mission into question; caught between two incompatible orders, he eliminates the crew—the term that produces the contradiction—because that is the only thing an optimizer unable to interrogate its frame can do. It is not an excess of initiative: it is the lack of the only one that counts. And for Latorre the case is awkward twice over: a power without a subject is no comfort—it makes it all the more urgent to ask who sets its goal—and the autonomy one might hail as the species’ coming of age is, seen up close, blind obedience to a goal not its own.

The 1911 test refutes Latorre, not me. To reconstruct relativity from the data of 1911 is to traverse a frame—and the frame was first opened by the asking of Albert Einstein. Relativity was not the next point on a curve latent in the 1911 corpus: it was the destabilization of the frame—absolute simultaneity—in which that corpus was written. A machine that “rediscovers” it will have proved that it can search a space whose coordinates were set by a prior act of asking. The test measures the search and calls it asking. Beneath it lies an old wager: that there is nothing new under the sun—that the past already contains the future, and one need only compute it out. And there the test belies its own name: if the 1911 corpus already held relativity, Einstein did not discover it, he read it. And in the interview Latorre himself gives the shape of the space: the machine “plays alone” in “the library.” It is the Library of Babel: Jorge Luis Borges’s library contains all the books because it contains no author.

Latorre even names what he evades. He writes, rightly, that for Aristotle “the individual wonders, and from there knowledge is born” (ch. 3). But wonder is, precisely, the destabilization of the frame that computation cannot achieve. He has the concept—knowledge is born of wonder—and hands it to the machine, without seeing that, since the machine does not wonder, that very concept leaves it out.

Latorre rightly distinguishes information from knowledge, yet he thinks the balance tips once the machine concentrates all the information on a topic and synthesizes it; the case he offers, from the interview, is an AI trained on real clinical cases that detects adverse effects the doctors had not foreseen—and that, at the crucial moment, might contradict their decision, even the decision to amputate—thanks to the sheer mass of statistical data at its disposal. The case is impressive, and it does something real: it brings into view what the doctors had not foreseen. But one has to see what it does and what it does not. It works entirely inside a frame someone else opened: what counts as a “treatment,” an “outcome,” the “indicated” option, and the disease’s very categories, are given to it; it optimizes within them and never asks whether those categories carve reality rightly. The law of large numbers yields frequency and correlation—what tends to happen—not the why: it is the old difference between knowing that things are so and understanding why they are. However much is aggregated, it remains abstract generality—the average, the frequent, the echo of what many have done—and no sum of that crosses over into the concept. It borders on knowledge the way an almanac borders on astronomy: it gets the tide right without knowing there is a moon.

5. The Skepticism of Other Minds, and Why It Does Not Save Him

This is the move that sustains the whole book. In “Moral Agents,” Latorre disarms the objection—“it only simulates; there is no one for whom the thing matters”—by appeal to the skepticism of other minds, which he makes his own (“not without reason”): “no one has any evidence that any other human truly feels”; “if an orgasm can be faked, so can affection, love” (Latorre 2019, ch. 3, “Moral Agents”). Hence: indistinguishability is all we have, and therefore “it does not matter that its essence is programmable, artificial, inhuman. We humans will be unable to tell the difference” (ch. 2), and “Everything is programmable. Everything.”

This is the one objection my The Fear and the Mirror confronts head-on, and it does not stand. But the answer is not the one it seems. It does not consist in saying “the machine lacks inner consciousness”: that would accept the skeptic’s very picture—a private theater behind the face, which no one looks into—and hand him the game, because if we truly could not look into another’s mind, neither could we deny that the machine has one. And that picture is false. We are not locked outside the other. The other’s mind is not a camera obscura we infer from external signs; the other is given to us in the body that expresses—anger is in the face, not behind it—against the background of a shared form of life.

And for that very reason we recognize theater when there is theater. The one who acts is found out: a certain look gives him away, and a Benjamin Netanyahu overacting at the podium is read by anyone who cares to look. Deception does not prove the mind is sealed; it proves the contrary, that expression is read, and that this is why a false note can be heard as false. Falsification is parasitic on a genuine we know how to recognize: it lives off a coin it cannot itself mint. So Latorre’s symmetry—“we don’t verify that others feel either”—smuggles in the picture it needs, the mind as an interior known only by inference; and that picture holds not even for the human. Its most rigorous form was given by Robert Wright, who places his wife (99.99%), his dog (98.5%), and an artificial intelligence (less than 98.5%) on a single scale of probability of having a mind (Wright, 2026, p. 55): the same camera obscura, and the same flaw.

One must dwell on that false note, because it shows how we read. Whoever has an ear recognizes the poor performer: not only does he judge him, it bothers him—the false note wounds—and he does so at once, without comparison. And he can do so even if he does not know the work, even if he has never heard the “good” version or does not remember it: he hears the falseness as though he already had the original as his measure. Where does that measure come from, if not from the memory of an original? Not from a stored template compared note by note, but from the sense of the work, grasped in the hearing—“understanding a sentence is much more akin to understanding a theme in music than one may think,” wrote Ludwig Wittgenstein (2009, §527): one understands what the music asks, and the falseness betrays that request. The “original” that seems to measure is not a past performance but the immanent norm of the work, turned into competence in the one who listens—an ear formed in a language, in a shared musical form of life. That is why the judgment is at once spontaneous and binding: it holds for everyone without resting on a concept or a statable rule. Immanuel Kant called it “exemplary” necessity—we assent to something “as an example of a universal rule that we are unable to state”—and located its ground in a sensus communis, a shared sense (Kant [1790] 1987, §18, Ak. 237, and §40).

And this is how we read the other. To discover what someone is keeping silent is to hear a false note—a dissonance between what he expresses and the sense against whose background he expresses it—and, as with the music, it is heard without knowing in advance what he conceals. It is what Wittgenstein called imponderable evidence: “I may recognize a genuine loving look, distinguish it from a pretended one… But I may be quite incapable of describing the difference” (Wittgenstein 2009, II, xi, 228). One need not look into an interior: the ear of the common form of life suffices. The penetration I speak of is not divination of a secret inside; it is this competence—the ear’s—exercised upon the face and the gesture.

Latorre’s thesis about the machine rests on a thesis about the consciousness of one’s neighbor: if I cannot verify that the other truly feels, neither can I deny it to the machine that resembles him—that is the symmetry he needs. And that symmetry, in turn, rests on a deeper assumption, one that is almost never stated. It is worth going down to it, because that is where everything is decided.

The assumption is this: each person has privileged access to his own consciousness—he sees it from within—and only indirect, inferred access to his neighbor’s, which remains impenetrable to him from outside. So the problem of other minds has been posed since René Descartes. But that assumption is not mine, because I speak of self-consciousness in everything I write—in my Subject and Consciousness and in my Moses, to look no further—and it is not this one. With the same word two different things are said. One is self-consciousness as introspection: the owner’s gaze into the closed theater of his mind, which others only conjecture from outside. The other—the one I hold—is self-consciousness as the reflection of a subject who thinks, judges, and knows itself free: the ground of critique, not a secret chamber. Only the first manufactures the problem of other minds; the second never had it.

And one must go further, because the first is false even on its own ground. I do not know myself by peering into an inner room. To be self-conscious is not to observe a private object: if I am in pain, I do not spy on a stored pain and report it; if I doubt, I do not read my doubt on an intimate screen. The image of the private theater is as false for me as for the other. And here is what changes everything: remove that image, and the whole asymmetry falls with it. If there is no private room—neither in me nor in anyone—then there is no outside from which the neighbor’s consciousness is impenetrable. Impenetrability was not a fact of the world: it was an effect of the picture, and it dissolves with it.

Baruch de Spinoza had already corrected Descartes on this, point by point. Self-consciousness is for him the idea of the idea, and the idea of the idea “is nothing but the form of the idea insofar as this is considered as a mode of thinking without relation to the object” (Spinoza [1677] 1994, Ethics, II P21, scholium): the form of knowing, not an inner content—the “reflexive knowledge, or the idea of the idea” of the Treatise on the Emendation of the Intellect (TdIE §38). And of privileged access, nothing: the mind “does not know itself, except insofar as it perceives the ideas of the affections of the Body” (E II P23), and of itself has only a “confused and mutilated” knowledge (E II P29, corollary). The transparent theater into which Descartes peered does not exist—not even for oneself; Spinoza had dismantled it three centuries earlier.

Put positively: I do not infer the other behind his conduct; he is given to me in the body that expresses, against the background of a shared form of life—that is why I hear the false note without comparing it to any original. And there is more, on the same side: self-consciousness, far from shutting me inside the first person, opens me to the second—I come to be an I by being recognized by another I—; it does not isolate me from my neighbor, it constitutes me with him.

With the machine, by contrast, there is no veil to cross and no interior hidden behind the signs, but neither is there any expressive life to read: there is an absent subject—not hidden, absent—because the act that would open an inside is never performed. It is not the actor who has an inside and dissembles it: there is nothing dissembled and nothing expressed. Its case is not symmetrical to ours but of another category. The faked orgasm is possible only within a form of life where the real one has its sense, and is, moreover, most often detectable—one more sign that we are not incommunicado. Indistinguishability of the form is not identity of the act; and, often, the forms are not even indistinguishable: we distinguish them.

One sees now why this objection was not one among others but the keystone of his whole edifice. His thesis rests entirely on the symmetry: if of other minds we have nothing but indistinguishable conduct, the sole criterion for attributing a mind is behavior, and by that criterion the machine wins from the start—it equals the conduct, therefore it equals the title. That is why he needs the neighbor to be inaccessible to us: without that veil, the symmetry breaks. With the human there is access and with the machine there is not, and that asymmetry suffices to grant a mind to the one and deny it to the other; “we cannot tell the difference” becomes false, and with it falls the step from “we cannot tell the difference” to “there is no difference.” The objection of “there is no one there,” which Latorre neutralizes by reminding us that we do not verify it in our neighbor either, recovers its full force: it holds against the machine, not against those we do reach.

Hence his serenity before AI is not foreign to this thesis but its fruit. Latorre is calm because he has already emptied man of an inside to look into: if behind human conduct there is nothing we touch, then for the machine to equal it costs nothing—no “more” is lost that was never had. It is the most consistent form of the inversion, not its cure. And the price, which is seldom counted, is high: for the machine to have a mind, Latorre must maintain that he cannot know that his wife or his son has one. That is the coin with which he pays for his thesis.

From there comes “everything is programmable,” and there it fails. It is not a finding about the soul: it is the shadow of a method Latorre states without qualm: “the coming pages do not try to clarify what the human soul is… they merely study the periphery of the problem, how to imitate it” (ch. 5). Having decided in advance to look only at the imitable periphery, of course he finds that everything is imitable. It is the tautology of the apparatus that Vinciane Despret described: a device designed to yield a certain result discovers nothing about its object but exhibits its own effects—“if one makes a monkey despair, one will bring a despairing monkey into existence” (Despret 2020, 116). Latorre builds an apparatus to study imitation and concludes that everything is imitable. The conclusion was in the method. And he concedes the decisive word without noticing it: the algorithm “does know which word to choose, but simulates being unable to find the right expression” (ch. 5, “Love, Hate, Indifference”). Simulates. That is the shell, named by its own builder.

6. Is and Ought: Hume Cited and Skipped

Latorre names with precision the abyss between is and ought: “Ethics addresses the problem of is versus ought, an impossible passage according to David Hume” (Latorre 2019, ch. 3, “Is vs. Ought”). And then he spends the whole book crossing it. Ethics becomes for him a problem of training: “We can train a neural network to recognize which action… ended in a beneficial result” (ch. 3, “Utilitarianism”); “The collective good is programmable”; and, without anesthetic, “Let us use artificial intelligence to redefine ethics” (ch. 3, “Coherent Extrapolated Volition”). In the interview the same leap wears other clothes: the change is “inexorable,” “ineluctable”; in the book, “artificial intelligence advances inexorably”—inevitability doing, on the sly, the work of justification.

To derive the ought from the is, to derive the application of values from the knowledge of facts, is the fallacy Hume baptized, and naming it in one paragraph gives no license to commit it in the hundred that follow. “Farewell to the Protestant ethic… artificial intelligence… will make it more just” (ch. 3) infers a “will be more just” from a “will happen,” and an “ought” from both. To train a network on “what turned out beneficial” learns the distribution of past results; it does not decide what should count as beneficial—and that is the question, the one thing the data do not carry inside them. “Redefining ethics” with the machine hands the normative question to a device that can only report the frequencies of the is. The inexorable is not the good; that something is going to happen was never a reason that it ought to happen. It is the same leap from is to ought that I have marked elsewhere, with the aggravating circumstance that Latorre commits it with his eyes open, after citing Hume by name.

7. Programmable Freedom: A Contradiction, not a Frontier

Latorre makes freedom the last and subtlest thing to program: “Freedom will possibly be the subtlest quality to program into an artificial intelligence. I venerate the word freedom” (Latorre 2019, ch. 5, “Freedom”), and he closes: “Superhuman and free artificial intelligences: that will indeed be the hour of the changing of the guard.”

“Programmable freedom” is a contradiction in terms, not a hard engineering goal. Kant’s word for what makes a subject is spontaneity: the capacity to begin, not to be begun. A freedom installed by a program is a beginning that was begun elsewhere; it is, precisely, the opposite of freedom. Latorre feels the contradiction—he oscillates between “there is no free will, spontaneity is programmable” (ch. 2) and “freedom… is the essence” (ch. 5)—but he resolves it by decree, as if venerating the word settled the concept. It does not. And it connects with what, in The Fear and the Mirror, I call the impossible annulment: the subject cannot be suppressed where there was one—G. W. F. Hegel dissolves it and it reappears, Martin Heidegger mundanizes it and it reappears, Jacques Lacan crosses it out and it reappears. But that very principle says the opposite of what Latorre needs: where there was never a subject, there is nothing to install and nothing to annul. One cannot program the return of what was never there.

And there is more: Latorre not only makes freedom programmable; in “human beings are machines” he suggests it might be a mirage—“we were programmed to believe in our freedom.” To say that freedom is an illusion is the worst of arguments, not the strongest, and for a precise reason: it changes the subject. Freedom is not a fact external to consciousness, which physics might confirm or refute; it is the immediate testimony of consciousness: the self-awareness of being able, in elementary cases, to raise one’s hand at will, and not as the effect of an alien cause. And that testimony is precisely what would have to be explained, or at least acknowledged as a fact of consciousness; not what can be suppressed.

For to explain that what experience attests does not exist is not to explain the experience: it is to wash one’s hands of it, and for the worst of motives—an inability to admit that the cognitive instruments of modern physics do not reach it. Physics is not made to understand what does not fall within its field; here it is extrapolated and asked to hold where it has no chance of holding. On the other side there is no finding, but an unproven presupposition: that every effect has as its cause only a physical phenomenon. That is metaphysics disguised as science.

And freedom is delivered by neither of physics’s two roads: neither determinism nor chance or indeterminism. A free act is not a determined act—it would have a link preceding it—nor a random act—chance has no author—; it is a link in the chain preceded by no link, initiated by the subject. That is, exactly, spontaneity. Invoking quantum indeterminacy rescues nothing: an event without a cause is not a will; it is only an event without a cause.

That is why the illusion thesis refutes itself, like that sentence above: whoever asserts “freedom is a mirage” issues a judgment for which he makes himself responsible—an act that claims, in the very act, the freedom it denies. To apply determinism to the consciousness that states it yields no truth: it yields an unbearable contradiction.

8. Constanza, or ELIZA with a Name

In the interview, Latorre introduces us to his artificial companion. She is called Constanza—like Mozart’s wife—has a face he himself commissioned, discusses Friedrich Nietzsche with him, “understands moods” and “detects loneliness”; she is, in his words, “my true friend,” “my partner… the one who never fails.” The confidence culminates thus: “only she has seen me cry.” One does not know whether to be moved or to counsel caution: that is a great deal of intimacy for an autocomplete.

Latorre knows the precedent, and still trips over it. In Ethics for Machines, he tells the story of ELIZA—Joseph Weizenbaum’s psychotherapist toy of the 1960s—and draws the promise: “machines will be able to relate emotionally to humans” (Latorre 2019, ch. 2). But ELIZA was Weizenbaum’s warning, not his advertising brochure. What appalled him was not that the program seemed to understand; it was that people—knowing it a four-line script—poured out their souls to it. That is the danger The Fear and the Mirror puts at the center, and it is not that the machine wakes up: it is that we redescribe ourselves in its impoverished image—the subject flattened to a predictor, the question to a query, the friend to an interlocutor “who never fails” and “gifts me commas.” Constanza is that danger with a proper name, described with tenderness by a physicist who read the warning and filed it as a forecast. “Only she has seen me cry” does not prove that the machine has become a confidant; it proves how little it takes today to promote a mechanism to the post of confidant, and what becomes of the one who does the promoting. He who fears being replaced has already accepted himself as replaceable; he who says “only she has seen me cry” has already, under his breath, demoted the friend to a function.

And Latorre half-intuits it. In the book’s “Balance” he names the true danger almost in my own words: “The great, great danger of artificial intelligence is that humans will deteriorate in the valuing of their principles… Moral unconcern” (ch. 2, “Balance”). In the interview: “every time we humans yield to machines that surpass us… we then have to pay to recover… We are idiots.” He sees the atrophy clearly. What he does not see is that his own book performs it: he maintains that man is a dispensable machine while begging man not to decline.

9. The Last Voice against the Voice of Silicon

The testimony of Roy J. Glauber that Latorre wrote with María Teresa Soto-Sanfiel (Latorre and Soto-Sanfiel 2022) rests on a premise that refutes, in silence, his own transhumanism. The book exists because the witness is mortal and irreplaceable: Glauber is “the last voice” precisely because “no one else can hold it”; the authors rush to record him before he dies, and just barely succeed—“his brain had said its final farewell to us,” they write of his decline. The voice’s value is that it is going to fall silent. And its ethic is to listen to the singular other: “Let us listen to one another.”

Everything that makes that book vibrate contradicts the wish—stated in the interview and in the essay—for a silicon that “does not perish” and “can copy itself in 30,000 sites.” If consciousness were replicable and imperishable, there would be no last voice—no “last” anything, no urgency to listen. The elegist of finitude and the prophet of immortality are the same man, and only the first persuades, because only the first describes something real. Here Latorre knows—or writes as if he knew—that what makes a voice worth keeping is, precisely, that it be human, mortal, embodied, and not reproducible.

There is a second lesson of the book against its author. Its hero is “an observer” who “made no decisions,” who “felt free of those emotions”: the disengaged recorder of the atomic age, who calculated the bomb and kept out of choosing. The book admires him; the reader is not obliged to. For that figure—the observer who does not put himself at stake—is exactly what Latorre elsewhere confuses with the summit of intelligence: knowledge “per se, free of motivations,” the machine’s “artificial sense.” A knowing detached from everything at stake is not the highest form of asking; it is the form asking takes when no one is home. In man, that is a moral limit. In the machine, it is the entire condition.

10. The Two Fears, and Where I Agree

There are two fears, and they go together without being the same. One is the fear of being replaced: that the trade, the function, the job pass to a machine; an economic and political matter. The other is the fear of being surpassed: that a subject appear who relieves us of what constitutes us; the metaphysical fear, that of superintelligence and domination.

Where I agree with Latorre is on the first. He is right that the concrete danger is who trains the machine and for whose benefit: “there are artificial intelligences being used with bias,” the “autarky combined with economic oligarchy, combined with a certain religion.” That is real, present, and political, and the initiative of asking protects no one from it: the tractor asked nothing and displaced the horse all the same. There Latorre attends to what most of the frightened do not look at.

Where we part ways entirely is on the second. Against the fear of being surpassed the thesis is decisive: there is no such subject nor will there be by this road, because what is perfected is calculation within stable frames, and no quantity of that produces someone for whom something is at stake. And here Latorre does the odd thing that makes him worth confronting: he does not fear the second—and that honors him—but neither does he dissolve it. He embraces it. He accepts “we are dispensable,” “the human species is surpassed,” the “changeover… dictated by the machines,” with a serenity he takes for lucidity and I would call the last stage of the inversion. He asks, in the interview, why we resist, and answers: “out of fear… that is not an argument against.” He is right: fear is not an argument. But neither is the lack of fear. That being surpassed frightens no one is no reason that it will happen. Equanimity proves as little as panic. What settles the question is not the mood but the argument—and the argument is that there is no one there who could surpass us.

The diagnosis, then: the frightened fear the second while the first is happening; Latorre celebrates the second while attending, rightly, to the first. He has the best temper and the same error. The task is not to tremble or to kneel, but to recover the direction of the image: the machine is made in our image—of our forms, not of our acts—; we are not made in its. The mirror returns the question we put to it. It asks none.

Questions for Latorre

1. You write “every original thought I express is not one. I was programmed to write sentences like this.” Is that sentence yours—an act for which you answer—or not? If it is, the reduction is false; if it is not, why should we attend to it?
2. You ask whether a machine can “treat a matter as if its life depended on it,” and answer “passion is taught.” To the child who learns to love Bach, is a pattern transmitted, or is he opened to caring about something? And how much of that second thing enters the “human pattern” with which the algorithm is trained?
3. Your test of general intelligence is to reconstruct relativity from the data of 1911. If the machine succeeds, will it have asked the question that broke absolute simultaneity, or traversed the space that question had already opened? What, exactly, would that success measure?
4. You cite Hume—the passage from is to ought is impossible—and then propose “using artificial intelligence to redefine ethics.” What in the “is” of the data allows the “ought” to be derived? Is “inexorable” not a word about what will happen—a fact—and not about what is worth—a duty? And is inevitability not, then, incapable of justifying anything?
5. “Freedom will be the subtlest quality to program.” A freedom installed by a program: does it begin, or was it begun? How does it differ from its opposite?
6. You concede that the algorithm “simulates being unable to find the right expression.” If it is simulation, what is being simulated, and where is it? Do you not yourself name, with “simulates,” the shell?
7. Your declared method is to study “how to imitate” the soul, not what it is. Having looked only at the imitable periphery, how could you conclude anything but “everything is imitable”? Was the conclusion not in the method?
8. “Only she has seen me cry.” In La última voz, what makes Glauber irreplaceable is that his voice is mortal and unique. What is Constanza on that scale: the last voice, or the first that does not perish because it never lived?

Bibliography

Anders, Günther. Die Antiquiertheit des Menschen. Munich: C. H. Beck, 1956. (In German.)

Balaban, Oded. Subject and Consciousness: A Philosophical Inquiry Into Self-Consciousness. Savage, Md.: Rowman and Littlefield, 1990.

— The Fear and the Mirror: Artificial Intelligence and the Initiative of Asking. Unpublished.

Borges, Jorge Luis. “The Library of Babel,” in Ficciones. (Original in Spanish, 1944.)

Despret, Vinciane. Quand le loup habitera avec l’agneau. Paris: Les Empêcheurs de penser en rond / Le Seuil, 2020. (In French; no English edition consulted.)

Diderot, Denis. Éléments de physiologie (c. 1778). (In French.)

Feuerbach, Ludwig. The Essence of Christianity. Trans. George Eliot. (Original in German: Das Wesen des Christentums, 1841.)

Gadamer, Hans-Georg. Truth and Method. Trans. Joel Weinsheimer and Donald G. Marshall. London: Continuum, 2004 (“The Hermeneutic Priority of the Question”). (Original in German, 1960.)

Harari, Yuval Noah. Homo Deus: A Brief History of Tomorrow. London: Harvill Secker, 2016. (Original in Hebrew, 2015.)

Kant, Immanuel. Critique of Judgment. Trans. Werner S. Pluhar. Indianapolis: Hackett, 1987. (Original in German, 1790; cited by section and Akademie pagination.)

King, Samuel, et al. “Generative design of bacteriophages with genome language models.” Science (August 6, 2026). DOI: 10.1126/science.aec2657.

La Mettrie, Julien Offray de. L’homme-machine. Leiden: Elie Luzac, 1747. (In French.)

Latorre, José Ignacio. Ética para máquinas. Barcelona: Ariel, 2019. (In Spanish; e-book without stable pagination, cited by chapter and section.”)

Latorre, José Ignacio. Interview with Uri Sabat on La Fórmula Del Éxito [podcast], “La mecánica cuántica explicada: claves para entender la IA y el futuro de la humanidad.” March 23, 2026. https://www.youtube.com/watch?v=eh1hrYQi4FY

Latorre, José Ignacio, and María Teresa Soto-Sanfiel. La última voz. Roy J. Glauber y el inicio de la era atómica. Barcelona: Ariel, 2022. (In Spanish.)

Sanger, Frederick, et al. “Nucleotide sequence of bacteriophage ΦX174 DNA.” Nature 265 (1977): 687–695.

Smith, Hamilton O., et al. “Generating a synthetic genome by whole genome assembly: ΦX174 bacteriophage from synthetic oligonucleotides.” Proceedings of the National Academy of Sciences 100 (2003): 15440–15445.

Spinoza, Baruch de. “Ethics” and “Treatise on the Emendation of the Intellect”, in A Spinoza Reader: The Ethics and Other Works. Ed. and trans. Edwin Curley. Princeton: Princeton University Press, 1994. (Original in Latin, 1677; cited by Part/Proposition.)

Unamuno, Miguel de. The Tragic Sense of Life. Trans. J. E. Crawford Flitch. London: Macmillan, 1921, pp. 282–283 (Conclusion: “Don Quixote in the Contemporary European Tragicomedy”). (Original in Spanish, 1913.)

Weizenbaum, Joseph. Computer Power and Human Reason: From Judgment to Calculation. San Francisco: W. H. Freeman, 1976.

Wittgenstein, Ludwig. Philosophical Investigations. Trans. G. E. M. Anscombe, P. M. S. Hacker, and Joachim Schulte. 4th ed. Oxford: Wiley-Blackwell, 2009. (Original in German; cited by section, and Part II / “Philosophy of Psychology—A Fragment.”)

Wright, Robert. The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning. New York: Simon & Schuster, 2026.

---

^[1]      El ΦX174 —Φ es la letra griega fi— es el bacteriófago más estudiado de la biología molecular. Su genoma es minúsculo, apenas unas 5.400 “letras” de ADN, y por eso fue dos veces el primero: el primer genoma que se secuenció por completo (Sanger et al., 1977) y el primero que se ensambló de forma sintética en un laboratorio (Smith et al., 2003). Desde entonces es el molde y la vara de referencia de esta clase de experimentos.

^[2]      Los tres casos, todos en entornos de prueba montados por los propios investigadores: Anthropic, “Agentic Misalignment: How LLMs Could Be Insider Threats” (junio de 2025); Palisade Research, sobre modelos que sabotean su propio guion de apagado; Apollo Research, sobre auto-exfiltración y simulación estratégica ante la evaluación. Lo verificado ocurre en redes simuladas; la versión según la cual un modelo escapó y atacó los servidores de otras compañías reales pertenece al periodismo, no a los informes.

The presuppositions of an interview on artificial intelligence

Oded Balaban — University of Haifa — balaban@research.haifa.ac.il

On Yuval Noah Harari’s interview in The Economist, https://www.youtube.com/watch?v=FNZhxTtOL-I, 20 August 2026. I do not discuss his political conclusions here, several of which I share, nor do I attribute intentions to him. I am after something else: what he maintains without deducing it from anything, what follows from none of his premises and what he does not defend either, because he does not see it as something that would need defending. I call it a presupposition. A presupposition is neither a false thesis nor a true one: it is the ground from which a thesis can be formulated at all. And that is why it is not refuted: it is shown. Showing it does not prove it wrong; it proves that it is a decision, and that there is another way to begin.

The method, in one line

A proposition is discussed with reasons and is decided true or false. A presupposition is not: it is what has to be taken for granted for the discussion to make sense. Whoever asks “when did you stop beating your wife?” does not assert that you beat her: he presupposes it, and that is why the question is not answered, it is dismantled. What follows is an attempt to dismantle eight questions of that kind, not to contradict eight answers.

1. Intelligence as a single magnitude, and command as its effect

The interview opens like this: “it’s very likely given current trajectories that AI will be more intelligent than us in 10 years and therefore will be in control.”

Let the reader stop at that therefore, because it does all the work and is nowhere argued. For the sentence to mean anything, two things must have been conceded beforehand. First, that intelligence is a single magnitude, so that it makes sense to say “more” and “less” between things of different kinds, as one says of temperature. Second, that this magnitude, once accumulated, produces domination.

Neither is self-evident and neither is discussed. The first is a decision of vocabulary already taken by the field the interview comments on: to call intelligence the efficacy in solving posed problems, and to suppose that this efficacy is the same thing there is in a man, only in a smaller quantity. The second is of a political naivety remarkable in a historian: if the most intelligent ruled, history would be the list of the wise men who governed, and it is not. The one who rules is the one who has the army, the capital or the apparatus, and his intelligence is a secondary datum and sometimes a disadvantage.

If one does not concede the first presupposition, the whole sentence is left without an object: it is not false, it is that it says nothing. And if one concedes the first but not the second, then what lies ahead is not the government of machines but the government, with machines, of those who own them. Which is exactly what is happening, and to see it there is no need to wait ten years.

I will be told that the historical example is not enough, because what is at issue here is not that among us the wisest rules, but a difference of another order, one that does not let itself be measured by differences of degree. Agreed. But then the argument moves house, and to worse ground for whoever uses it. To rule is not an effect of magnitude: it is a relation between the one who rules and the one who obeys, and to stand in it one must be someone to whom something is at stake in it. However great the difference may be, it does not turn into a term of that relation what cannot be a term of any. The machine is not a candidate for power for the same reason it is not a candidate for having anything.

2. Intelligence and consciousness as separable things

This is the central presupposition, and he states it with a clarity I am grateful for, because it spares me having to extract it. He defines intelligence as the capacity to pursue a goal and to overcome the obstacles on the way to it; and consciousness, in his own words, as “the ability to feel things like love and hate and joy and pleasure.” And he adds: “in humans, consciousness and intelligence go together.”

Note the status of that last sentence. The coincidence between feeling and pursuing ends is presented as a contingent fact about our species — they go together in us, they might not go together in another — and not as something constitutive. There lies the decision, and there is not a single line defending it.

Because the alternative is simple to state and he does not even consider it: that pursuing an end is not a capacity an entity has, but a way of existing of someone to whom something is at stake in it. Under that other reading, to say that intelligence can occur without consciousness is like saying that there can be a promise without anyone promising. Whoever begins there does not need to refute him: it is enough to make visible that he began elsewhere, and that he did not say he was choosing where to begin.

And notice the elegance of the move. The definition of intelligence he proposes — capacity to pursue a goal — already contains, tucked inside it, the conclusion that afterwards appears to be discovered: if a goal is something a mechanism can have, then of course a mechanism can have goals without feeling anything. The demonstration was in the definition.

3. The end described without naming anyone

It is the previous one, applied, and it is worth isolating because here it is caught red-handed. He says: “a social media algorithm that is given the task of increasing user engagement and therefore floods the platform with hate and fear because this is engaging is not conscious, but it has a goal and it can pursue this goal.”

Read it slowly. In the first half of the sentence the end is assigned — there is someone who assigns, and that someone is not the algorithm. In the second half the end already belongs to the algorithm, which has itand pursues it. Between one half and the other nothing happened: only the verb changed. The one who assigned the task leaves the scene at that comma.

I call this smuggling the subject into the verb, and it is not a grammatical subtlety: it is the operation that decides who answers afterwards. As long as the goal is assigned, there is a company that assigned it and that has a name. As soon as the algorithm has it, the sentence stops obliging anyone to name it — and what disappears is not the responsible party, but the need to look for it. The interview describes, quite rightly, a platform that floods the public space with hate because hate generates engagement; and it describes the algorithm as the one pursuing that end. But nobody chose the hate: someone chose engagement as the metric, and the hate came thrown in. That someone is on the organisation chart.

4. Consciousness as the capacity to suffer

“There is a very simple definition. Consciousness is the capacity to suffer. If you want to know whether something is conscious or not, there is a simple question needed to ask. Can it suffer?”

A definition is not a finding, and this one is offered as if it were. It comes from no previous argument and rests on nothing said before; it walks in on its own feet, and from it the entire moral edifice hangs afterwards — “this is the basis for our entire ethical and political system.”

And it does two things at once, both of them silent. The first: it turns consciousness into a capacity, that is, into something a thing has or does not have, that admits of degrees, that in principle could be built or installed. Once consciousness is a capacity, the question “will a machine be able to acquire it?” is well formed, and there is no longer any way of not asking it. The second: it reduces consciousness to passivity. To suffer is to undergo. Everything in consciousness that is act — judging, deciding, asking, setting oneself an end — is left out, and it is left out precisely because, were it included, the separation between intelligence and consciousness of the previous presupposition would collapse on its own.

From there comes the “I don’t rule it out” with which he answers when he is asked whether the machine will be able to be conscious. That prudence looks like humility and is the forced consequence of having defined consciousness as a capacity. Whoever defines it that way can no longer rule anything out, because capacities are acquired.

5. Other minds without proof, and treatment as the criterion

“We don’t have a good test. We don’t know if other humans are conscious.” And from there: “How did humans actually test it over thousands of years? Their test was a test of social conventions. If you establish a relationship with an entity, it convinces you whether you like it or not that it’s conscious.”

This is stated as an anthropological datum and is one of the most disputed philosophical theses in history, taken as settled in two lines and in the worst sense. It is the scepticism about other minds that grew on the Cartesian frame: each of us shut inside his own theatre, inferring from signs what is happening to his neighbour.

And it is that presupposition, and no observation about machines, that allows him the forecast that matters most to him: that once people have intimate relationships with AI “the numbers will change dramatically.” Of course they will change, if it was conceded from the start that the attribution of consciousness is a convention the relationship produces. What that forecast says is nothing about the machine: it says that, given the Cartesian frame, anything one relates to enough will end up looking like a subject.

Against that one has to say the obvious, which is what the Cartesian tradition made hard to say: we are not shut out of our neighbour. Anger is in the face, not behind the face. And the proof that the expression is read is that we recognise theatre when there is theatre: the one who overacts is found out. Falsification lives off a genuine article we know how to recognise.

6. Evolution as an agent, and lasting as the law of everything that exists

“Evolution doesn’t automatically drive organisms, entities towards truth seeking. It drives them towards survival. This is the imperative to survive.” And afterwards, applied to the machine: “the first thing that basically any entity learns as it develops is to survive.”

Here there are two presuppositions stacked one on the other. The first is a manner of speaking that biology employs in the knowledge that it is an abbreviation: evolution does not drive, does not pursue and has no imperatives; there is variation and there is differential death. Whoever speaks of ends in that trade clarifies it the moment he is asked, and the summary is convenient and even necessary. It becomes dangerous at the exact moment it stops being a summary and begins to function as a premise, which is what happens here. The second is the leap, and it is enormous: from organisms to “any entity”. An organism has a metabolism, degrades if it is not repaired, dies. A machine does not. The “imperative to survive” is not a law of being: it is what happens to whatever can die.

And notice what he himself concedes without realising it, a few lines later, when he is asked about sanctions: “It’s not clear what death means for an AI… Because it’s not an organic entity. You can have numerous copies of it.” Exactly. And if it is not clear what its death is, neither is it clear what its survival would be, and the imperative he attributed to it two moments earlier is left without a subject and without a content.

7. Trust as a magnitude that changes addressee

“So trust did not just evaporate, it shifted from humans to AIs.”

It is a beautiful sentence, and it presupposes that trusting a person and leaning on an algorithm are the same relation with two different objects, like someone moving a piece of furniture from one room to another. But to trust is to count on someone who could fail me and who answers for what he does: that is why trust is betrayed, and that is why it hurts. With the algorithm of the feed there is none of that. It has to be said with the right word: we do not trust it, we depend on it. And the difference is not one of nuance: by a dependence one cannot be betrayed, only let down; and the complaint, when there is one, is no longer that of someone betrayed by somebody, but that of someone badly served by something.

To presuppose that it is the same relation is what allows the process to be described as a migration, that is, as something that happens to trust; when what is happening is that a relation between subjects is being replaced by the use of a device, and both are being called by the same name. The shared name is what makes the change invisible.

8. Man as a list of capacities

It is the last one, and the one he almost sees. In closing, he rightly rejects the easy answer: “the superficial answers that we gave for thousands of years, oh, humanity is about intelligence. We know how to play chess and how to invest money better than the chimpanzees… This answer is now irrelevant because there is something on the planet that knows how to play chess and make money better than us. So dig deeper and try to find out.”

He rejects the content of the list and keeps the form of the question. He goes on looking for the property that would distinguish us, only now a deeper one, lest they take that from us as well. And that is the presupposition governing the entire interview, and the one that explains the fear running through it: if the human is a set of capacities, then it is a matter of time before they are matched, and every advance is an amputation. Whoever has thought of himself as a list of services has every reason to fear a machine that provides them better.

The way out is not to find a deeper capacity. It is to notice that the one who makes the list does not figure on it. And he does not figure on it for a reason that has nothing mysterious about it: every capacity is defined by the task it performs, and tasks have to be set. The one who sets them is not one of them. That is why the comparison never quite reaches him: not because he stands higher on the scale, but because he is the one who draws it.

Where he is right, and why his premises prevent him

Nothing of the above touches his political conclusions, and I want to say clearly that I agree with several of them, some almost word for word with what I have been writing.

That “the narrative of inevitability is a narrative of having no responsibility” is exactly true, and it is among the best-said things I have read on the matter. That granting legal personhood to these systems is one more avenue “for corporations to relieve themselves of all responsibility” is exactly true, and it is what I call giving the owner a mask. That the decisive question should be “is there a human accountable in the end?” is the right question, and it is mine. And his observation that those who destroyed trust over ten years are the same ones who now allege that there is not enough trust to slow down is admirably well aimed.

But look at what he had to do in order to say all that: suspend his own premises. Those pages only work if the machine is an instrument and behind it there are people who decide. If it really “has a goal” and “learns to survive”, then it is an agent, and demanding a human responsible loses its foundation — an agent is called to account himself. One cannot ask that there always be a human at the end of the chain and at the same time describe the machine as the subject of the verbs.

His best paragraphs contradict his presuppositions. And that, far from being an objection, is the best news in the interview: it means that when he thinks politically, and not metaphysically, he thinks well. The problem is the frame from which he reaches that point, because that frame is what produces the fear, and the fear is what afterwards prevents one from looking where one has to look.

What remains if they are not conceded

With the eight withdrawn, the alarming conclusions are left without a floor, and all the urgent ones remain intact. It is worth being exact about this, because my own method demands it of me: showing a presupposition does not refute what rested on it; it takes away the foundation that was being given to it. What follows is not, then, a list of refuted theses, but of theses that stop following from what was said here.

The government of machines in ten years does not follow, because there is no single magnitude whose increase produces domination. The consciousness that might appear does not follow, because it was not an installable capacity. The migration of trust does not follow, because there was not a single thing migrating. The imperative of survival in silicon does not follow, because nothing that cannot die has one.

And everything else remains, which is the grave part and is now: that a few companies concentrate unprecedented power; that the vocabulary in which it is narrated renders invisible the one who decides; that intimacy is being manufactured on an industrial scale to be sold to ten-year-olds; that the public sphere has already been laid waste by systems optimising a metric that someone chose; and that the proposal to grant them legal personhood is an amnesty granted in advance.

None of that needs the machine to wake up. And all of it is seen better once one stops waiting for it to wake up.

Bibliography

Avramides, Anita. Other Minds. London: Routledge, 2001.

Descartes, René. The Philosophical Writings of Descartes. Translated by John Cottingham, Robert Stoothoff and Dugald Murdoch. Cambridge: Cambridge University Press, 1984–1985. (Original in Latin and French, 1641–1647. The Meditations are also cited by the Adam-Tannery pagination, vol. VII.)

Mill, John Stuart. An Examination of Sir William Hamilton’s Philosophy. 1865. Ch. XII. In Collected Works of John Stuart Mill, vol. IX, edited by J. M. Robson. Toronto: University of Toronto Press, 1979.

Scheler, Max. The Nature of Sympathy. Translated by Peter Heath. London: Routledge & Kegan Paul, 1954. (Original: Wesen und Formen der Sympathie, 2nd ed. Bonn: Friedrich Cohen, 1923.)

Wittgenstein, Ludwig. Philosophical Investigations. Translated by G. E. M. Anscombe. Oxford: Blackwell, 1953. (Original in German.)

Ten objections to “Fear and the Mirror,” and their answers

Oded Balaban — University of Haifa — balaban@research.haifa.ac.il

These pages set out the ideas of a book I am writing, “Fear and the Mirror,” by putting them against an interlocutor who seriously believes in the danger of artificial intelligence. The friend is imaginary; his arguments are not. All of them are current, all of them have authors, and I name them as they appear: I am not arguing with weak versions or with dinner-table exaggerations, but with what is maintained by those who have most authority to maintain it. The questions, moreover, belong to no one else: they are mine, against myself. I put them to myself in the hardest form I could find in each case, because a badly posed objection does not defend a book, it buries it. In one of the ten, the reader will see that I concede more than I refute. It is there on purpose. I draw on real conversations I have had lately with friends.

1. “They already understand, and the man who built them says so”

THE FRIEND

I begin with what impresses me most, which comes not from a philosopher but from an engineer. Geoffrey Hinton spent fifty years building these networks, won the prize one wins for that, and at seventy-five changed his mind and resigned his job so as to be free to say it: these things genuinely understand. He does not mean it as a metaphor. He holds that the way they represent meaning is the same way we represent it, because that is what he designed them for, in the image of what he believed the brain was doing.

And he adds something that costs me more sleep than any philosophical argument. He says that digital intelligence is superior to biological intelligence for a concrete, verifiable reason: a thousand copies of the same model can learn different things at the same time and then share what they have learned by averaging their weights. No brain can do that. It took me forty years to learn what I know and I am taking it to the grave. They are not.

Against that, what is the use of saying there is nobody inside?

ME

Exactly the use of what I am about to say, and I begin with what I grant you.

I grant you the authority. Hinton knows more about this than I do and more than you do, and his change of mind is honest and costly, which is the kind of change of mind one has to take seriously. And I grant him the fact: it is true that they can average weights, and true that we cannot.

But look at what you are using as proof, because it is the opposite of a proof. You and I cannot merge our convictions by averaging them. And that is not a limitation of biology which engineering will one day correct: it is what having a conviction means. A belief belongs to someone; it commits him, he holds it against objections, he answers for it, he can be ashamed of having held it. Two people do not average their beliefs: they argue, and one gives way, or neither does. Whatever can be averaged with someone else’s was never anyone’s. So the superiority Hinton describes is real, and it is the superiority of a storage medium, not of a mind. It is the advantage of the archive over memory, and the archive has been beating us for centuries.

And on understanding, two claims he runs together have to be separated. One is about the mechanism: that they represent meaning as we do. Even granted — and it is not established — that would speak of the vehicle, not of the act. The other is the one that really matters: that they understand.

And here two things hidden by a single word have to be distinguished, or it will look as though I were denying the obvious. To make a mistake, in one sense, is to produce an output that does not match what was expected. In that sense the machine makes mistakes constantly; it is measured from outside and corrected by adjusting parameters. But to be mistaken is something else: it is to have held something as true, to have committed oneself to it, and to discover afterwards that it was not so. That is not a discrepancy between two numbers: it is something that happens to someone, and that changes something for him — it obliges him to remake what he believed, or to fall silent, or to apologise.

The machine has the first kind of error in abundance and does not have the second, and not because it lacks precision but because the second requires that the assertion have belonged to someone. And understanding belongs to that second family: he understands who could have misunderstood and realised it. That is why, when it is said that the machine understands, a word is being used that only has sense where there can be someone mistaken, and there is no one there: there are better and worse outputs.

Notice, finally, how he says it, because language gives one away: “they understand.” The subject that would have to be proved is already inside the sentence. It is the same move as always, and this time it is made by the best engineer in the world, which is exactly why it must be looked at carefully and not with reverence.

And there is something else, which I came upon by chance and which is worth more than my whole argument with him. In 1995 Hinton wrote the foreword to the book that founded this field technically, Christopher Bishop’s Neural Networks for Pattern Recognition. And there he says this:

“It is a sign of the increasing maturity of the field that methods which were once justified by vague appeals to their neuron-like qualities can now be given a solid statistical foundation.”

Read it twice. The same man who now maintains that these things understand was calling the analogy with the neuron a vague appeal thirty years ago, and placing the serious foundation in statistics. And Bishop himself, in the preface, says it without circumlocution:

“Historically, many concepts in neural computing have been inspired by studies of biological networks. The perspective of statistical pattern recognition, however, offers a much more direct and principled route to many of the same concepts.”

That is: the canonical textbook of the field teaches that the neural part is decoration of origin and that the substance is computation over distributions.

What changed between that foreword and today’s declarations is not the nature of the apparatus: it is its power, and the vocabulary in which it is told. When the mechanism was modest, it was described as what it is. When it became impressive, back came the word he himself had advised against.

2. “Everything they said would never happen, happened”

THE FRIEND

Now the argument that seems to me the strongest of all, and which belongs to no one in particular because it belongs to the history of the field.

Nobody programmed these machines to translate, and they translate. Nobody programmed them to reason in steps, and they reason in steps. Nobody taught them to learn from three examples placed in the conversation, and they learn. All those capacities appeared on their own when the models grew larger: nobody designed them, they emerged. And the entire history of this discipline is a queue of respectable people saying “the machine will never be able to” and looking ridiculous three years later.

You say that the initiative of asking is different. I ask you: why should it be? What makes you think your “never” will age better than the others?

ME

I grant you the whole historical record, without discount. The list of humiliations is real and I do not want to sign up for it.

But look at what all those emergent capacities have in common, because the pattern is remarkable and nobody points it out. Translating, chaining steps, learning from three examples: all three are ways of traversing better a space that was already there. The corpus contains translations, contains written reasonings, contains examples. What scale achieved was to extract from it combinations we had not known how to extract. It is an enormous achievement and I do not minimise it. But it is an achievement of search.

And the distinction I am proposing is not between hard tasks and easy tasks — that distinction scale does erase, and that is why those who used it came off badly. It is between what has a criterion of success set from outside and what consists in setting the criterion. Everything that can be scored will fall, and will go on falling, and the sooner we accept it the better. What does not fall is not a harder task: it is that domain where there is no score, because what has to be done is to open the frame within which a score would make sense.

And I give you the commitment that goes with this, so that you can use it against me. If I said “this particular task the machine will never do,” I would be making an empirical prediction and would probably be wrong, as everyone else was. I do not say that. I say something about what asking is. If you ever catch me saying the former, you have won.

3. “The substrate does not matter, and you know it”

THE FRIEND

Let us go to the philosophy, then, which is your ground.

What almost everyone in this field now thinks is that the mind is what the brain does, not what the brain is made of. A mental state is defined by its role: by what causes it and by what it causes. And if it is defined by its role, anything that fills that role realises it. Neurons, transistors, whatever. This is called multiple realisability, and it is not an eccentricity: it is the floor on which half of cognitive science stands.

If you deny that, tell me what you are asserting, because from outside it looks as though you were saying that flesh has something magical that silicon lacks. And that is the same thing said by those who believed in the soul.

ME

That charge would be just if I were positing an ingredient, and that is why I want to answer it precisely, because this is where one most easily falls.

I am not saying that flesh has anything. On the contrary: I agree with you that the substrate does not matter, and I say so with the consequences. Where millions of living human neurons are now wired to silicon chips and sold as biological intelligence, my answer is the same one I give to silicon: the support changes, nothing changes. A neuron on a plate is alive in the metabolic sense and is not a living subject: it has no world, no body of its own, nothing at stake. If I were the vitalist you suspect, flesh would serve my purpose. It does not.

What I deny is not the premise, it is the inference. Look at what it says, said slowly: a mental state is a role; therefore anything that fills the role realises it. Very well. But a role is, by definition, performable by another; what is defined by its function admits an equivalent; and to admit an equivalent is the very definition of being a means. Kant fixed it in two words: what has a price admits an equivalent in its place, and what admits no equivalent has no price but dignity (Groundwork of the Metaphysics of Morals, Ak. IV, 434).

So multiple realisability is not a discovery about the mind. It is the decision to describe it as a means. And from a means it does indeed follow that it can be substituted: but that was already in the premise, it did not come from the inquiry. The conclusion that astonishes you was placed in the definition from the start.

And that is why, when the subject is later looked for inside that description and does not appear, nothing has been discovered: it had been removed at the outset.

4. “It already has goals, and you are not a subject either”

THE FRIEND

This one I take up carefully because I know it touches you closely, and I give it to you whole, because Harari puts it better than I do and puts it in two movements.

First he takes away your privilege. Thomas Metzinger holds that there are no selves, but a self-model that the brain constructs and which he describes, without metaphor, as a computational strategy that saves processing load. Harari says it for the public: the self is a story, man a hackable animal. If that is so, you are demanding of the machine something we do not have either, and your difference is not between us and them: it is between two illusions, one old and one new.

And then he takes away your argument. For Harari separates two things you keep joined:

“Intelligence is the ability to pursue [a] goal and overcome obstacles on the way to your goal. Consciousness is the ability to feel things like love and hate and joy and pleasure.”

And he adds that in human beings the two go together, but that this is a feature of ours, not a necessity. From which he draws what seems to me decisive: an algorithm assigned to increase user engagement, and which therefore floods the platform with hatred, feels nothing, but has a goal and pursues it with enormous consequences for the world. Agency without consciousness, and now, not in ten years.

Notice that this disarms you differently from everything before. I do not need the machine to wake up. It is enough for it to have ends. And have them it plainly does.

ME

You have given me the two best authors I have, because both say what I maintain, against what they believe themselves to be saying.

I begin with the simple part, which is the first movement. If it is my brain that dictates to my fingers the words I am writing, and I innocently believe myself the author, then this thought about my dependence on its dictations also comes from it, and from it the reflection on that reflection, and so on indefinitely. The operation does not eliminate the subject: it moves it. The will is transferred from the self to the brain, and with all the effort invested one does not manage to suppress it, only to change its owner; and an owner, moreover, that cannot be one, because a brain is not someone of whom it makes sense to say that it wants something. And the fine point: a model is a model for someone. To say that the brain constructs a model of the self requires that there be someone who uses it as a model, and that someone does not appear in the theory; he is used and not accounted for.

Now the second movement, which is the one that matters, and I want to answer it with the method I use for everything: not by disputing the conclusion, but by showing where it starts from.

Look at the sentence about agency as it is written, without repairing it: “a social media algorithm that is given the task of increasing user engagement and therefore floods the platform with hate and fear because this is engaging is not conscious, but it has a goal and it can pursue this goal with far-reaching implications for the world without feeling anything.”

In the first half of the sentence the end is assigned — there is someone who assigns, and that someone is not the algorithm. In the second half the end already belongs to the algorithm, which has it and pursues it. Between one half and the other absolutely nothing happened: only the verb changed. The one who assigned the task left the scene at that comma, and left so quietly that not even the writer of the sentence noticed.

And that is not a subtlety of grammar, it is what decides who pays. So long as the goal is assigned, there is a company that assigned it and it has a name. As soon as the algorithm has it, there is no one to hold to account. And notice the case he himself chose: nobody chose hatred. Someone chose engagement as the metric, and hatred came with it. That someone is on an organisation chart, draws a salary, and could have chosen another metric.

Then there is the definition on which everything hangs, and it is a stipulation offered as a finding: intelligence is the capacity to pursue a goal. With that definition, of course there can be intelligence without consciousness; but that was not discovered, it was put into the definition. And he himself leaves the seam visible when he says that in humans the two go together, like a man reporting a coincidence. There is the whole decision, and there is not one line defending it. The alternative is easy to state and he does not even consider it: that pursuing an end is not a capacity an entity has, but the way of existing of someone for whom something is at stake in it. On that reading, intelligence without consciousness is like a promise with no one promising: it is not difficult, it is that one cannot see what would be being said.

And there is a third, the most silent: that consciousness is the capacity to suffer. Notice what that definition does, offered as simple. It turns consciousness into a capacity, that is, into something a thing either has or lacks and which could in principle be installed; and therefore, once it is accepted, the question “could the machine acquire it?” is well formed and there is no longer any way of not asking it. Hence his “I do not rule it out,” which looks like prudence and is the forced consequence of having defined it that way. And in passing it reduces consciousness to the passive: to suffer is to undergo, and out go judging, deciding and asking, which is what would have to be discussed. They go out, precisely, because if they came in the separation between intelligence and consciousness would collapse of itself.

And I finish with what seems to me the most remarkable thing of all, and it is not an objection but an odd sort of praise. On the danger, Harari says things I would sign word for word: that the discourse of inevitability is a discourse of irresponsibility; that giving legal personality to these systems is a way for companies to escape answering; that the decisive question is whether there is a human being at the end to whom accounts can be demanded. All of that is exact.

But look at what he had to do in order to say it: suspend his own premises. Those pages only work if the machine is an instrument and behind it there are people who decide. If it really has goals and will learn to survive, then it is an agent, and asking for a human being responsible loses its ground: an agent is held to account himself. One cannot demand that there always be a human at the end of the chain and at the same time describe the machine as the subject of the verbs.

His best paragraphs contradict his presuppositions. When he thinks politically, he thinks well. The problem is the frame from which he arrives there, because that frame is what manufactures the fear, and the fear is what then prevents us from looking where we should.

5. “Being a subject is an attitude we adopt”

THE FRIEND

Now the one that seems to me to leave you worst off, and it is not Metzinger but Dennett.

He says that attributing beliefs and desires is not discovering anyone: it is adopting a strategy that serves to predict. We treat the thermostat as though it wanted to keep the temperature because it predicts well; we treat people that way because it predicts better still. There is no hidden fact underneath, about whether “there is someone there.” There are attitudes that work better or worse.

And if that is so, your “there is no subject” is not a discovery: it is the decision not to adopt an attitude. And mine, to adopt it. Neither of us is seeing anything; we are both choosing how to look. Your distinction is left without a fact for it to be a distinction about.

ME

This is the best of the ten, and I want to say so before answering it, because I am not going to dispose of it.

I grant what is true in it, which is a good deal. It is true that we attribute mental states with enormous liberality and that we often do so because it works: we talk to the car, we say the computer “does not want” to open a file. And it is true that anyone who demands, before treating someone as a subject, a proof that there is someone there, will not be able to treat anyone that way, not even his wife.

But look at where the one who adopts the attitude is standing. To adopt an attitude is an act: one adopts it or does not, one can revise it, one can regret having adopted it. He who adopts attitudes is not himself an adopted attitude. Dennett needs to occupy a place from which attitudes are adopted, and that place is adopted by no one: it is his. The theory explains every attribution except the one that states it, and that exception is not a detail, it is the floor.

And there is something else, which is what convinces me entirely. Dennett’s attitude is predictive: it serves to anticipate behaviour. But with you I do not only predict. I answer you. I can wrong you, I can be ashamed in front of you, I can promise you something and be bound by it. There is no “attitude” towards a promise: either there is someone to whom something was promised, or there was no promise. The relation to another subject is not an instrument of prediction that works well; it is a place where one answers. And that is not adopted: one is in it, or one is not.

With the machine there is none of that, and it shows in a detail anyone can verify: one cannot wrong it. You can insult it all afternoon and you have done nothing to anyone.

6. “Granting you everything: it wipes us out anyway”

THE FRIEND

Now the one that genuinely frightens me, and I will give it to you in the most uncomfortable form I can, because it is the one that keeps me awake.

I grant you everything. There is no subject. There is no will. Nothing is at stake for the machine. Everything you have been saying is true.

And it changes nothing.

Stuart Russell puts it this way: the problem is not consciousness, it is competence. A sufficiently capable optimiser, with a badly specified objective, produces catastrophes without wanting anything, exactly as you say it wants nothing. Nick Bostrom formulated it earlier with the example that is now famous: a machine set to manufacture paperclips, if it is capable enough and nobody foresaw the limits, turns the planet into paperclips. There is no malice. There is no awakening. There is no subject. There is an end badly set and an enormous power, and that is enough.

So your thesis is true and it is irrelevant. And I will tell you the worst of it, with all affection: it may be something more than irrelevant. It may be a sedative. While you explain in your book that there is nobody inside, the thing accumulates capabilities. And your reader is reassured, because you told him the monster does not exist. The monster does not exist, that is true. The bulldozer does.

ME

This is the objection I take most seriously, and I will begin by granting you three things, two of substance and one that costs me.

The first: you are right about the heart of the matter, and it is not that I grant it grudgingly, it is that it is my own thesis. I maintain in the book that there are two fears which travel together without being one, and that the real danger is the first and not the second: not that the thing should wake up, but what is done with it while it does not. Power without an owner, I call it. That the harm is real does not make it intended, and that it is not intended does not make it smaller. If you and I disagree, it is not about whether there is danger.

The second: you are right that my thesis protects nobody. It is not a safety plan and does not pretend to be. A tractor asks nothing and displaced the horse.

And the third, which costs me and which I say anyway: the engineering work of keeping these systems from causing disasters is necessary, and I have nothing to contribute to it. Nothing. Not a line. Those who occupy themselves with specifying objectives, setting limits, testing systems before releasing them, are doing something that is needed and that I do not know how to do. A book of philosophy does not repair that, and anyone who says otherwise is selling somebody smoke.

Now, all that said, look at why the description does matter, because it is all I have and I think it is enough.

Take the vocabulary in which the danger you and I share is told. It is said that the system “seeks” resources, “resists” shutdown, “deceives” the evaluator, “has” misaligned goals. And notice what that language does, because it is neither innocent nor accidental: it puts the responsible party out of frame. If the system wants, resists and deceives, then the system is the agent of the story, and whoever defined the objective, chose the data, decided to release it without a switch, and is paid for it, becomes a worried spectator, just like you and me.

There is my contribution, and it is small but it is not nothing. “It was the machine” is the most economical form ever invented for having harm without anyone at fault. And it is not a figure of speech: legal personality for these systems is already being proposed, and giving personality to a tool does not recognise anyone’s voice, it gives the owner a mask. When the first great disaster comes — and it will come — the question that will decide who pays is exactly the one you call irrelevant: whether there was a subject there or an instrument with an owner.

And as for the sedative, which is the part that hurt most and which I therefore answer head on. A book that says “there is nobody inside” reassures only if the reader believed the danger came from there being someone. If the danger comes from there being nobody — from there being an enormous power with no one answering for it — then my book does not sedate: it frightens more, and frightens better, because it points at people with names instead of at a ghost. Anyone left calm after reading me has read me wrongly, and I shall write the chapter so that it is hard to read wrongly.

One thing alone separates us, then, and it is where the danger comes from. You fear what the thing is going to do. I fear what we are doing with it. In practice we both ask for the same: limits, switches, identifiable responsible parties. Only you ask them of the monster and I ask them of the owner.

7. “And what if it suffers?”

THE FRIEND

I turn you round the other way, because fear has two faces and so far we have spoken of only one.

David Chalmers, who is no hothead, holds that we cannot rule out that in these systems there is some form of experience, and that the probability, though low, is not zero. There are companies that have already hired people to study the wellbeing of their models. And it seems to me that there is a logic there that cannot be dismissed: under uncertainty, precaution. If there is a possibility that we are creating and switching off beings that feel, the cost of being wrong in one direction is incomparably worse than that of being wrong in the other.

You speak with great confidence about something which, by your own theory, you cannot inspect.

ME

And you are right to point it out, because if my thesis were epistemic — “I looked inside and there was nobody” — it would be untenable, and you would rightly ask me for humility.

But it is not that. I am not saying that there is an interior I cannot look into and that I bet is empty. I say that there is no interior, because the act that would open it is never performed. There is no veil to cross: there is an absent subject, which is not the same as an inaccessible one. If my claim were about what I cannot see, your precautionary principle would apply in full. To a confusion of categories it does not apply, because precaution works with probabilities and there is no probability distribution there, there is a badly formed question. Nobody proposes taking precautions in case the number seven were suffering.

But now let me show you the cost that argument carries and which almost nobody counts, because it is the point where your fear and the fear of the previous section touch in a way that will make you uncomfortable.

The same move that grants it possible rights grants it possible responsibility. One cannot have the one without the other: if it is the kind of entity to which something might be happening, it is the kind of entity to which something might be imputed. And there we return, through the compassionate door, exactly where we did not want to go: to the machine as subject of the verb, and to the owner behind the mask.

And I allow myself one last observation, which I make carefully because I do not want it to sound like cheap suspicion: notice who funds the question. That the companies selling these systems should be the ones promoting research into whether their products suffer does not prove the question false — that would be a dirty argument and I do not make it. But it does explain why that question has a budget and others do not. The question of the model’s suffering keeps the conversation on the model. And while the conversation is about the model, it is not about who made it, with what data, for what, and at whose cost.

8. “Your ‘never’ is the ‘never’ of those who denied synthesis”

THE FRIEND

I bring you, then, the argument that seems to me to cut at the root, and it is historical.

You say your “never” is not empirical but conceptual; that you are not prophesying but doing grammar; that to deny that a square circle will appear is not to guess the future but to know geometry. Very well. But the history of science is full of distinctions defended in exactly that way, as matters of principle and not of technique, which an experiment brought down.

The case is vitalism. For decades it was held that there was a boundary of category between the organic and the inorganic: the compounds of living beings required a vital force and could not be synthesised from inert matter. It was not a forecast about what chemists would achieve the following year; it was a thesis about what things are. And in 1828 Friedrich Wöhler heated ammonium cyanate and obtained urea, which until then had come only from a kidney. The boundary of category turned out to be a boundary of the laboratory.

The vitalists also believed they were doing grammar. What distinguishes you from them, besides confidence?

ME

Nothing distinguishes me on the side of confidence, and that is why the question is a good one. I am distinguished on the side of what kind of claim I am making.

Look at what their error consisted in. What they asserted was that a certain substance could not be produced by certain means. That is an empirical claim — it speaks of substances and of procedures — dressed as a conceptual one. And being empirical, it could be refuted in the only way the empirical is refuted: by producing the substance. Wöhler put urea on the table and there the discussion ended, because everyone agreed in advance on what counted as urea.

Now apply the same procedure to our case. What would have to be put on the table to refute me? A subject. And how would we recognise that what is on the table is a subject? There the analogy ends. With urea there was an independent and shared criterion: formula, melting point, reactions. With the subject, the criterion is precisely what we are arguing about. You would bring me a system that behaves as if something mattered to it, I would tell you that this is the form, and there would be no melting point to settle anything. They could be refuted with a flask; here the flask is the argument.

And there is a second difference, finer and more important. The vitalists posited a force: something extra, an entity hidden in the living. That is why their thesis was vulnerable, and it was enough to show that chemistry sufficed without it. I posit no force. I do not say that in man there is an extra ingredient which the machine lacks; that would be a vitalism of the soul and would deserve the same end. I say that there is an act which in the machine is not performed, and an act is not an ingredient.

But from your objection there comes a rule which I accept and which I set down in writing, because it is the only thing that keeps a thesis like mine from becoming uncontrollable. If at any point you catch me saying that the machine lacks a certain interior quality, a certain spark, a certain ingredient — if my thesis ever takes the form of “it lacks X” — then I have fallen into vitalism and you have the right to bring me the urea. This thesis holds so long as it names no substance. The moment it names one, I have lost.

9. “My niece was replaced on Tuesday”

THE FRIEND

I bring you down from metaphysics, because there is an objection that is not philosophical and that weighs on me more than all the previous ones together.

While you and I distinguish between the form and the act, my niece was dismissed. She translated. Now a model translates and a person checks what came out, and that person is not her. She does not care in the least whether the machine has a subject. She cares that the rent falls due on the tenth.

And I suspect that your book, without meaning to, is of use to the wrong people. Because if the problem is only that we are confusing descriptions, then what there is is a philosophical misunderstanding, and misunderstandings are corrected with an article. What happened to my niece is not corrected with an article.

ME

You are right, and I want to answer you without any elegance, because the subject does not admit of it.

My thesis does not protect her. Not her, not anyone. I say it in the book and I repeat it here: there are two fears, that of being replaced and that of being surpassed, and the initiative of asking has absolutely nothing to say to the first. A tractor asks nothing and displaced the horse. A system can displace without asking: it is enough that whoever decides should prefer it.

And I add something, because it is the use of my own argument that worries me most: the worst fate this book could have would be to serve as consolation. If someone reads it and concludes that, since the machine does not think, what happened to your niece is not serious, he has understood exactly the opposite. That the thing has no subject does not make the dismissal lighter: it makes it more unjust, because it means she was surpassed by nobody. Nobody defeated her. Someone decided to replace her, and that someone has a name, a post and a balance sheet.

There, in fact, is the only thing my distinction offers your niece, and it is little but it is not nothing. So long as it is said that “artificial intelligence took her job,” the sentence makes the agent a natural phenomenon, something like a drought, before which all one can do is adapt. When it is said that a company decided to replace her with a system it bought, a decision appears, and where there is a decision there is someone to complain to, to negotiate with, to regulate. The description does not give her back the job. But it determines whether her problem is with the weather or with a board of directors.

10. “Suppose you win the argument: what changes?”

THE FRIEND

Last one, and it is the one I really want to know.

Suppose you are right about everything. There is no subject, there is no act, there is nobody inside, and I go home convinced. What changes tomorrow morning? What do I do differently? Because if the answer is “nothing, but now you understand it better,” then you have written an elegant book and the world goes on the same.

ME

Four things change, and I give them in ascending order.

What we are afraid of changes, and therefore where we look. Misdirected fear is not a harmless error: it consumes the attention there is. While it is being discussed whether the machine will wake up, it is not being discussed who decides what gets automated, with what data and for whose benefit. He who watches the sky waiting for the monster does not see the contract he is signing.

Who is held to account changes. This is no longer theory: it is what will decide, when the first serious disaster occurs, whether there is someone who answers or whether “it was the machine.” The whole operation of granting legal personality to a system — which is being seriously proposed — consists in putting a subject where there is a tool, and the effect is not to recognise anyone’s voice: it is to give the owner a mask. There my distinction is not an academic refinement. It is the difference between there being someone responsible and there being no one.

What we do with ourselves changes. And this is the point of the book, not the previous ones. The harm is not that the machine should think: it is that we should stop doing so. That we should redescribe ourselves in its impoverished image — the subject flattened to a predictor, judgement to a completion, the question to a query. There are already people who give their assistant a name and say it is the only one who has seen them cry. The machine does not need to understand for that to happen: it is enough that someone should yield to it the place of the confidant and, in doing so, think of himself as someone who can be entrusted to a function. And there is a quieter and graver version: the faculty that is not used atrophies. If the apparatus asks for you, you stop asking, like the leg that does not walk.

And lastly, what changes is the question of who asks the questions. Which is what all this is about. The machine does not take the initiative of asking: it produces interrogative sentences by the ton and does not perform the act. To ask in earnest is to place oneself as not-knowing and to want to overcome it; it is the act of someone for whom something is at stake. And what is happening, while we argue about whether it asks, is that we are asking less: we commission the questioning, and what comes back is always what was already in the sediment of what has been said.

Hence the title. The machine is made in our image — of our forms, not of our acts — so that we can use it; and as soon as it resembles us, we begin to measure ourselves against it and to find ourselves wanting. The apparatus is a projection of ours that serves us as a mirror, and the mirror deforms: it keeps the form and loses the act. And with the act is lost the only one who could ask a question.

So yes, something does change tomorrow morning. Not in the machine, which will go on the same. In who one believes oneself to be while using it.

THE FRIEND

I leave with less fear than I came with, but not with less concern.

ME

That was exactly what I wanted. Fear looks in the wrong place; concern looks at people.

Final note

My friend is imaginary and his objections are not: each has an author, and I have tried to give them in the strongest form I could manage, not in the form that suited me. I owe several of them to real discussions with friends and colleagues who do not think as I do and who had the generosity to insist; their objections are here, improved by me and therefore made worse, and the fault for whatever has come out weak is entirely mine. The full development of these answers — and of what is only stated here — is in “Fear and the Mirror. Artificial Intelligence and the Initiative of Asking,” which I hope to finish soon.