The Original AI Doomer: Dr. Norbert Wiener
Long before Geoffrey Hinton and Eliezer Yudkowsky sounded warnings about artificial intelligence, there was Dr. Norbert Wiener in the 1950s and 60s. Wiener was a pioneer in the field of Artificial Intelligence, or as he called it ‘Cybernetics.’
His groundbreaking work laid the foundations that our current AI boom is built atop of and his concerns about AI have no doubt influenced those being voiced by his successors in the field today.
In late 1959 Wiener gave a speech to the American Association for the Advancement of Science. In early 1960 it would be adapted and published as a paper titled ‘Some Moral and Technical Consequences of Automation.’ The speech began by citing the predictions of Samuel Butler in the 1860s that foretold machines would eventually out-evolve and rule over man, noting they had not come to pass. However, he said it was different this time since machines had become so much more advanced at such a rapid pace.
Read Butler’s prediction at the link below:
Wiener observed that generally people wished slaves to be intelligent and subservient, however “Complete subservience and complete intelligence do not go together.” He’d posit that like the philosopher slaves of Ancient Greece, machines would eventually outwit their masters, similarly:
“if the machines become more and more efficient and operate at a higher and higher psychological level, the catastrophe foreseen by Butler of the dominance of the machine comes nearer and nearer.”
These scary predictions, from a pioneer in the field, were taken seriously and generated headlines - much like more recent concerns from AI experts have. Coverage of Wiener‘s warnings referred to him as a new, better educated Samuel Butler. While Butler was a writer and cultural critic, Wiener was an expert and pioneer in computer science - so who were we, or anyone else to question his well informed prognostications?
Larger players in the industry would push back on this rhetoric - such as IBM expert Arthur L. Samuel - making Wiener appear like a conscientious Goliath verses a Corporate David. Samuel’s retort to Wiener: "A machine is not a genie, it does not work by magic, it does not possess a will, and, Wiener to the contrary, nothing comes out which has not been put in”
This wasn’t the first time Wiener had voiced concerns about thinking machines, a decade prior his book ‘Cybernetics’ (1948) would voice a number of dystopian predictions about AI. A year after its publication - In 1949 - the editor of The New York Times asked Wiener to write an op-ed about “what the ultimate machine age is likely to be” exploring some of they key topics in the book. The first draft was rejected by the editor, who requested another draft, which was thankfully archived (unlike the first):
In the second draft, Wiener would cite old fables - such as genies in bottles - that he said contained a powerful lesson: if we have the power to realize our wishes “we are more likely to use it wrongly than to use it rightly, more likely to use it stupidly than to use it intelligently.” His prediction, one familiar in the current dialog around AI: “the machines will do what we ask them to do and not what we ought to ask them to do.”
Wiener also predicted mass unemployment and an “industrial revolution of unmitigated cruelty” by making “possible the factory substantially without employees”, factories that could bankrupt their owners buy taking orders too literally. He voiced similar concerns about autonomous weapons too, which he said "may produce a policy which will win a nominal victory on points at the cost of every interest we have at heart, even of national survival.”
His final line and parting message:
“We can be humble and live a good life with the aid of the machines, or we can be arrogant and die.”
The editor of The New York Times would reject the second draft too, requesting a copy of the first draft - so the best of both could be combined into one piece. Wiener would decline the request citing it would require an impractical amount of co-ordination, something unavoidable in a pre-internet age. If only they’d had internet, Google Docs and even better - a large language model to help combined the first and second drafts, as we do 74 years later.
Three quarters of a century on the concerns and debates remain the same. While some experts like Yann LeCun push back and are labelled corporate stooges - others make wild prognostications, ones the press treat seriously. They are, in turn heralded as new Samuel Butler’s, as Norbert Wiener was. Wiener’s prophecies didn’t come true, like Samuel Butlers before him, but this time - like last time - it is different.
People have foreseen and written about the consequences of human-level and superintelligent AI for a long time, which if anything suggests that such concerns are grounded. Weiner & others of his era underestimated the complexity of the task of creating AGI. We've never actually been close to it before in history. But we are now.
Weiner vs Wiener