There is a debate now underway among AI experts that shadows and taints the whole field. One side of the argument is concerned about the possibility of severe dangers arising from the development of superintelligent general AI. The other is not. They think such fears are groundless and could needlessly impede AI development with unnecessary regulations.

Some on the unconcerned, fearless side are not worried because they do not think superintelligent AI will ever come to pass, or if it does, that it will only happen in the remote future, too far off to be concerned about now. This group is called the Techno-Skeptics by Max Tegmark in his book Life 3.0: Being Human ion the Age of Artificial Intelligence (Knopf 2017). Others on the fearless side agree with the concerned group, which we sometimes call the Pro-Caution side, that superintelligent AI will likley come to pass in this century, but think it will be all good. Max Tegmark, who himself is on the Pro-Caution side, calls this group the Digital Utopians.

Aside from the small number of AI experts who are Techno-Skeptics, the debate between the Digital Utopians and Pro-Cautionary proponents, which Tegmark calls the Beneficial AI proponents, arises out of an underlying agreement that general artificial intelligence has the potential to become smarter than we are, superintelligent. Both side of this debate agree that super-evolved AI could become a great liberator of mankind that solves most problems, cures diseases, extends life indefinitely and frees us from drudgery.

Out of the common ebullient hope of the benefits of AI superintelligence arises the Cautionary group that also sees a potential dystopia from artificial superintelligence. They are concerned about doomsday scenarios if AI is not properly regulated. In this camp are some of the smartest people around, including Stephen Hawking, Elon Musk and Bill Gates. They fear that superintelligent AIs could run amuck without appropriate safeguards. These utopia cautionaries fear that a super-evolved AI could doom us all to extinction, that is, unless we are very careful to make sure the superintelligence is beneficial to mankind. As Tegmark explains, this fear does not arise from concern that the superintelligent AI will become conscious or evil, but because the AI will have goals that are misaligned with our own.

Some in the Pro-Cautionary, Beneficial AI camp have only small concern about possible extinction, but fear that super-AIs, if developed, could diminish us in many other, less obvious ways. Think of kept human beings in gilded cages of luxury. Think of future humans guided and cared for by benevolent AIs who have higher than human intelligence. Many long for such a life of luxury, but others are not so sure.

We could lose abilities by our dependency. This is already happening with the technology we already have, such as cursive handwriting, spelling and rudimentary arithmetic. Could super AI take this to the next level? Could we lose the ability to think for our self? Could we be denied the adventures of discovery and creativity? Could we be devalued, indoctrinated and lose freedoms, choices, emotions and experiences without even knowing it? Could we lose fundamental human rights, including the right of self-determination, the right to work, to self-care, privacy and basic human dignity? What price utopia?

The No-Fear, Digital Utopian camp strongly disagrees with all doomsday fears, both hard and soft. This side includes both Google’s Larry Page and Facebook’s Mark Zuckerberg. Mark is a strong proponent of the No-Fear group. His company is a leading researcher in the field of general AI. In a backyard video that Zuckerberg made live on Facebook on July 24, 2017, with six million of his friends watching on, Mark responded to a question from one: “I watched a recent interview with Elon Musk and his largest fear for future was AI. What are your thoughts on AI and how it could affect the world?”