IMDEA Networks Institute Publications Repository

Superintelligence Cannot be Contained: Lessons from Computability Theory

Alfonseca, Manuel and Cebrian, Manuel and Fernández Anta, Antonio and Coviello, Lorenzo and Abeliuk, Andrés and Rahwan, Iyad (2021) Superintelligence Cannot be Contained: Lessons from Computability Theory. [Journal Articles]

[img] PDF
Download (652Kb)

Abstract

Superintelligence is a hypothetical agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. In light of recent advances in machine intelligence, a number of scientists, philosophers and technologists have revived the discussion about the potentially catastrophic risks entailed by such an entity. In this article, we trace the origins and development of the neo-fear of superintelligence, and some of the major proposals for its containment. We argue that total containment is, in principle, impossible, due to fundamental limits inherent to computing itself. Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world, strict containment requires simulations of such a program, something theoretically (and practically) impossible.

Item Type: Journal Articles
Subjects: UNSPECIFIED
Divisions: UNSPECIFIED
Depositing User: Antonio Fernandez
Date Deposited: 12 Jan 2021 09:12
Last Modified: 12 Jan 2021 17:24
URI: http://eprints.networks.imdea.org/id/eprint/2274

Actions (login required)

View Item View Item