ABSTRACT. In his paper “The Superintelligent Will,” Nick Bostrom formalized the Orthogonality thesis: the idea that the final goals and intelligence levels of artificial agents are independent of each other. This paper presents arguments for a (narrower) version of the thesis. It proceeds through three steps. First it shows that superintelligent agents with essentially arbitrary goals can exist in our universe – both as theoretical impractical agents such as AIXI and as physically possible real-world agents. Then it argues that if humans are capable of building human-level artificial intelligences, we can build them with an extremely broad spectrum of goals. Finally it shows that the same result holds for any superintelligent agent we could directly or indirectly build. This result is relevant for arguments about the potential motivations of future agents: knowing an artificial agent is of high intelligence does not allow us to presume that it will be moral, we will need to figure out its goals directly. pp. 68–84

Keywords: AI, Artificial Intelligence, efficiency, intelligence, goals, orthogonality

This email address is being protected from spambots. You need JavaScript enabled to view it.
Future of Humanity Institute, Oxford Martin School
Philosophy Department, University of Oxford

Home | About Us | Events | Our Team | Contributors | Peer Reviewers | Editing Services | Books | Contact | Online Access

© 2009 Addleton Academic Publishers. All Rights Reserved.

Joomla templates by Joomlashine