Thread:Daitiansg/@comment-93.169.86.247-20150509140609/@comment-125896-20150513184550

“A powerful AI system tasked with ensuring your safety might imprison you at home. If you asked for happiness, it might hook you up to a life support and ceaselessly stimulate your brain's pleasure centers. If you don't provide the AI with a very big library of preferred behaviors or an ironclad means for it to deduce what behavior you prefer, you'll be stuck with whatever it comes up with. And since it's a highly complex system, you may never understand it well enough to make sure you've got it right.”

― James Barrat, Our Final Invention: Artificial Intelligence and the End of the Human Era