[intelligence explosion] The first ultraintelligent machine is the last invention that man need ever make
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion’, and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.
We are still the masters of our fate. Rational thinking, even assisted by any conceivable electronic computors, cannot predict the future. All it can do is to map out the probability space as it appears at the present and which will be different tomorrow when one of the infinity of possible states will have materialized. Technological and social inventions are broadening this probability space all the time; it is now incomparably larger than it was before the industrial revolution—for good or for evil.
The future cannot be predicted, but futures can be invented.
It was man’s ability to invent which has made human society what it is. The mental processes of inventions are still mysterious. They are rational but not logical, that is to say, not deductive.
Social media has given everyone a virtual megaphone to broadcast every thought, along with the means to filter out any contrary view [...] The result is a creeping sense of isolation and emptiness, which leads people to swipe, tap, and click all the more. Digital distraction keeps the mind occupied but does little to nurture it, much less cultivate depth of feeling, which requires the resonance of another’s voice within our very bones and psyches.
Moravec's paradox is the observation by artificial intelligence and robotics researchers that, contrary to traditional assumptions, reasoning (which is high-level in humans) requires very little ...
Almost always the men who achieve these fundamental inventions of a new paradigm have been either very young or very new to the field whose paradigm they change. And perhaps that point need not have been made explicit, for obviously these are the men who, being little committed by prior practice to the traditional rules of normal science, are particularly likely to see that those rules no longer define a playable game and to conceive another set that can replace them.