“The Choice Transition” by Owen Cotton-Barratt, Raymond D

This is a link post. On the emergence of history's reins One general law, leading to the advancement of all organic beings, namely, multiply, vary, let the strongest live and the weakest die. ― Charles Darwin We are a way for the universe to know itself. — Carl Sagan If you don’t know where you are going, you’ll end up someplace else. — Yogi Berra Imagine you were trying to explain how the world worked 10 billion years ago. Back then, the best explanation would be in terms of physics: galaxies forming and supernovas producing heavy metals. Ten million years ago, though, you’d talk about evolution: plants, mammals, and early hominids. Ten thousand years ago, when agriculture was being established, you might talk about culture and the spread of ideas. Each of these forces — physics, evolution, and culture — gave rise to the next, producing [...] ---Outline:(00:07) On the emergence of historys reins(04:15) The forces shaping history(05:56) The blind forces(07:48) The anthropocene(12:49) The ascent of choice(14:45) Is a Choice Transition inevitable?(16:10) What the Choice Transition is not(16:40) It doesn’t mean omnipotence(17:18) It needn’t mean a single chooser(18:12) It needn’t mean the end of new forces(19:04) AI and the Choice Transition(19:44) 1) Better foresight capabilities and understanding enable steering(20:21) 2) Better coordination capabilities could allow for more coherent steering(22:20) 3) Agentic AI systems could be (among) the entities steering(23:20) 4) Advances in AI could lead to centralization of power(25:03) 5) AI might empower forces that squeeze out deliberate steering(27:22) 6) Automation of research means it might all happen quickly(28:35) The space of possible Choice Transitions(30:27) Acknowledgements(31:05) Appendix: comparison to existing frames(31:11) Comparison to normal AI x-risk frames(33:30) Related concepts(33:34) Bostroms notion of a Singleton(33:54) Yudkowskys notion of a Pivotal Act(34:09) Finnveden, Riedel, and Shulman's notion of lock-in(34:33) Ords notion of the Long Reflection(34:54) Drexler's notion of Paretotopia(35:10) Carlsmiths notion of yang(35:38) Buterin's notion of d/acc(35:59) Alexander's notion of MolochThe original text contained 22 footnotes which were omitted from this narration. The original text contained 4 images which were described by AI. --- First published: November 18th, 2024 Source: https://forum.effectivealtruism.org/posts/SrjrytewFfvZ8pCm4/the-choice-transition --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.