Three Paths to Existential Catastrophe from AI

[EDIT 6/17: A previous version of this post had the title “Three Paths to Existential Catastrophe from AI”. I changed it to more clearly convey my point] In the past, I have pointed out that AI risk is one of the most significant, near term problems we face as a society. It is particularly importantContinue reading “Three Paths to Existential Catastrophe from AI”

A Formula for the Value of Existential Risk Reduction

How important is existential risk reduction (XRR)? Here, I present a model to estimate the value of eliminating existential risks. Note that this model is exceedingly simple; it is actually contained within other studies of existential risk, but I feel the implications have not been fully considered. The Model Let’s imagine that, if we didContinue reading “A Formula for the Value of Existential Risk Reduction”

Thoughts on Existential Risk

Existential Risk is an important topic for long-termists. The idea being that since humanity has the potential to spread to the universe (or at least continue existing), any extinction event would prevent potentially trillions of happy lives from occurring. In other words, extinction has a high opportunity cost. Nick Bostrom refers to this as “AstronomicalContinue reading “Thoughts on Existential Risk”

Grey Goo Requires AI

Summary: Risks from self-replicating machines or nanotechnology rely on the presence of a powerful artificial intelligence within the machines in order to overcome human control and the logistics of self-assembling in many domains. The grey goo scenario posits that developing self-replicating machines could present an existential risk for society. These replicators would transform all matterContinue reading “Grey Goo Requires AI”