You are currently browsing the tag archive for the ‘existential risk’ tag.

Two academics from my university think so:

Australian astronomers say finding planets outside the solar system that can sustain life should be made a top priority.

Dr Charley Lineweaver and PhD student Aditya Chopra from ANU have reviewed current research into environments where life is found on earth and the environments thought to exist on other planets.

They say understanding habitability and using that knowledge to locate the nearest habitable planet may be crucial for our survival as a species.

While I agree that in the long run space colonisation is central to humanity’s survival, this is not really sensible and is probably a misrepresentation of their research. We are far away from being able to establish self sustaining colonies on planets in our solar system let alone travelling to other star systems. By the time we have the technology to contemplate doing that we will long since have identified habitable planets without having gone out of our way to do so.

While space colonisation would help reduce the risk of human extinction the unfortunate reality is that the technologies that threaten to ruin us are going to come well before independent, robust and self-sustaining colonies are possible. Risky technologies like mind uploading or machine intelligence are probably prerequisites for colonising other star systems and maybe even long-term survival on Mars. Slowing the development of the most risky technologies, controlling their use, and developing safe havens on Earth itself are likely to be more cost effective strategies than space travel for the foreseeable future.

Nick Bostrom of the Future of Humanity Institute has a new interview in The Atlantic. It’s one of the more sophisticated discussions of existential risk I’ve seen in the mainstream press and is worth sharing and reading in full.

One possible strategic response to human-created risks is the slowing or halting of our technological evolution, but you have been a critic of that view, arguing that the permanent failure to develop advanced technology would itself constitute an existential risk. Why is that?

Bostrom: Well, again I think the definition of an existential risk goes beyond just extinction, in that it also includes the permanent destruction of our potential for desirable future development. Our permanent failure to develop the sort of technologies that would fundamentally improve the quality of human life would count as an existential catastrophe. I think there are vastly better ways of being than we humans can currently reach and experience. We have fundamental biological limitations, which limit the kinds of values that we can instantiate in our life—our lifespans are limited, our cognitive abilities are limited, our emotional constitution is such that even under very good conditions we might not be completely happy. And even at the more mundane level, the world today contains a lot of avoidable misery and suffering and poverty and disease, and I think the world could be a lot better, both in the transhuman way, but also in this more economic way. The failure to ever realize those much better modes of being would count as an existential risk if it were permanent.

Another reason I haven’t emphasized or advocated the retardation of technological progress as a means of mitigating existential risk is that it’s a very hard lever to pull. There are so many strong forces pushing for scientific and technological progress in so many different domains—there are economic pressures, there is curiosity, there are all kinds of institutions and individuals that are invested in technology, so shutting it down is a very hard thing to do.

What technology, or potential technology, worries you the most?

Bostrom: Well, I can mention a few. In the nearer term I think various developments in biotechnology and synthetic biology are quite disconcerting. We are gaining the ability to create designer pathogens and there are these blueprints of various disease organisms that are in the public domain—you can download the gene sequence for smallpox or the 1918 flu virus from the Internet. So far the ordinary person will only have a digital representation of it on their computer screen, but we’re also developing better and better DNA synthesis machines, which are machines that can take one of these digital blueprints as an input, and then print out the actual RNA string or DNA string. Soon they will become powerful enough that they can actually print out these kinds of viruses. So already there you have a kind of predictable risk, and then once you can start modifying these organisms in certain kinds of ways, there is a whole additional frontier of danger that you can foresee.

In the longer run, I think artificial intelligence—once it gains human and then superhuman capabilities—will present us with a major risk area. There are also different kinds of population control that worry me, things like surveillance and psychological manipulation pharmaceuticals.

If I wanted some sort of scheme that laid out the stages of civilization, the period before machine super intelligence and the period after super machine intelligence would be a more relevant dichotomy. When you look at what’s valuable or interesting in examining these stages, it’s going to be what is done with these future resources and technologies, as opposed to their structure. It’s possible that the long-term future of humanity, if things go well, would from the outside look very simple. You might have Earth at the center, and then you might have a growing sphere of technological infrastructure that expands in all directions at some significant fraction of the speed of light, occupying larger and larger volumes of the universe—first in our galaxy, and then beyond as far as is physically possible. And then all that ever happens is just this continued increase in the spherical volume of matter colonized by human descendants, a growing bubble of infrastructure. Everything would then depend on what was happening inside this infrastructure, what kinds of lives people were being led there, what kinds of experiences people were having. You couldn’t infer that from the large-scale structure, so you’d have to sort of zoom in and see what kind of information processing occurred within this infrastructure.

It’s hard to know what that might look like, because our human experience might be just a small little crumb of what’s possible. If you think of all the different modes of being, different kinds of feeling and experiencing, different ways of thinking and relating, it might be that human nature constrains us to a very narrow little corner of the space of possible modes of being. If we think of the space of possible modes of being as a large cathedral, then humanity in its current stage might be like a little cowering infant sitting in the corner of that cathedral having only the most limited sense of what is possible.

Earlier today I had the pleasure of a long Skype with Seth Baum about existential risk and how I could best contribute to reducing it. Among other things, Seth studies climate change as a global catastrophic risk at Colombia University. He is taking it on himself to work to help network people studying different aspects of global catastrophic risks across universities, governments and the private sector. He does not accept Nick Bostrum’s quip that “there is more scholarly work on the life-habits of the dung fly than on existential risks.” According to him there is a lot of research on some existential risks, in particular nuclear war and disease pandemics – it is just not organised into a cohesive literature on ‘global catastrophic risk’ as such. One of Seth’s goals is to connect people studying these risks and working in related fields in order to encourage them to study the characteristics and possible solutions they have in common. He is organising the global catastrophic risk symposium at the World Congress on Risk 2012 in July which I am looking forward to attending. Think about coming as well if you will be in the area.

He shared links to a number of organisations that were new to me which I thought I would pass along.

Seth and his colleague Tony Barrett are founding a new organisation, the Global Catastrophic Risk Institute. Their hope is to evaluate which existential risks are most important to focus on, and which techniques are most likely to succeed at reducing them, for instance stockpiling food or building bunkers. Unlike GiveWell they will be rating strategies rather than organisations. The effectiveness of different approaches presumably varies by orders of magnitude, so this is incredibly important work. It will be a useful guide to those who become concerned about global catastrophic risk and make a big difference to the universe. Sister organisations, Blue Marble Space and One Flag in Space aim to promote space colonisation in order to reduce the risk of human extinction and conflict between nations, among other reasons.

A similar organisation is Saving Humanity from Homo Sapiens which is attempting to link donors concerned about existential risk with organisations that can most effectively use extra funding. This will hopefully in the future also involve evaluating their effectiveness.

Skill Global Threats Fund is a charitable foundation aiming to support those dealing with a range of catastrophic risks such as climate change and nuclear war. It’s goal is to “work proactively to find, initiate, or co-create breakthrough ideas and/or activities that we believe will have large-scale impact, either directly or indirectly, and whether on cross-cutting issues or individual threats.”

The Tellus Institute engages in future scenario mapping, including potential collapses of humanity and growth into post-human or space-faring civilizations. The paper Great Transitions is an example, though I am yet to read it.

The UPMC Biosecurity is a leading research organisation on catastrophic biosecurity threats. The Cultural Cognition project at Yale is moving into studying duel use problems in technology, including Nanotechnology Risk Perception.

Finally, if you haven’t checked out Nick Bostrom’s personal site then you really should. He has some excellent papers on existential risk, among other futurist issues. I hope to blog about some of the highlights in the near future.

One of the big challenges in life is finding friends and colleagues who support and bring out the best in you. This is especially the case when your goals are unusual and you are less likely to be assisted by the people you already know. I expect that the lack of social networks and esteem are important reasons why few people invest many of their resources into effectively improving the world, even when convinced on a philosophical level that it is the right thing to do. The supportive social networks are so much larger for ineffective forms of altruism that it is no surprise many more people are drawn to them.

For that reason I am very happy to see the growth of the 80,000 Hours organisation, which is a part of the Centre for Effective Altruism. They are collecting people who are researching and writing about how to most effectively make a difference, encouraging folk to commit to doing good, and then providing support for them the meet their giving goals. Their mission statement is to “help you be an effective altruist by providing a supporting community of dedicated members who share their insights and experience.”

Similarly, whatever their imperfections, it is very valuable to have the Less Wrong community and Singularity Institute as magnets for people who are concerned with existential risk, clear thinking and the future, who might otherwise give up from the lack of anyone to talk and work with.

If you haven’t thought much about how you can have a big positive impact on the world, watch the short video below and then the longer presentation underneath and consider taking the 80,000 Hours pledge. By giving a bit of thought to how you can best contribute you can do a lot more good for others at no extra cost to the rest of your life.

At least the Japanese can video conference when their bullet trains break

In my last post I discussed the short and medium run effects of increased trade and specialization on a society’s robustness. What is the effect over decades?

Complexity through new technology and specialisation, enabled by interpersonal and international trade, increases the productivity and productivity growth in an economy. Changes in productivity accumulate exponentially over time so in the long run they dominate any other impact. There are two ways this greater productivity can be used: in a purely Malthusian economy (which would include all of the world pre-1800 and the poorest parts of Africa today) all of the extra efficiency goes towards increasing population density while incomes remain roughly constant; in a modern post-Malthusian economy it mostly goes towards making each person richer.

When complexity results in greater population density and unchanged near-subsistence incomes, a society may be more or less vulnerable to collapse than it was before. A disturbance of similar magnitude is just as harmful for each person. The greater number of inputs makes disturbances more frequent, but the larger size of the economy allows greater diversification which is a buffer against some kinds of industry specific shocks. A larger economy can come up with more innovations as there are more people to do experiments and solve new problems. In the views of some historians complexity in subsistence income societies has resulted in the collapse of many large and sophisticated empires through history, so this is some evidence that the net effect is towards greater fragility. If our concern is not the harm a disaster inflicts on each person or the probability of a society collapsing, but rather population falling below an absolute number, which is the case when we worry about human extinction, a higher population is surely better. A disaster has to be much more severe to take the world’s population from 7 billion to zero than from 100 million to zero.

When a society uses complexity to make each person richer it is also unclear clear if it is more vulnerable than before. Though reliance on many inputs would suggest any single good or service is more likely to be disrupted, each person has a bigger ‘buffer zone’ before their income falls low enough to threaten their survival. If every person in Australia found their income halved tomorrow, we could continue surviving comfortably; when the same happened to countries in 1750 were halved, famine, riots and a cascading collapse of law and order were frequent outcomes. A rich society has other advantages that make it more robust. It can afford to stockpile more goods for security or add redundancy to any supply chain to make disruptions less frequent. It will also have more capital, idle labour and land which it can potentially reallocate to make more of anything it is struggling to obtain. A rich society, like a high density society, will have a greater capacity for innovation and problem solving and will foster some people who specialise in that task specifically. As in the Malthusian case, a richer society produces more kinds of goods across more places and sometimes in more varied ways, and this diversification makes it more resistant to shocks to any specific process. As modern industrial society is the first post-Malthusian civilization, the fate of historical empires may not have so much to teach us about the impact of complexity on robustness today.

The picture isn’t all rosy. Though more resources are available to cover for any problems, it is possible that the more complex production techniques familiar to rich countries are hard to scale up (or down) in the short term, and the specialised skills and machines found in complex economies may not be as easily reallocated to different tasks as more basic ones. The time necessary to create the physical and human capital necessary to open a nuclear power plant is probably greater than that for a traditional coal plant; a shovel can be applied to a greater range of tasks than a dental drill. The skills and technologies found in rich societies may also be less adapted to disaster scenarios than those found poorer ones. For example, nobody I know would be able to grow all their own food.

Though there are things about modern societies which make them more robust and others which make them less robust, I think the overall movement is clearly towards robustness. At a guess, rich societies today even with their 20 varieties of mobile phone charger and ‘just in time’ supply chains, are more resilient in the fact of disasters than any others in history and they will become more so the more complex and rich they get. Complexity in the financial system probably contributed to the 2008 financial crisis but after decades of productivity gains a recession is much less painful now than in the 1930s. Then many people went without food, today people tough it out without Wiis. At the other end of the spectrum, any very crowded and poor country which relies on complex technology to get its necessities is probably the most vulnerable to disaster in history.

Related thought from George Monbiot. He is more pessimistic than me.

In general as a production process gets more complex and requires many specialised and non-substitutable inputs, it is more vulnerable to disruption. This is proposed as a cause of collapse for many sophisticated empires throughout history. The economic fallout from the Iceland volcano fiasco got me wondering: does trade follow the rule that complexity leads to fragility, and if so how can we reduce that?

Short run effects

There is a compensating benefit to trade which increases stability which complicates the situation: unexpectedly low production in one place can be made up by unexpectedly high production in another. To give us some context I’ll consider robustness of food supply, one of the most important production and distribution processes for a society’s stability. For simplicity I’ll imagine three kinds of disruption: local crop failure (a local 30% output shortfall), global crop failure (a randomly distributed 30% output shortfall globally) and a halving of trade (from war, protectionism or natural disaster for example).

Self-sufficiency but no trade: Imagine a world where there is no trade in food between regions. Each region has to aim to produce enough food to feed itself. Each region has to worry about a bad season and regional output can be very variable and if they don’t produce enough food, they will starve. Given this they will aim to produce more food than they need and stockpile lots of grain in order to make a famine very unlikely. They are nonetheless more vulnerable to local shocks than if they could buy food from elsewhere in these emergency scenarios. A global shock has the same impact. However such a situation is not at all vulnerable to trade disruption as nobody relies on trade.

Self-sufficiency and trade: Imagine that in this world trade was suddenly opened up so that all regions could trade with one another. Initially production patterns do not change, so each region is still dedicating the same resources to food production and has the same distribution of expected outputs. In the case of a global supply shock, trade will help a little if a region can get by for a while on a low food supply; those regions with an especially bad crop failure can buy from those regions with a more mild failure. But local crop failure can now be covered with imports from other regions which had bumper crops that year. Vulnerability to trade disruption is no greater now because no region relies on trade except when there is a local crop failure and previously they would have been ruined in that situation anyway. This situation is much more robust than the previous one

No self-sufficiency but trade: Now over time people adapt to this new trade and some regions start producing a lot of food and others start producing less food. In a market system this concentration and specialisation will be based on comparative advantage. Local crop failure is only an issue when trade also fails. Assessing vulnerability to global failure is hard. The total global expected output and stockpiles of food would probably go down a bit over this time. The ‘law of large numbers’ means less excess capacity or stockpiled food is necessary to keep the same risk of famine as existing in scenario 1 because total global food production is less variable than local food production. However, specialization between regions makes food cheaper making stockpiling cheaper and allows us to produce most food in the regions with the least variable output. Compared to self sufficiency and trade we are more vulnerable to trade disruption.

Local crop failure

Global crop failure

Trade disruption

Self-sufficiency but no trade



Not vulnerable

Self-sufficiency and trade

Not vulnerable

Vulnerable (but less?)

Not vulnerable

No self-sufficiency but trade

Not vulnerable

Vulnerable (?)


As we can see, self sufficiency and trade is unambiguously the most robust of the three options, but it is ambiguous which of ‘self-sufficiency but no trade’ and ‘no self-sufficiency but trade’ is the worst. That will depend on how frequently local crop failures, global crop failures and trade disruptions occur and how harmful each is.

What can we do?

If we can have both lots of trade and some back-up systems appropriate for a non-trade world we will be especially robust.

A subsidy for local production to make you less reliant on trade, but open trade when local output falls short is a possibility, though it would be costly and useless against global failures as other countries will proportionally reduce their production. Robin Hanson proposes among other things that previously agreed prices be allowed to rise in emergencies, which gives private parties a reason to maintain excess and robust local production capacity if a supply shock seems likely. The more lucrative opportunities to price gouge during disasters a society commits to having, the more robust it will be.  The subsidy of stockpiles which are only released in a disaster would increase robustness to all kinds of failure, even those where law and order break down. To avoid crowding out other stockpiles or more robust production methods, government stockpiles should commit to sell at high prices rather than give the stockpile away.

In my next post I’ll consider long run effects of trade and complexity on robustness and explain why restrictions on trade or any other policy which reduces productivity growth would be counterproductive.

Reaching The Stars Is Easy Compared to Some Things, from Philhellenes. Very nicely put together. HT Alexander Kruel.

“Since, in the long run, every planetary society will be endangered by impacts from space, every surviving civilization is obliged to become spacefaring — not because of exploratory or romantic zeal, but for the most practical reason imaginable: staying alive.” – Carl Sagan

Exciting breaking news from the LHC!

Past reasons:

Carl Sagan on Earth

Why existential risk is the most important thing

Reasons to care

Unsurprisingly given our psychology’s origin in evolution, humans spend most of their time thinking about everyday concerns: how to get food, stay clean, find friends, get laid, etc. Most of our thinking and talking about far away issues we don’t have much control over is just for signalling nice things about ourselves. There is little reason to direct those efforts towards the things which really matter most as our views change nothing; instead it’s safest to go along with the idealistic fashions of our social group at any point in time.

Unless you’re really smart. In that case, you can go out and show just how brilliantly smart you are by forwarding strange positions no mediocre wit would feel smart enough to defend. Nick Bostrum, busy signalling his superior smarts with an unusual but consistent worldview, swims against the current of his day and proposes these fairly unusual answers to the most serious problems humanity faces: Death, Existential Risk, Suffering and Mediocre Experiences. If you knew you were going to (have the chance to) be born again in the year 3000, I think these are just the issues you would want us to start dealing with seriously now, not most of the nonsense we ostensibly do to help the future. Or you could just save some money (PPT) for them instead, if you care.

Lucky we have some really smart people: to show us how smart they are, sometimes they go out and say really outlandish but important things.

A new paper from one of the editors of Global Catastrophic Risk:

“Another partial solution of Fermi’s famous paradox is proposed, based on our increased understanding of geophysics, geo-engineering and climatology. It has been claimed in the recent astrobiological literature (for instance, in the recent controversial “rare Earth” theory of Ward and Brownlee), that geological activity of a terrestrial planet is an important precondition for the emergence of complex metazoan life forms. Technological civilizations arising on such planets will be, at some point of their histories or another, tempted to embark upon massive geo-engineering projects. If, for some reasons only very recently understood,large-scale geo-engineering is in fact much more dangerous than previously thought, the scenario in which at least some of the extraterrestrial civilizations in the Milky Way self destructin this manner gains plausibility. In addition, we speculate on possible reasons, both physical and culturological, which could make such a threat even more pertinent on an average Galactic terrestrial planet than on Earth.

We hereby propose an apocalyptic scenario of greenhouse gases vented from the Earth’s crust and mantle because humans undertook Stevenson’s MTEC macroproject,as a prototype of these additional existential risks. Since humans are currently without means of escape from the planet (the state-of-affairs likely to persist for at least several decades and probably much longer), and their Macro-engineering experiment meets with misfortune, they will perish as a species due to the excessive heat possibly Venuforming the Earth, i.e. permanent locking in the runaway greenhouse heating. Of course, it is possible that the Earth-biosphere continues to function and that subsequently, owing to the atmosphere’s increased perennial warmth10 and the aerial fertilization of expelled CO2, vegetation will be caused to flourish. Surface life forms must thereafter endure much different solar ultraviolet radiation (UVR): under heavy cloud cover the scattered UV component of sunlight—often termed “skylight”—is seldom less than 10% of that under a clear, blue sky; very heavy storm clouds canal most eliminate terrestrial UVR even in summertime.”

And here is a much less wildly speculative reason to worry about sudden climatic changes:

“A section of the Arctic Ocean seafloor that holds vast stores of frozen methane is showing signs of instability and widespread venting of the powerful greenhouse gas, according to the findings of an international research team led by University of Alaska Fairbanks scientists Natalia Shakhova and Igor Semiletov. The research results, published in the March 5 edition of the journal Science, show that the permafrost under the East Siberian Arctic Shelf, long thought to be an impermeable barrier sealing in methane, is perforated and is leaking large amounts of methane into the atmosphere. Release of even a fraction of the methane stored in the shelf could trigger abrupt climate warming.”

I haven’t had a chance to watch it yet but the first few minutes of Geoengineering: Global Salvation or Ruin? looked good.

Enter your email address below to receive new posts by email.

Join 50 other followers

Robert WiblinHi! I am a young Australian man ostensibly interested in the truth and maximising the total number of preferences that are ever satisfied, weighted by their intensity. I also enjoy reading and writing about the topics listed above. If you share my interests, friend me on , , or or subscribe to my RSS feed .

All opinions expressed here are at most mine alone, and have nothing to do with any past, present, future or far future employers.

Some popular posts:

The lives you could have saved
How feasible is a ‘charter city of refugees’ in Australia?
NEWS FLASH: multiverse theory proven right
Choosing the best status games for society
Eat cows to save mice? Hold your horses!
Beeminding your way to greatness
Should you floss: a cost benefit analysis
What should we do about wilderness?

Twitter Updates

Past Posts


Get every new post delivered to your Inbox.

Join 50 other followers

%d bloggers like this: