4.4 Small probabilities, big stakes

Most of the risks in this unit are unlikely in any given year. How should you weigh a small chance of something enormous?

The standard tool is expected value: multiply each possible outcome by its probability and add them up. Suppose you could choose between two projects:

• Project A is certain to help 100 people.
• Project B has a 10% chance of helping 10,000 people, and otherwise helps no one.

Project B's expected value is 10% × 10,000 = 1,000 people, ten times Project A's. If you could repeat the choice many times, B would do far more good overall, even though it fails nine times out of ten.

Some funders work this way on purpose. Coefficient Giving (formerly Open Philanthropy) calls it hits-based giving: make many bets that will probably fail, because a few large successes can outweigh them all. Venture capital works the same way. The intro course covers this in Week 3.

A related idea is the crucial consideration: a single fact or argument that, if true, would reverse your conclusion. When the stakes are large and the evidence thin, it is worth actively looking for these.

Where the case is weakest.

• Fragile numbers. When you multiply a tiny probability by a vast outcome, small changes to the probability swing the result enormously. Few of the probabilities in this unit are known with confidence.
• Pascal's mugging. If any large enough stake can justify any action, something has gone wrong. Most people who work on these risks accept that expected value needs guardrails.
• Can we actually help? It is hard to know whether extra money or effort reduces these risks, or just funds people to think about them.
• Is the future what matters? Denise Melchin, writing on the EA Forum, explains why she is probably not a longtermist. She cares about the future being good, "as opposed to long or big". She also doubts we can predictably influence events more than about a century away.