Estimating the Unknowable: An interview with GiveWell's James Snowden

Estimating the Unknowable: An interview with GiveWell's James Snowden

From an 80,000 Hours Podcast By Rob Wiblin

You take the number of expected years of life left and you multiply it by this pretty subjective factor, which accounts for… does this person have cognitive function? Can they make plans? Are they a functioning agent in the world? You multiply those two things together and end up with a roughly log-normal distribution over age.

James Snowden"

What’s the value of preventing the death of a 5-year-old child, compared to a 20-year-old, or an 80-year-old?

The global health community has generally regarded the value as proportional to the number of health-adjusted life-years the person has remaining – but GiveWell, one of the world’s foremost charity evaluators, no longer uses that approach. They found that contrary to the years-remaining’ method, many of their staff actually value preventing the death of an adult more than preventing the death of a young child. But there’s plenty of disagreement, with the team’s estimates spanning a four-fold range.

As James Snowden – a research consultant at GiveWell – explains in this episode, there’s no way around making these controversial judgement calls based on limited information. If you try to ignore a question like this, you just implicitly take an unreflective stance on it instead. And for each charity they investigate there’s 1 or 2 dozen of these highly uncertain parameters that need to be estimated.

GiveWell has been working to find the best way to make these decisions since its inception in 2007. Lives hang in the balance, so they want their staff to say what they really believe and bring whatever private knowledge they have to the table, rather than just defer to their managers, or an imaginary consensus.

Their strategy is to have a massive spreadsheet that lists dozens of things they need to know, and to ask every staff member to give a figure and justification. Then once a year, the GiveWell team gets together to identify what they really disagree about and think through what evidence it would take to change their minds.

Often the people who have the greatest familiarity with a particular intervention are the ones who drive the decision, as others choose to defer to them. But the group can also end up with very different answers, based on different prior beliefs about moral issues and how the world works. In that case then use the median of everyone’s best guess to make their key decisions.

In making his estimate of the relative badness of dying at different ages, James specifically considered two factors: how many years of life do you lose, and how much interest do you have in those future years? Currently, James believes that the worst time for a person to die is around 8 years of age.

We discuss his experiences with doing such calculations, as well as various other topics:

  • Why GiveWell’s recommendations have changed more than it looks.
  • What are the biggest research priorities for GiveWell at the moment?
  • How do you take into account the long-term knock-on effects from interventions?
  • If GiveWell’s advice were going to end up being very different in a couple years’ time, how might that happen?
  • Are there any charities that James thinks are really cost-effective which GiveWell hasn’t funded yet?
  • How does domestic government spending in the developing world compare to effective charities?
  • What are the main challenges with policy related interventions?
  • What are the main uncertainties around interventions to reduce pesticide suicide? Are there any other mental health interventions you’re looking at?
  • How much time do you spend trying to discover novel interventions?

To hear or read the full interview, click here.

Robert Wiblin
Robert Wiblin
Director of Research at 80,000 Hours I do research into the best ways to do good in your career for 80,000 Hours. I also host the 80,000 Hours podcast on 'the world's most pressing problems and how you can use your career to solve them'. Here's some of my written work at 80,000 Hours. Here's some of my past writing on the: Overcoming Bias blog, Giving What We Can blog, and Effective Altruism Forum.
The views expressed in blog posts are those of the author, and not necessarily those of Peter Singer or The Life You Can Save.

Comments

What will your impact be?

Find out using our Impact Calculator.

CALCULATE

What's the most effective way to fight extreme poverty?

Stay informed with our latest news!

Yes, I would like to subscribe to your newsletter and receive further emails with your latest news. I understand that I can unsubscribe at any time and I agree to your privacy policy.