🛑 How not to work with data?

Let me tell you a story about how - alone - a smart man has saved millions of life thanks to data analysis.

Most of the info of this blog post came from Jeremy N. Smith’s outstanding book: Epic Measures that you can buy here (most recommended!): https://amzn.to/2LcXMhV

WHO influences most of the international public health decisions ⚕️

World health organization (WHO) is a specialized agency of the United Nations whose primary aim is “the attainment by all peoples of the highest possible level of health.”. This organization advises ministries of health on technical issues and provides help in scaling up essential prevention, treatment, and care services throughout the health sector.

WHO has a tremendous impact on how to measure health and how and where nations fight against diseases (Malaria, Polio...). WHO is making (almost) all the shots in international public health: what disease to fight against, where are the most preoccupying situations, and where goes the budgets!

In the 1980s, WHO used incomplete indicators

Let’s look at their key indicators at the time: Infant-mortality rate, life expectancy, and the number of death.

Infant-mortality rate 🤰

Needless to say, looking at the infant-mortality rate is important, but if you focus only on this indicator, it means nobody watches at devastating diseases affecting people after their first months of life.

Life-expectancy 👴🏼

Life expectancy doesn’t completely fill the gap as a healthy person can live to be 80 years old, but so can someone who spends most of his life sick as hell.

Number of death 💀

Simply counting the number of deaths in an area is insufficient as well, since it leaves causes out of the equation - an infant who dies of malnutrition is not the same deal as a 90-year-old who dies in his sleep.

WHO used poor data quality 👎

All indicators relied on the worst kind of raw data source possible: declarative data written on forms... by doctors (we all know their hand-writing sucks, look at your last prescription if you aren’t convinced) treating patient difficult to communicate with. Around the 1970s, in Africa, around 3 out of 4 people were illiterate. In Asia, illiteracy affected almost half of the population. Source: UNESCO.

WHO used unclear methods

From these questionable raw data, WHO didn’t define best practices, blueprints, and unique shared protocols to measure indicators.

For example, there were not less than 5 different approaches to measure life-expectancy... which resulted in uncertainty in results (15 years span for the same country).

WHO’s organization was perfectible 🌐

WHO was divided into many teams with a small scope of responsibility (a single disease for example, or a single area or a single treatment). What about subjects attributed to no one? Congenital diseases, Infant cancer, Death by fire, or drownings... 🤦‍♂️

There was no incentive for teams to work together or challenge their method. Worst, budgets and positions were related to the subject of work. So if your treatment was not efficient or your disease not scary enough, it might end up bad for your career. The competition for scarce resources may bias even well-intentioned professionals.

There was no central oversight 🔎, as a result of all, WHO’s statistics were terrible. For example, the sum of all infant deaths by disease was - way- superior to the total number of infant death all causes combined the UN accounted for (~ 10 million discrepancy!).

Policymakers didn’t have any way to measure the impact of their decisions

Traditional health statistics did not allow policy-makers to compare the relative cost-effectiveness of different interventions. At a time when people’s expectations of health services were growing and funds were tightly constrained 💸, such information was essential to aid the rational allocation of resources.

By default, people always assume their making the best decision possible. In my experience, most of the time, it's a wrong assumption.

How Christopher Murray saved the day?

Christopher Murray is an American researcher in global health and public health. Beginning in 1990, he has worked on ways to measure the burden of disease and disability around the globe. His goal is to provide policymakers a clear picture of world health and help them define their priorities.

New indicators 📊

During his quest to measure the burden of diseases, Murray defined new indicators to watch for:

  • YLL: years of life lost from premature death.
  • YLD: years of “equivalent healthy life” lost (YLD) through living in states of less than full health. On Murray’s initiative, a scale was built to rank illness by severity. 0 means illness had no impact on your global health, 1 means illness or injury lead to death. Experts from all over the world reached a consensus and positioned diseases (communicable and non-communicable diseases) and injuries on this scale. That way, you can have on your radar illnesses that aren’t fatal but shouldn’t be ignored in public health decisions as they reduce your life expectancy like loss of sight, loss of hearing, loss of a limb...
  • DALY: These two previous indicators are perfectly compatible. For example, if you had an illness ranked 0.5 it means you lose 5 years of life expectancy every 10 years. Statisticians and public health nerds could now count all the years lost to both premature deaths and non-fatal illnesses. It’s the disability-adjusted life year (DALY). The DALY naturally weights deaths at younger ages more heavily, but also explicitly included time discounting (of future years of life lost) and age-weighting (lower weight for younger and older years of life). 1 DALY is 1 lost year of healthy life.

You can play with visualize these indicators for 2019 here. It's published by the IHME whose director is no other than Christopher Murray.

GBD 2019 - Data visualisation

What Murray did expose in the early 1990s?

Murray’s team published the first Global Burden of Disease papers in 1993. Unsurprisingly, the results of the original GBD study were groundbreaking to a good deal of health policymakers.

Murray showed that:

  • The burdens of mental illnesses, such as depression, alcohol dependence, and schizophrenia, have been seriously underestimated. Especially throughout Asia, where mental health issues were way more problematic than malnutrition.
  • Tobacco 🚬 was expected to kill more people than any single disease, surpassing even the HIV epidemic (Remember, it was published in the early 1990s).
  • In Sub-Saharan Africa, simple dental problems 🦷 were as problematic as anemia.
  • In almost all regions, unintentional injuries were a much bigger source of ill-health than intentional injuries such as interpersonal violence and war.
  • For men aged 15-44, road traffic accidents 🚗 are the biggest cause of ill- health and premature death worldwide, whereas it has received relatively little attention from public health specialists in the past.
  • And many other things... You can read more here.

Everybody did not appreciate Murray’s work, but he set a track for them they gladly followed. Indeed, WHO makes publications following the Burden of Disease methods regularly to guide policymakers as well as other organizations.

Actionable advices for your business 🚀

  • Identify people from different fields - responsible for selecting key data and key metrics for your business.
  • Make sure methods are openly shared and incentivize people to challenge them.
  • Be aware of your data source flows and develop countermeasures.
  • Be aware of your organization structure and make sure it’s flexible so teams can switch focus without hurting people’s careers. The last thing you want is people becoming overprotective of their scope and developing bias regarding their impact.

Want to know more about Epic Measures? Find below the best conference from Jeremy Smith on the subject.

Paul-Mehdy M'Rabet

Paul-Mehdy M'Rabet

I help CIO and CDO from global manufacturers and industrial leaders to design ambitious roadmap their teams can actually deliver and I love it! I write about #ai, #data and #finance.