The Lessons of Theranos

I recently read Bad Blood by John Carreyrou, the Wall Street Journal reporter who broke the story of the massive fraud that brought down Theranos, the high-flying Silicon Valley company founded by Elizabeth Holmes.

The story is relatively familiar and easily found via an online search of “Theranos” or “Elizabeth Holmes,” other than to note that what she claimed to have, a new, breakthrough, proprietary system for doing over 1,000 diagnostics with a single drop of blood, was never achievable.   Bad Blood added more context to the story we have followed on mass media, and is well worth reading.

What initially triggered my curiosity to read the book was how Holmes could have fooled so many ultra-sophisticated investors, Board members, and large-company customers, as well as exceptionally talented employees and kept the story from emerging earlier than it did.  The first evidence of fraud was discovered by Holmes’ CFO in 2006, but the first story that brought her down was published in late 2015.

She secured over $700 million of investor money. At its high, her company had a $9 billion valuation.  She owned more than half the stock and was heralded as a uniquely talented entrepreneurial woman, a billionaire on paper.  In 2018, after she was indicted for fraud against patients sand after settling with the SEC, the company was shut down.

Elizabeth Holmes was uniquely brilliant in telling her story, keeping as many people as possible away from the facts that could have exposed the fraud, intimidating anyone who attempted to challenge her, and making persuasive, reassuring statements.

I am less interested in this part of the story, because nearly every proven fraud has at its center a brilliant sociopath.  Of greater interest to everyone is what it reinforced about our deeply inadequate knowledge of the complexity of blood testing and its vital health-monitoring role.

Think of the people who clearly either did not understand the basic flaws in her blood testing methodology: Rupert Murdoch (who invested $125 million in her company), Henry Kissinger, General James Mattis, former Secretary of State George Schultz, reporters who are usually quite knowledgeable about medical devices and technologies, and those who gave her a prominent TED Talk speaking slot in 2012.

She also secured partnership agreements with Walgreens and Safeway.  I knew the CEOs of those companies at the time, Greg Wasson of Walgreens (a pharmacist by training) and Steve Burd of Safeway.  Each of them invested significant time and money to prepare for a launch of her blood analysis hardware and processes in their retail stores.  Steve Burd, a CEO who cared deeply about employee and public health, may have lost his job because of failing to achieve his company’s earnings targets due to the hundreds of millions of dollars Safeway invested in the partnership.

What did all these people fail to understand about blood testing?

• A single drop of blood is wildly inadequate to perform most diagnostic analyses we secure from blood tests.

A properly-captured, processed, stored, and analyzed drop of blood of sufficient size may have sufficient volume to detect the presence or absence of a chemical element or biological substance.  It cannot reliably determine the percentage or quantity of that element or substance because the sample size is too small.  A statistical principle explained to anyone who takes a basic statistics course or reads polling or survey data is that different sample sizes produce different confidence levels that the data captured is accurate within a certain range.

For example, when we hear or read that a politician has a 45% approval rating, the pollster will state that 1,000 people have been polled and that there is a high confidence level that the true approval rating for the much larger relevant population is accurate within a relatively narrow range.  At a much smaller sample size, that range might be 35-55% or even 25-65%.  The smaller the sample, the wider the error range.

Since doctors need precise measurements of many chemical elements in the blood, a sample size of one drop is useless to detect quantities or percentages of an element or substance, because the error range is so great.  The notion that a single drop of blood, even one perfectly captured, stored and analyzed, could be separately useful for the hundreds of different precise metrics on which clinicians rely is wildly unbelievable.

In fact, the more blood elements clinicians want to test and analyze, the larger the sample that has to be drawn.  That is why very complete physical examinations often cause multiple vials of blood to be drawn and analyzed.  The analytic equipment is getting better and less blood is necessary to get reasonably accurate readings.

• To do accurate blood analyses, enough blood has to be captured and separated into its component parts.

As noted in the website shown below, blood analyses require separation of blood into components.

“Blood is separated in bags and blood tubes into its various components:

• Red Blood Cells (RBC), erythrocytes, carries oxygen from the lungs to the rest of the body.
• White blood cells (WBC), or leucocytes, aid the immune system in fighting infections. There are five types of WBC: lymphocytes, monocytes, eosinophils, basophils, and neutrophils (granulocytes).
• Plasma is a yellowish component in which the blood is normally suspended. Plasma contains proteins, glucose, clotting factors, hormones, and carbon dioxide.”

Each of these components has to be analyzed separately. To do even basic blood analysis with a small number of drops of blood, the fluid has to be injected into the blood to build up to a minimum fluid volume. As a result, blood separation and component testing accuracy deteriorate as a result.

https://www.hettweb.com/blood.html

• Drawing blood and placing it in any blood analyzer for further analysis is a highly-skilled operation that even experienced professionals sometimes get wrong.

The notion that millions of Americans could buy kits to capture individual drops of blood, perfectly follow instructions on placing them immediately in an analyzer, and get precise diagnostic data is also unbelievable.  Even more unbelievable was Holmes’ claim that these results could be reliably and quickly communicated over the Internet to a central server for precise analysis.

Professional phlebotomists (those who draw blood for a living) sometimes get results wrong, and they are drawing much larger samples in skilled laboratory environments designed for proper blood draws and analyses.  These labs are regulated by the Food and Drug Administration and, in many states, must meet even higher certification standards.

The notion that this level of quality control could be reached in tens of millions of homes in which people not trained to draw blood are responsible for drawing it was ludicrous.  Home-originated blood samples can detect the presence of a substance when the substance is present in every bit of a blood sample; it cannot reliably detect quantity or percentage composition, metrics far more important for most clinical analyses.

• Blood proteins and other elements are often inherently unstable and have to be analyzed extremely rapidly to be accurate.

One reason we are seeing the emergence of DNA analyses from bodily fluid collections for research and analysis, but no comparable collections of blood, is that DNA is highly stable, permanent and discoverable in small human cell volumes, as we know from watching detective shows like CSI and Law and Order.  Many critical components of blood are unstable when the blood is drawn from the body.

The use of blood samples for large populations of people requires exceptionally difficult handling processes.  Many blood proteins deteriorate within a few minutes, if not properly stored in airtight and watertight temperature and humidity-controlled chambers.  This website details the temperatures at which various blood components must be stored to retain their stability.

https://rmlh.nic.in/index1.aspx?lid=552&lsid=558&pid=47&lev=4&langid=1

• Blood testing requires the person whose blood is being drawn to take certain steps prior to testing.

We are routinely told to fast for 8-12 hours before a blood test, because the failure to fast will alter the test results that matter. What this should impress on us is that blood test results can vary significantly and increase the magnitude of errors if there is less control of the behaviors of the individuals being tested before blood is drawn.

• Experienced clinicians are needed to interpret blood test results.

Some years ago, one of my physicians ordered a blood test.  When we reviewed the results, he immediately recommended a retest.  When he looked at the laboratory report, he said: “If this were an accurate reading, you would be in a wheelchair or on a stretcher.  You would not have the strength to walk around for more than a few seconds.”

One of the other pernicious actions Elizabeth Holmes took was to persuade the State of Arizona that consumers should be able to order blood tests without a clinician’s order.  Unfortunately, although we all believe that patients should be empowered to take care of their own health needs, their ability to understand a blood test or other complex diagnostic tests is extremely limited.

While offering patients the convenience of doing blood tests in their own homes on their own time is superficially appealing, Holmes did them a disservice by causing them to overreact to faulty results, either by paying for unnecessary additional diagnostic tests or visiting specialists, who instantly figured out that the blood test results could not be correct.

• Medical diagnostic processes are inherently complex.

For a decade, I was the Chairman and, later, the CEO of the Dossia Service Corporation, which was created by a consortium of CEOs who wanted to empower patients to have their own patient-controlled, lifelong, portable health records, and be able to provide a complete medical history to every physician.  We believed that not getting lab results directly downloaded into a patient health record was inconsistent with what HIPAA was all about: the patient’s absolute right to own all medical data and have it available on demand.

This book made me understand better why unfettered access to health records poses risks. The risks are manageable, but they exist and the best thing we could have done for patients was to guide them to secure relationships with single primary care physicians.  My friend Dr. Ray Fabius of HealthNext has told many audiences that the single most important factor in high-quality healthcare is not to be “medically homeless,” that is, not to be without a primary care physician relationship.  This book’s commentary on blood tests reinforced the wisdom of that recommendation.

Today, the healthcare system has a hopelessly fragmented, redundant, and inconsistent set of lifetime health records on each of us.  There is a great deal of unnecessary and, sometimes, dangerous and harmful, healthcare, because physicians do not have a complete record of our medications and our past medical diagnoses.  In a perfect world, every physician would have a complete and precise record that would yield consistent insights about us.  However, almost never is it the case that even our primary care physician knows everything he or she needs to know about us.

This book and the educational process it triggered for me make it clearer that have a deep and long-term relationship with a highly-capable primary care physician is more critical than ever.  Blood tests, as well as other diagnostic tools, are significantly more useful when the clinician evaluating them can put them in the much broader context of insights he or she has about a patient.

Diagnostic tests need to be part of that complete record, as do prescriptions, medical claims, and notes from clinicians.

If any good can come from the Theranos fraud, it should be a greater sensitivity to how much we need to know about the laboratory diagnostic tests on which we and our clinicians rely so heavily.