The Rush to Vaccinate: '95% Effective' May Not Mean What You Think It Means

By Thomas Neuburger

"Done right, vaccines end pandemics. Done wrong, pandemics end vaccines."

—Andy Slavitt here

People in the United States, along with people in all of the rest of the world, are eager for a vaccine that provides immunity to the Covid-19 virus. Drug manufacturers, with a market of tens of billions of injections to sell into, are eager to roll one off the production line. Both groups are highly incentivized to get a vaccine into distribution quickly.

Let's look at the revenue side first. Here, for example, is what the three leading vaccine candidates are projected to cost in the UK according to a recent Sky News piece:

In two years the earth is projected to hold 8 billion people, and most leading vaccine candidates require at least two doses. Let's be conservative: If Moderna, say, sold its Covid vaccine to 1 billion people at ₤28 (about $37) per dose, the revenue stream from those sales would turn into real money fast — $74 billion in revenue at retail prices in less than two years.

And that's for capturing less than a sixth of the global market. A vaccine manufacturer that captures a third of that market would swim in wealth till the climate crisis took us all.

For comparison, consider Moderna's recent revenue profile. For the last few years, Moderna income has run between $60 and $200 million per year. Revenue for just the last quarter, however, jumped to $158 million. Moderna is clearly set for a windfall.

Needless to say, something like $100 billion in revenue would more than cover the cost of Covid vaccine development, so why the high price retail prices? One can only guess.

How Effective Is "Effective"?

About effectiveness, much is claimed. From the Sky News article:

The UK has become the first country in the world to approve the Pfizer/BioNTech COVID-19 vaccine for use. The government says the jab [vaccine], which has been given the green light by independent health regulator MHRA, will be rolled out across the UK from early next week. Studies have shown the jab is 95% effective and works in all age groups. [emphasis added]

Moderna claims similar effectiveness — 94% — for its own vaccine candidate. But what does effectiveness mean?

To a lay person, a phrase like "95% effective" means one of two things: either that she or he, upon exposure to the virus, is protected 95% of the time, or that 95% of the people who take the vaccine are protected 100% of the time.

And this is where the mutual eagerness of the two highly motivated groups — the public; the profiteers — intersect. The public wants to hear "95% effectiveness" and think it knows what those words means. The drug companies want the same thing as the public; it wants the public to think it knows what those words mean.

But in the world of drug advertising, the word "effective" does not mean what you think it means. The other way to look at effectiveness is this: Based on the numbers released from phase 3 trials, the Pfizer vaccine is 95% effective, but 1% of the time. In the same way, the Moderna vaccine is 94% effective, but 2% of the time.

Relative Effectiveness

To sort this out, let's look at real numbers, thanks to Twitter friend David Windt.

For the Moderna product, the phase 3 trial contained 30,000 individuals divided between those given the vaccine and those given a placebo. Let's assume that individuals in each group were allowed to roam freely "in the wild" — that is, told to live their regular lives among the general population, including going out infrequently, staying masked, and practicing social distance — as opposed being proactively and aggressively exposed to the virus by the researchers, which would be highly immoral, to say the least.

In the Moderna vaccinated group, 11 people out of 15,000 got the virus (by Moderna's definition of what "got the virus means") for an overall infection rate of 0.07%. (There's disagreement about whether the drug company's "got the virus" measurements are well chosen; see the Forbes article "Covid-19 Vaccine Protocols Reveal That Trials Are Designed To Succeed." But we'll ignore that point for now.)

In the Moderna placebo group, 185 people of 15,000 got the virus, for an overall infection rate of 1.23%.

Do you see where this is headed? If you divide 0.07% by 1.23%, you get a 5.7% infection rate — or inversely, a 94% protection rate, which is what's claimed. But that's a percentage of a percentage, a ratio of a ratio, something called the "relative rate" in the medical profession. What this really means is that, of the 1.23% of people who would have gotten the virus in the vaccinated group, 94% of them didn't.

But Moderna isn't testing 30,000 people who are infected with the virus, or even 15,000 people. Only 185 people "got the virus" (by their definition) in the placebo group. That population was reduced to 11 people with vaccination. These are very small numbers. As stated above, the Moderna vaccine is 94% effective — but only 1.23% of the time.

(For another way to see that using a percentage of a percentage, or a ratio of a ratio, is confusing, consider an advertisement that claims a company's new product is "twice as effective" as its old one. If the old product was effective only 2% of the time, and you knew this, would you buy the new one?)

Infection rates in those clinical trials seem low, by the way, which could be just an accident of statistics, or something off in their way of measuring who is counted as infected. From the start of the pandemic until now, the overall disease rate for Maricopa County, a high-infection zone, is 5034 per 100,000 people, or 5%. At the lower end, the overall disease rate for Multnomah County, a less-infected but still urban county, is 2363 per 100,000 people, or 2.4%.

Both rates are higher than the infection rates of the Moderna and Pfizer placebo groups. As stated, Moderna's placebo group experienced a 1.23% infection rate, and Pfizer's placebo group was infected just 0.75% of the time.

None of this is to imply dishonesty on the part of the drug companies. Measuring "effectiveness" using the relative rate of infection is common in that world. It's just more meaningful when the overall infection rate of a pathogen is, say, 70% or higher, instead of 5% or less.

Absolute Effectiveness

For comparison, let's look at the absolute numbers from the Moderna test. In the unprotected population, 1.23% of the people who could have been exposed to the virus, got it. In the vaccinated population exposed to the same conditions, a little less than 0.07% got the virus. Subtracting the two, the absolute gain in protection was 1.16% — that is, taking the vaccine bought you a little over 1% in absolute protection.

The numbers for the Pfizer vaccine are similar. According to Windt, "the infection rate was reduced slightly, from 0.75% to 0.04% - that's "95% efficacy" [but] these results do NOT mean that 95% of those vaccinated are protected." In absolute terms, taking the Pfizer vaccine reduced the risk of getting the virus by just 0.71%.

Do you trust any of these drug manufacturers and their massively under-tested vaccines enough that you would take whatever risk is associated with their product to gain that amount of protection? I know good doctors who won't.

Testing and Public Trust

I want to point to two articles about testing and public trust. First from MIT in November, consider this caution about public trust:

Covid-19 vaccines shouldn’t get emergency-use authorization

Public trust in vaccines is already in decline. The FDA should proceed with caution.

...The pace of covid-19 vaccine research has been astonishing: there are more than 200 vaccine candidates in some stage of development, including several that are already in phase 3 clinical trials, mere months after covid-19 became a global public health emergency. In order for the FDA to approve a vaccine, however, not only do these clinical trials need to be completed—a process that typically involves following tens of thousands of participants for at least six months—but the agency also needs to inspect production facilities, review detailed manufacturing plans and data about the product’s stability, and pore over reams of trial data. This review can easily take a year or more.

Excessive haste could have fatal consequences, since public confidence and wide vaccination are the only ways any vaccine, even the best ones, can work: "Public health experts caution that vaccines don’t protect people; only vaccinations do. A vaccine that hasn’t gained enough public trust will therefore have a limited ability to control the pandemic even if it’s highly effective." [emphasis mine]

This Forbes article from September argues that the vaccine trials it examined then are measuring efficacy by testing for the wrong things — the absence or presence of symptoms, even mild ones:

One of the more immediate questions a trial needs to answer is whether a vaccine prevents infection. If someone takes this vaccine, are they far less likely to become infected with the virus? These trials all clearly focus on eliminating symptoms of Covid-19, and not infections themselves. Asymptomatic infection is listed as a secondary objective in these trials when they should be of critical importance.

It appears that all the pharmaceutical companies assume that the vaccine will never prevent infection. Their criteria for approval is the difference in symptoms between an infected control group and an infected vaccine group. They do not measure the difference between infection and noninfection as a primary motivation.

Is this still true? I'm not sure — last September is forever in Covid years and the latest trials may have been redesigned.

To Vaccinate or Not To Vaccinate?

This is an not an encouragement reject the vaccines. It's an encouragement to decide about them wisely by considering a number of factors — your need to feel "safe," your need to end this constant quarantine, and society's need to inoculate nearly everyone, versus your trust in the approval process, your personal level of caution, and the benefit of taking a relatively untested product to reduce your Covid risk by maybe 2% in absolute terms.

There are Covid hot-spots after all, areas of the country and the world where infections are soaring, and even low infection rates come at a heavy price — Covid has changed for the worse both the way we live and our economy, and people do die from it.

Maybe the first vaccines out of the gate, perhaps these three, will be everything a mother could want for her family and nation. But even if these products are are very very good, they have to be trusted to be effective.

If that trust is given blindly, and then betrayed, the consequences will be severe.

---

(I've launched a Substack site to greet the post-Trump era. You can get more information here and here. If you decide to sign up — it's free — my thanks to you!)