The effectiveness of cloth masks has been misrepresented by #Masks4All

I recently advised caution about COVID-19 research performed by people without a background in infectious diseases. Some people hated that advice. I’m going to show an example of why it matters.

In recent weeks, entrepreneur Jeremy Howard has led the #Masks4All campaign to make it mandatory to wear cotton face masks in public. Howard claims to have led “the world’s first cross-disciplinary international review of the evidence” for the effectiveness of masks, but he has no formal scientific training. In spite of that, he’s gained coverage from organisation like The Washington Post, The Atlantic, the BBC and the Guardian.

His key claim is that “cotton masks reduce virus emitted during coughing by 96%”, citing a recent South Korean study. He also quotes people like Prof David Heymann of the WHO (for example in his Guardian article):

Untitled-2

Sounds compelling, right?

But the South Korean study did not reference a 96% reduction anywhere. In fact, the paper’s conclusion is ‘Neither surgical nor cotton masks effectively filtered SARS–CoV-2 during coughs by infected patients.’

How does Howard’s “review of the evidence” report this negative finding ?

Another relevant (but under-powered, with n=4) study (31) found that a cotton mask blocked 96% (reported as 1.5 log units or about a 36-fold decrease) of viral load on average…

Hold on a second. It simply isn’t true that the Korean group ‘found that a cotton mask blocked 96%’ of viral load. Deliberately misrepresenting the results of a peer-reviewed publication would be academic misconduct. Assuming an honest mistake, I pointed this out by email. Howard issued a justification in a series of tweets:

Whether the Korean team made a mistake or not – and I don’t believe it did – for a literature review to silently ‘correct’ the scientific record is a breach of ethics. To make matters worse, Howard’s ‘correction’ is itself wrong, and distorts the experimental findings. (If you’re interested in the technical details, please head to the appendix below.)

Even more seriously, the quote from Prof Heymann is not accurate. David Heymann has never said those words and his office has asked Howard to stop misquoting him (but not before Howard published the misquote in both the Washington Post and the Guardian).

But that’s not all. The #Masks4All review omitted the central finding of one of its key references, which was that cotton masks filtered out only 3% of particles during testing. Go back and read that again. The researchers found that 97% of particles penetrated through cotton masks. Why would a ‘review of the evidence’ neglect this key finding?

The evidence for mask wearing by the general public is weak, but I’m not claiming that people shouldn’t wear masks: more research may yet emerge. At a time when many are suggesting to cancel lockdown in favour of mandatory mask-wearing, we need to keep a clear view of the scientific evidence. The claims of the #Masks4All campaign should be treated with caution.

Martin Goodson (Chair of the RSS Data Science Section)

Appendix

The #Masks4All review states that the South Korean study ‘found that a cotton mask blocked 96% (reported as 1.5 log units or about a 36-fold decrease) of viral load on average, at eight inches away from a cough from a patient infected with COVID-19.’

The original study reports:

The median viral loads after coughs without a mask, with a surgical mask, and with a cotton mask were 2.56 log copies/mL, 2.42 log copies/mL, and 1.85 log copies/mL, respectively.

The difference between the median viral loads after coughs without mask and with a cotton mask is 0.71 (or a ratio of about 5, after converting from log units). So about 20% of the virus particles got through the masks. This is a bit rough and ready because the Korean scientists have excluded some of the data points, those marked as ‘ND’ for not detected.

This is where the Howard’s ‘correction’ comes in. His method was to replace all of the results marked as ‘ND’ with zero. This is never done by experimental scientists, because it’s very unlikely that the true values are zero. If nothing is detected, the experimenter must record an ‘ND’ and replace this figure with a standard value when analysing the data.

Every laboratory test has a limit under which detection is unreliable, the limit of detection (LOD). All we know with certainty is that the undetected values must lie somewhere below the LOD.

Here is one authority on this topic (emphasis mine):

…account must be taken of the data below the limit of detection; they cannot be excluded from the calculation nor assumed to be zero without introducing unnecessary and sometimes large errors…. The simplest method is to set all values below the LOD to a half the LOD.

So, what the experimenter must not do is analyse the result as zero. It’s like if you measure your finger for a wedding ring and your finger is smaller than the smallest hole in the ring measurement tool. You know your finger is smaller than the smallest hole but you definitely haven’t measured its size as 0cm.

I have reanalysed the Korean data using the suggested replacement value of half the LOD and the results don’t change very much, suggesting a reduction of 70% of virus particles when using cloth masks. There might be a million virus particles in a single cough. This is a tiny study with only four participants—one of whom could not produce detectable virus particles even without a mask. The authors were correct to draw their weak conclusion that cotton masks do not effectively filter the COVID-19 virus.

Thanks to Piers Stobbs, who edited an earlier draft of this post.

Published by martingoodson

Chair of the Royal Statistical Society Data Science Section. CEO of Evolution AI. @martingoodson

15 thoughts on “The effectiveness of cloth masks has been misrepresented by #Masks4All

  1. “Effectively filter” means something very different based on context. An 80% reduction in viral load is useless in a hospital setting. All doctors get sick, just a little later. It’s a godsend in a public health setting. It drives R0 way down. And it costs $2. It’s just about the highest-ROI intervention you can make.

    Shutting down the city is more effective, and should still be done, but it’s a much lower ROI intervention.

    Like

  2. A 70% reduction in virus particles could easily make a major difference in whether the total number of cases in a population increases or decreases, it seems to me, so the advice for everyone to wear them still seems valid. It is possible for someone to reach a valid conclusion through invalid reasoning.

    Like

    1. People are likely to take unnecessary risks if they think a homemade mask blocks 96% of virus particles. If they know the truth, that the masks are only partially effective, they are likely to act accordingly. The truth matters here.

      Like

  3. Even if masks are only 5% efficient, that’s still a huge number of infections prevented, once you take exponential effects into account.

    Like

  4. Let’s say a cotton filter double layer blocks fifty percent of your nose/mouth opening that translates into fifty percent reduction in your cough emissions (it’s more than that) now the person in contact blocks another fifty percent. That translate to 75 percent inhibitions. This is simple logic but in practice it is more than 90 percent close to 99 percent. You don’t need rocket science to understand it.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: