A rapid Covid-19 antibody test bought by the Government may wrongly tell one in five people they have had the coronavirus.
Number 10 placed an order for one million AbC-19 tests made by UK-based company Abingdon Health in October to be used for research purposes, in order to paint a better picture of how far the virus has spread across the country.
But its self-funded analysis of the pregnancy-stick type tests, published in the British Medical Journal, has suggested they are much less accurate than originally thought.
The study, which was conducted by Public Health England scientists and included almost 5,000 people, showed it was actually only 81.7 per cent accurate at spotting people who should test positive.
Controversially, officials did not publish data about how accurate the tests were before the Department of Health announced it had bought them.
Researchers estimated that if the test were used on one million people, of whom 100,000 had actually had coronavirus, 84,700 results would be correct positives, 15,300 people would get false negatives, and 18,900 people would get false positives.
False negatives mean people are told they have never had the disease when they actually have – of which there are not many negative consequences – while false positives tell people they have had the illness but they actually haven’t, which can lead people to wrongly believe they are immune and take more risky actions.
The results contrast with an earlier, but not yet peer-reviewed study, that claimed the test gives no false positive results. The study, published by the University of Ulster, has since been labelled as ‘critically flawed’.
The figures may be even worse if the antibody tests are self-administered at home because the general public are more likely to make mistakes, such as inaccurately placing a drop of blood into the test cartridge.
Researchers today claimed the Government had ‘jumped the gun’ on purchasing the tests in an undisclosed deal before thorough analysis was published.
The test gives results in 20 minutes and doesn’t need to be read by a lab technician. They have the potential for home use, but regulations say they should only be administered by health professionals.
The UK Government placed an order for one million for AbC-19 tests made by Abingdon Health in October for research purposes, helping to build a picture of how the virus has spread across the country
The AbC-19TM Rapid Test uses a drop of blood from a finger-prick to see if someone has antibodies against SARS-CoV-2 — the virus that causes Covid-19.
Antibodies are part of the immune system. If they are present in the blood, it suggests the person has already been infected with the coronavirus.
The device is known as a lateral flow immunoassay, which officials hoped would play a crucial role in understanding exactly how many people in Britain had been infected since the pandemic began.
Officials tested the accuracy of the AbC-19 test on thousands of key workers, who have been given antibody testing since the summer.
It was commissioned by the Department of Health and Social Care and also involved scientists at the Universities of Bristol, Cambridge and Warwick.
First, the researchers looked at 1,995 blood samples that were taken in 1995 — well before the Covid-19 pandemic. These were used as ‘known negatives’ because they did not have antibodies against the coronavirus.
In the pre-pandemic samples, the antibody test’s specificity — the ability to correctly identify a true negative — was 97.9 per cent. This means 2.1 per cent of results were incorrectly reported as positive.
In the second arm of the study, scientists tested blood samples from 2,847 key workers in England in June. This included NHS workers, fire, and police staff.
Of these, 268 had already been told they were positive for the disease through a PCR swab — the gold-standard of testing. They were categorised as ‘known positives’.
Scientists compared the AbC-19 test with two more robust and accurate laboratory antibody tests.
They estimated the true sensitivity of the AbC-19 test — the ability to correctly identify a true positive sample — was 92.5 per cent.
It means 7.3 per cent of those people were wrongly told they did not have antibodies — known as a ‘false negative’. This is not as serious as a ‘false positive’.
The remaining 2,579 key workers were categorised as those who had not had a Covid-19 test before, who did not know if they had previously been infected.
This represents a real life situation, in which antibody testing would be conducted on people who have no idea if they have had Covid-19 or not.
Among this group, the test sensitivity was 84.7 per cent, based on comparison with other tests.
It means that 15.3 per cent of people with a previous SARS-CoV-2 infection would be missed by the test.
The specificity in the remaining key workers was around 98.9 per cent, meaning 1.1 per cent were given a false positive.
Sir John Bell, regius professor of medicine at Oxford University, who is the government’s main adviser on tests, has previously called the AbC-19 test ‘truly amazing’
Experts conceded that the test is probably more sensitive in those with a confirmed diagnosis because their antibody levels were higher. This is because their infection was more severe – assumed because they sought having a test.
The team said the lower sensitivity figure of 84.7 per cent is probably a more realistic estimate of test sensitivity in the real world, if people were to choose to take the test to find out their own previous infection status.
And those who already had a PCR diagnosis are not as likely to seek an antibody test because they already have confirmation they’ve had the coronavirus.
The team calculated how this would have an impact in the real world, working out what’s known as the ‘true positivity value’ – how accurate it would be when conducted on a population.
The team estimated how many false positives would occur in a situation where five per cent, 10 per cent, 15 per cent, 20 per cent or 25 per cent of the population had already had the disease.
If 10 per cent had been previously infected with Covid, it would mean around one in five positive results were wrong — a positive predictive value of 81.7 per cent.
If one million people were tested with the AbC-19 kit, there would be 18,900 false positive results, the team estimated.
If five per cent of the population had previously had the disease, then around one in three would get a false positive — a positive predictive value of 67.9 per cent.
As the proportion of people who have really had the disease gets smaller, the margin of error gets bigger and the results become more and more inaccurate – because the proportion of results that are wrong will never change, regardless of the proportions of people getting accurate positive results.
Previous research reported 97.7 per cent sensitivity and 100 per cent specificity of the AbC-19 Rapid Test, meaning no one would be given a false positive.
It was conducted by Ulster University at the laboratories of Abingdon Health – who said it ‘would be inaccurate to suggest our numbers are lower than we’ve stated’.
Experts pointed out the study excluded samples for which it was considered ‘unclear’ if a person had antibodies, which statisticians have warned will lead to over-estimates of accuracy.
No10 has already got one million AbC-19 tests under its belt for research purposes on behalf of the UK-Rapid Test Consortium (UK-RTC).
The CEO of Abingdon, Chris Yates claimed 10million tests could be manufactured within a six-month period.
Sir John Bell, a government testing adviser, has previously called it ‘truly amazing’.
But Dipender Gill, a clinical scientist at Imperial College London, and Dr Mark Ponsford, an immunologist at Cardiff University, warned that positive results from these tests does not necessarily mean someone is immune to catching Covid-19.
The team calculated how this would have an impact in the real world, working out what’s known as the ‘true positivity value’ – how accurate it would be when conducted on a population. This graphic shows how it can vary with scale
In a linked editorial, the research experts said the study ‘identifies notable limitations of the UK government’s antibody test of choice and provides good evidence that its specificity in a ‘real life’ setting is highly unlikely to be 100 per cent’.
They said ‘a clear message must be communicated to the public that positive results from these assays do not provide evidence of immunity.’
‘Apart from limited surveillance to estimate the proportion of a population that has been infected, widespread use of this assay in any other role could risk considerable harm,’ they concluded.
Even when the news of the deal broke on October 7, scientists had concerns about the data.
Professor Jon Deeks, a biostatistician at the University of Birmingham, said: ‘The only available study of this test was undertaken by UK RTC members, and was shown to be critically flawed several weeks ago.
‘Whilst we are waiting to see valid evidence of how well it performs, the Government has jumped the gun in purchasing the test before the independent studies evaluating it are finished or available.’
The large study published today still has some limitations.
For example, the test was evaluated in a laboratory, rather than having participants perform the test themselves. Therefore, the accuracy may have overestimated.
It is possible that other pregnancy test style devices for antibodies may also be less accurate than claimed, the researchers said.
They said their work ‘highlights the scope for overestimation of SARS-CoV-2 antibody test sensitivity in other studies in which sensitivity has been estimated only from PCR confirmed cases’.
MailOnline has contacted Abingdon Health and Ulster University for comment.
Source link