Author: Stanley S. Levinson, PhD, DABCC // Date: OCT.13.2020 // Source: Scientific Shorts
- Lab Management,
- Infectious Diseases/Microbiology,
- Polymerase Chain Reaction (PCR)
The following post was written in 2020, and has since been updated to reflect current guidelines. Please see an updated Short on this topic here.
There are multiple severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) emergency use authorization (EUA) tests among clinical laboratories and a large amount of cross-assay variation between different assays. Moreover, these assays were rapidly developed, minimally standardized and there is no well recognized external quality assessment program (EQA). As a result, good estimates of the diagnostic sensitivity and specificity are not available. There has been more focus on the diagnostic sensitivity and it is known that sensitivity is poor if the test is performed too early before detectable RNA is shed and that the viral RNA may be detectable for a period of time after active infection although the virus is no longer viable or infective. Therefore, the CDC does not recommend retesting after recovery, but CDC suggests symptomless persons who are immunologically normal are no longer considered infectious about 10 days after symptom onset.
There is less information about diagnostic specificity (false positives). Among others, false positives will depend on the length of the DNA probes, how many and which genes are measured and technical errors. The DNA probes used in the CDC rRT-PCR test kits for SARS-CoV-2 assay are only about 25 bases long which does not meet the FDA recommendation for nucleic acid-based molecular diagnostics for viral disease infections where 100 contiguous bases is desirable (1). Various methods use different genes and different probes that may not be equivalent. There is a 100-fold difference in limit of detection (LoD) between some assays (2). Technical error, especially due to contamination may cause false positives. Seventy-seven professional baseball major league players initially tested positive in one lab but negative elsewhere (3) in what was deemed Lab error. Except that they had multiple sources for testing, they might have been classified as asymptomatic. We don’t know how many other persons were classified in error from this incident.
Originally, PCR was followed by a second step where a separation technique such as a blotting method was used to confirm that the amplified substance was correct. rRT-PCR is usually not followed by a second step. RRT-PCR is usually applied for diagnostic purposes, not for screening. For acute viral infections, after symptoms appear, a rRT-PCR test battery may be performed. In diagnostic testing, symptoms or high-risk behavior cause an increase in prevalence because those with certain symptoms are classified into characterized groups and false positives are few.
Diagnostic applications are usually applied for chronic viral infections such as HCV, HIV and chronic HBV where symptoms or high-risk behavior initiates testing, although there are now screening recommendations for HCV. Still, in all these chronic diseases antibody concentrations are high and serology usually precedes rRT-PCR, so that false positives are rare. At present prevalence, COVID-19 testing is primarily widespread screening without confirmation.
For SARS-CoV-2 rRT-PCR, cycle threshold (Ct) of 24 or less has been shown to be highly predictable for identifying active COVID-19 cases (4), but since LoD of various methods drastically differ it is unclear which methods this applies to. Generally, methods do not amplify more than 40 cycles, but some systems go beyond 40 Ct. It seems likely that short probes in such systems could lead to amplification errors. Although there is no wide spread EQA proficiency programs for SARS-CoV-2, there is one report (5), of EQA in clinical laboratories for other RNA virus. The authors compiled 43 EQAs of rRT-PCR assays, conducted between 2004-2019. Each EQA involved between three and 174 laboratories, which together provided results for 4,113 blind panels containing 10,538 negative samples. 336 of the 10,538 negative samples (3.2%) were reported as positive. The authors defined the lowest percentage of the interquartile range which was 0.8% as a conservative estimate of the false positive rate. In another report, Sin Hang Lee found that 3 of 10 positive proficiency samples in the State of Connecticut were negative containing no SARS-CoV-2 RNA by a confirmatory assay (1). The Foundation for Innovative New Diagnostics (FIND) examined 22 rRT-SARS-CoV-2 diagnostic tests (6) and found diagnostic specificities ranging between 100% and 96% for 100 specimens assayed by each test. Although the great majority showed 100% specificity, given the small number assayed, the lower 95% confidence limit which was 95% for almost all assays would seem to be a better estimate (possible 5% error). Moreover, these were tested under controlled conditions, not at all similar to high output clinical laboratories running thousands of tests.
The Reverend Thomas Bayes (1701-1761) recognized a kind of statistic that predicts the posterior probability from the prior probability. For testing, this means the post test probability can be derived from the pretest probability if the prevalence is known. This sounds complicated but actually, Bayesian statistics are simple compared to classical frequentist statistics since one does not have to apply a null hypothesis, nor interpret p-values or effect-size and the results are obtained from simple mathematics. If, as discussed above (5), a 0.8% false positive rate is correct, at a six percent positive rate that some States claim, then there would be: 100 x 0.06 = 6 positives/100 tests. But if 0.8% are false positives, then only 5.2% are true positives with a positive predictive value (True positives/total positives x 100) of 5.2/6 x 100 = 86.6%. This means about 13.4% are false positive. Notice as the prevalence of disease decreases, the percentage of false positives to total positives increases because the true positive percentage decreases but the percent false positive (in this case 0.8%) stays the same. Thus, the percentage of false positives would be about 26.6% at a three percent positive rate.
The source of the problem is recognized from Bayesian analysis. If the prevalence is low (say a prevalence of 1%) even a very good screening test with 99% diagnostic specificity and 100% sensitivity will produce only 1% false positive results: (diagnostic specificity 1%) = 0.01 x 10,000 tests = 100 false positives/10,000 tests and (0.01% prevalence of disease at 100% sensitivity) = 0.01 x 10,000 = 100 true positive but for a poor positive predictive value of only 50% (100/200 x 100 = 50%). Recognizing this problem, the CDC suggests most testing should be diagnostic: “Considerations for who should get tested: People who have symptoms of COVID-19, people who have had close contact with someone with confirmed COVID-19, people who have been asked or referred to get testing by their healthcare provider, or state health department. Not everyone needs to be tested. (7)”
Because of rightful concern regarding disease transmission from asymptomatic and pre-symptomatic cases, this advice is not being followed. As a result, the great abundance of testing is screening not diagnostic. One way to reduce false positive results is to repeat the test using a test with a different format (different manufacturer). Due to limited testing facilities confirmation is not routinely performed and only a few positives are confirmed by a second rRT-PCR assay. I conclude it is likely that at current active disease prevalence the positive rRT-PCR results of many “asymptomatic” persons are false positives.
There are negative psychological implications of thinking one is infected when one is not and some persons with illness other than COVID-19 who test false positive might be hospitalized with COVID-19 patients and become infected. This may explain why some persons seem to have been infected twice: the first time being a false positive. It seems to me it is important for practicing medical professionals to be aware of these issues so that they can appropriately advise and direct suspect patients for additional testing.
- Lee SH. Testing for SARS-CoV-2 in cellular components by routine nested RT-PCR followed by DNA sequencing International Journal of Geriatrics and Rehabilitation 2020;2:69-96.
- FDA. SARS-CoV-2 Reference Panel Comparative Data. Accessed 9/20/2020 doi: 10.1093/cid/ciaa638 https://www.fda.gov/medical-devices/coronavirus-covid-19-and-medical-devices/sars-cov-2-reference-panel-comparative-data.
- Maske M. NFL’s 77 positive virus tests were ‘likely false positive results,’ company says. Accessed 9/20/2020 https://www.washingtonpost.com/sports/2020/08/23/nfl-teams-interrupt-practice-schedules-after-positive-coronavirus-tests-new-j
- Bullard J, Dust K, Funk D, Strong JE, Alexander D, Garnett L, et al. Predicting infectious SARS-CoV-2 from diagnostic samples. Clin Infect Dis 2020 doi: 10.1093/cid/ciaa638.
- Cohen AN, Kessel, B. False positives in reverse transcription PCR testing for SARS-CoV-2. Accessed 9/20/2020 https://doi.org/10.1101/2020.04.26.20080911 (not peer reviewed).
- Accessed 10/01/2020 https://www.finddx.org/covid-19/sarscov2-eval-molecular 7. Accessed 10/01/2020 https://www.cdc.gov/coronavirus/2019- ncov/testing/diagnostic-testing.html