Press Briefing On The Report Of The second batch Pre-Market Validation Of SARS-COV-2 Infection (COVID-19) Test Kits In Nigeria By Dr. Tosan Erhabor, Registrar/Chief Executive Officer, Medical Laboratory Science Council Of Nigeria (MLSCN) On Tuesday, October 13, 2020
PROTOCOL
Preamble
Gentlemen of the Press, I am delighted to welcome you to the MLSCN House, and to this important briefing. As you may be aware, this is the second briefing hosted by the Medical Laboratory Science Council of Nigeria (MLSCN) since the outbreak of the COVID–19 pandemic. Essentially, both briefings were aimed at updating the good people of this country on the outcome of the Pre-market Validation of COVID–19 test kits brought into the country by various promoters and vendors for potential deployment for diagnostic purposes and to help contain the menace of the COVID–19 pandemic in our country.
We appreciate the President and Commander-in-Chief of the Armed Forces, Muhammadu Buhari, the Presidential Task Force on COVID-19 (PTF), the Federal Ministry of Health (FMoH), and the Nigeria Centre for Disease Control (NCDC) for maintaining the gold standard by deploying PCR for COVID–19 testing, as well as supporting MLSCN in carrying out its mandate with respect to the validation of test kits in the country.
Ladies and Gentlemen, pursuant to its mandate as enunciated in Act 11 of 2003 (CAP M25 LFN 2004), MLSCN regulates the Production, Importation, Sales and Stocking of Diagnostic Reagents, Equipment and Chemicals, and as such commenced the process of Validating COVID-19 Rapid Test Kits (RTKs) in Nigeria (Section 4, sub-section e). The outbreak of the pandemic and the efforts to contain it has indeed brought the MLSCN mandate to the fore. MLSCN first made public its verdicts on the validation of the initial batch of COVID–19 test kits on the 29th May, 2020. Today’s briefing too, is to avail the public of the report of the second batch Pre-Market Validation of COVID-19 test kits carried out by MLSCN as part of its regulatory functions.
Request for Validation:
Consequent upon the advert by MLSCN in four National Daily Newspapers on the 30th March, 2020 calling for submission of COVID–19 test kits for the purposes of validation, MLSCN IVDs Laboratory, Yaba, Lagos received a total of 43 brands of test kits for validation. Also, a total of 300 worth of test kits were requested from each brand seeking validation. 3 Real-Time PCR-based kits, 28 RTK (2 Antigen and 26 Antibody kits), 1 Antibody Elisa kit and 1 Antibody Chemiluminiscence kit were selected for validation based on first come first serve for those that met the requirements.
Goals and Objectives of COVID-19 Test Kits Validation:
Goals:
To validate the laboratory performance characteristics of Rapid/PCR Test Kits using Real-Time-Polymerase Chain Reaction method as reference standard for the detection of SARS-CoV-2 infection.
Objectives:
The validation exercise sought to accomplish the following specific objectives:
- To determine the sensitivity and specificity of the kits
- To determine the accuracy of the test kits
- To determine the inter-reader variability of the test results of the kits
Results:
Table 1: Performance Characteristics of the Validated Kits Including Accuracy
| RTK | Sensitivity (95% CI) | Specificity (95% CI) | Accuracy |
| Kit 2 | 17.0 (10.4-25.5) | 90.9 (85.0 -95.1) | 59.4 |
| Kit 3 | 37.7 (28.5- 47.7) | 95.1 (90.2 – 98.0) | 70.7 |
| Kit 14 | 44.3 (34.7 – 54.3) | 94.4 (89.3 -97.6) | 73.1 |
| Kit 15 | 32.1 (23.3 – 41.8) | 99.3 (96.2 -100.0) | 70.7 |
| Kit 16 | 32.1 (23.3 – 41.8) | 84.6 (77.6 – 90.1) | 62.2 |
| Kit 17 | 42.5 (32.8 – 52.4) | 82.5 (75.3 -88.4) | 57.4 |
| Kit 18 | 39.6 (30.3 – 49.6) | 90.9 (85.0 -95.1) | 69.1 |
| Kit 19 | 46.2 (36.5 – 56.2) | 83.2 (76.1 -88.9) | 67.5 |
| Kit 21 | 28.3 (20.0 – 37.9) | 100 (100-100) | 69.5 |
| Kit 24 | 44.3 (34.7 – 54.3) | 90.2 (84.1 -94.5) | 70.7 |
| Kit 25 | 53.8 (43.8 – 63.5) | 76.2 (68.4 -82.9) | 66.7 |
| Kit 26 | 40.6 (31.1 -50.5) | 95.1 (90.2 -98.0) | 71.9 |
| Kit 27 | 41.5 (32.0 – 51.5) | 93.7 (88.4 -97.1) | 71.5 |
| Kit 28 | 40.6 (31.1 – 50.5) | 93.7 (88.4 – 97.1) | 71.1 |
| Kit 30 | 60.4 (50.4 -69.7) | 86.0 (79.2 -91.2) | 75.1 |
| Kit 31 | 30.2 (21.7 – 39.9) | 97.9 (94.0 -99.6) | 69.1 |
| Kit 32 | 50.9 (41.0 -60.8) | 80.4 (71.4 -85.4) | 71.1 |
| Kit 33 | 50.9 (41.0- 60.8) | 83.9 (85.0 -95.1) | 69.9 |
| Kit 34 | 57.7 (47.6 -67.3) | 80.1 ( 72.6- 86.4) | 70.6 |
| Kit 35 | 49.0 (39.1 -59.0) | 77.3 ( 69.5 -83.6) | 65.3 |
| Kit 38 | 45.9 (36.2 -55.9) | 83.7 (77.5 -88.8) | 69.7 |
| Kit 46 | 51.5 (41.8 -61.1) | 81.4 (75.0 -86.8) | 70.2 |
Table 2: Performance Characteristics of the Validated Non-Rapid Test Antibody Detection Kits
| Non-Rapid Test Anitibody detection Kits | Sensitivity (95% CI) | Specificity (95% CI) | Accuracy |
| NON-RT ANTIBODY A | 18.7 (9.0-32.6) | 87.1 (79.6 -90.3) | 68.9 |
| NON-RT ANTIBODY B | 21.3 (10.7- 35.7) | 90.1 (81.2 – 91.6) | 71.9 |
| NON-RT ANTIBODY C | 11.1 (3.7 – 24.1) | 91.9 (85.7 -94.9) | 70.2 |
Table 3. Performance Characteristics of Antigen-based COVID-19 Rapid Diagnostic kits
| Test Kit | Sensitivity | Specificity | Accuracy |
| Ag 004 | 10 | 87.2 | 74.2 |
| Ag 006A | 12.8 | 98 | 84.0 |
| Ag 006B | 58.1 | 92.9 | 80.4 |
Table 4. Performance of RT-PCR Assays for detection of SARS-CoV-2
| PCR ASSAY | Sensitivity (95% CI) | Specificity (95% CI) | Accuracy | |||
| PCR PRODUCT A | 78.8 (66.2 -88.3) | 91.5 (87.5 -95.1) | 87.8 | |||
| STD-M | 92.9 (80.5 -98.5) | 97.6 (93.1 -99.4) | 96.4 | |||
| Viasure | 98.6 (92.5 -99.9) | 89.0 (81.6 -94.2) | 92.8 | |||
NOTE: KITS NUMBERS AND PRODUCT ALPHABETS ARE NOT THE BRANDED NAMES OF THE TEST KITS; THEY ARE USED FOR THE PURPOSE OF THIS BRIEFING. HOWEVER, THE FMoH HAVE THE COMPREHENSIVE REPORT.
Observations:
A total of 33 test kits and systems were validated. All the antigen and antibody test kits, rapid or otherwise, failed to meet the minimum acceptable criteria.
The 22 rapid test kits being reported upon, while having fulfilled the requirement of rapid test kits, have not met the characteristics of sensitivity and specificity to qualify for deployment for purposes of testing in disease surveillance and routine diagnosis. The highest sensitivity of 60.4% found in these kits is too low to be used in the detection of SARS-CoV-2 infection. It is also far below the generally acceptable minimum in-vitro diagnostics (IVD) sensitivity and specificity of 95%. Although some of the kits demonstrated relatively good specificities of up to 100%, but these same kits have poor sensitivities making them unsuitable as IVDs. Similarly, the non-rapid antibody detection test kits had poor sensitivities and specificities and are therefore unsuitable for use as IVDs in Nigeria.
The PCR test kits are designed to detect two or three genes of SARS-CoV-2. All of them are in multiplex PCR format. They are all reverse transcriptase Real-Time PCR. The PCR test evaluated had a relatively better performance than the antigen and antibody detection test kits.
Real – Time PCR with Viasure had the highest sensitivity followed by STD-M then PCR PRODUCT A. Viasure and Standard M had sensitivities above 90%, while PCR PRODUCT A has a sensitivity of 78.8%. PCR PRODUCT A has not met the requirement of laboratory performance of 95% sensitivity and specificity for IVDs, and it is not recommended for use as IVD in Nigeria.
Viasure has met the requirement for sensitivity while Standard M met the requirement for specificity. These two kits can be utilized for SARS-CoV-2 infection testing in Nigeria. Thus, it will be most beneficial if the properties of the two kits can be harnessed by combining the two kits with Viasure as the first line and the Standard M as second line.
Viasure also has the advantage that the Master Mix is lyophilized and can be stored up to 30C. This means it can be stored at room temperature (15C to 25C). Viasure also requires less pipetting as the Master Mix is already in the PCR tubes requiring only rehydration and addition of template. This will benefit Turn Around Time (TAT).
It became clear from the standardized validation done by MLSCN, that the manufacturers’ claims as contained in the product insert about Sensitivity, Specificity and Accuracy are not always correct. In this case, a significant standard deviation was observed- an indication that Rapid Test kits that are not validated pose higher risks of providing false positive and negative results.
Comments:
All the Rapid Test kits validated fulfilled the requirement to perform laboratory test without the need of equipment and were able to produce results in less than twenty (20) minutes. However, for a rapid test kit to be deployed for disease surveillance and diagnosis, it must be able to detect a disease-causing agent when it is present and to return a negative result if the causative agent is absent. In other words, the kit must possess good sensitivity to be able to detect the presence of the disease, and good specificity, so that no interfering substances can lead to a false positive instead of true negative results in the absence of the etiological agent of the disease. Therefore, for a kit to be considered reliable for laboratory diagnosis and disease surveillance, the kit should have a high sensitivity and specificity. A kit performing very well in one of these characteristics without the other renders it unsuitable for diagnostic testing. It is also unsuitable if both characteristics are low.
Therefore, it is to be noted, that the 22 rapid Test kits in this report have not met the expected performance characteristics of sensitivity and specificity to qualify them for deployment for the purposes of testing in disease surveillance and routine diagnosis.
MLSCN Validation Verdicts:
- None of the antigen or antibody detection test kits is recommended for SARS-CoV-2 infection testing in Nigeria.
- For purposes of diagnosis, Viasure PCR kit is recommended as a first line test while Standard M may be used as second line test.
- For surveillance purposes, Standard MnCoV Real Time Detection kit is recommended.
- No SARS-CoV-2 Rapid Test kit is currently approved for use in Nigeria.
- MLSCN hereby cautions against the use of any non-validated Rapid / PCR Test Kits for COVID-19 testing in Nigeria as this will attract sanctions in accordance with the law.
Recommendation
- All indigenously-developed test kits for COVID-19 testing in Nigeria should be submitted for validation to support standardization and promote reliable and accurate test results.
- In view of these findings, it is pertinent to recommend the establishment and institutional participation in External Quality Assessment programme for COVID-19 testing in Nigeria.
Conclusion
The validation of other Rapid / PCR Test Kits for SARS-CoV-2 Infection remains an ongoing process, as Nigerians will be informed when suitable test kits meet minimum validation requirements. However, MLSCN encourages both local and other manufacturers to improve on the standardization of their products so as to meet the minimum validation requirements to guarantee reliable, accurate, and timely test results. Importantly too, MLSCN will work closely with NCDC to ensure that the two (2) WHO pre-qualified Antigen Rapid Test Kits go through validation as soon as they arrive the country. This will enable the country to expand the scope of access to quality COVID-19 testing as more sectors of the economy open. MLSCN assures the Federal and State Ministries of Health, and indeed the government of Nigeria of its commitment to ensuring quality healthcare delivery as envisioned in its statutory mandate.
Signed
Management
