A diagnostic accuracy study compares an index test with a reference standard. Its headline results, often sensitivity and specificity, can look portable. They are not. Performance depends on who was tested, how disease was defined, which threshold was chosen, and whether every participant received an adequate reference assessment.
Reading the two-by-two table is necessary. Reading how the table was created is more important.
Reconstruct the study question
Identify four elements before looking at results:
- Index test: the assay, score, image, device, or examination under evaluation.
- Target condition: the condition or state the test is intended to detect.
- Reference standard: the method used to classify the target condition.
- Intended use: screening, triage, confirmation, monitoring, or another role.
A test evaluated in symptomatic specialty-clinic patients may perform differently in population screening. The intended position in the pathway determines which errors matter most.
Build the two-by-two table
After applying a threshold, results fall into four cells:
- true positive: index test positive, target condition present;
- false positive: index test positive, target condition absent;
- false negative: index test negative, target condition present;
- true negative: index test negative, target condition absent.
Sensitivity is the proportion of reference-positive participants who test positive. Specificity is the proportion of reference-negative participants who test negative.
These are conditional proportions. Sensitivity is not “the probability that a positive person has disease,” and specificity is not “the probability that a negative person is healthy.” Those interpretations confuse test characteristics with predictive values.
Predictive values depend on the tested population
Positive predictive value is the fraction of positive index results that are true positives. Negative predictive value is the fraction of negative results that are true negatives.
Both depend strongly on prevalence, or, more precisely, on the pretest distribution of the target condition in the tested setting. Even a test with high sensitivity and specificity can produce many false positives when applied to a low-prevalence population.
Therefore, predictive values from a case-control sample assembled to contain equal numbers with and without disease may not describe routine practice.
Thresholds trade one error for another
Many index tests produce a continuous measurement. Turning it into positive or negative requires a threshold. Lowering the threshold usually raises sensitivity and lowers specificity; raising it usually does the reverse.
If investigators choose the “optimal” threshold after examining the same data, reported performance is optimistic. A credible report distinguishes:
- a threshold specified before analysis;
- a threshold estimated in a development set;
- performance confirmed in an independent validation set.
A receiver operating characteristic curve summarizes performance across thresholds. Its area under the curve measures discrimination, not calibration, clinical usefulness, or accuracy at the operating threshold.
The reference standard can be imperfect
There may be no flawless gold standard. Histology can have sampling error; expert adjudication can be subjective; a laboratory comparator can have its own measurement uncertainty.
The report should define the reference procedure, reader qualifications, blinding, disagreement resolution, and timing relative to the index test. If the index test contributes to the reference diagnosis, incorporation bias can make agreement look better than it is.
Time matters too. If the condition can change between index and reference testing, disagreement may reflect biology rather than test error.
Patient selection creates spectrum effects
A test may separate advanced disease from healthy volunteers very well yet struggle in patients with mild disease or clinically similar alternatives. Accuracy can change with age, stage, comorbidity, treatment, specimen quality, and setting.
Consecutive or randomly selected eligible participants generally provide a more realistic spectrum than a two-gate design that recruits obvious cases and healthy controls separately. Eligibility criteria and the participant flow diagram reveal how closely the sample matches intended use.
Verification bias can remove the hard cases
If only index-positive participants receive the reference standard, false negatives remain unknown and sensitivity is distorted. This is partial verification bias.
Differential verification occurs when different reference standards are used depending on the index result or clinical features. Sometimes that is unavoidable, but the study should explain the pathways and address the resulting uncertainty.
Uninterpretable and indeterminate results also belong in the flow of participants. Excluding them after the fact can make a test look cleaner than it will be in practice.
Confidence intervals belong beside every estimate
Sensitivity of 95% based on 19 true positives out of 20 reference-positive participants is much less precise than 95% based on 950 out of 1,000. The percentage alone hides that difference.
Reports should provide numerators, denominators, and confidence intervals. Subgroup estimates require the same caution, especially when a small sample is divided repeatedly.
Accuracy is not clinical utility
A test can classify well yet fail to improve outcomes. Utility depends on what happens after the result: additional testing, treatment, delay, anxiety, cost, or avoided procedures.
Decision thresholds should reflect the consequences of false positives and false negatives. Net-benefit or impact studies may be needed to show that using the test produces better decisions than existing practice.
A fast appraisal sequence
Use this order when reviewing a paper:
- Confirm intended use and setting.
- Check whether participants resemble the intended population.
- Inspect the reference standard and blinding.
- Determine whether all participants received verification.
- Find out when and how the threshold was chosen.
- Reconstruct the two-by-two table.
- Read confidence intervals and indeterminate-result handling.
- Decide whether the reported metric answers the clinical question.
Reporting details that should be visible
STARD 2015 calls for transparent reporting of participant selection, test methods, prespecified hypotheses, sample-size reasoning, handling of missing and indeterminate results, cross-tabulation, adverse events, and participant flow.
Registration and access to a protocol or analysis plan make it easier to distinguish planned analyses from data-driven choices. They do not guarantee quality, but they reduce ambiguity.
Bottom line
Diagnostic accuracy is not an intrinsic property that travels unchanged between populations. It is an estimate produced by a particular threshold, disease spectrum, reference process, and study design. The most useful paper lets the reader trace every participant from eligibility to the final two-by-two table, and shows what happened to everyone who did not fit neatly inside it.
Sources
- Bossuyt PM, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015.
- Cohen JF, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016.
- Cochrane. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy.
- U.S. Food and Drug Administration. Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests.
Continue reading
Editorial responsibility and corrections
Published by Serum Research. Sources checked October 1, 2026. This is educational material; no independent scientific, clinical, statistical, analytical, or regulatory review is claimed. Read how we review evidence or send a correction.
For Research Use Only. Not for Human Consumption. This page does not provide instructions for medical use or establish the identity, quality, safety, or equivalence of a catalog material.