How Often Do Psychiatrists Actually Agree on a Diagnosis?

Across 19 countries, doctors given the same written patient cases agreed on an ICD-10 diagnosis only about 55% of the time, with schizophrenia cases showing the shakiest agreement of all.

Researchers wanted to know how consistent psychiatric diagnosis really is when different doctors look at the same patient. They recruited 1,038 psychiatrists, psychiatry residents, and other medical doctors across 19 countries in Europe and South America and had them read nine standardized written case vignettes, then assign an ICD-10 diagnosis from a list of 30 common disorders. Using a statistical measure of agreement (Krippendorff's alpha), the team found only modest inter-rater reliability (alpha = 0.48), meaning clinicians landed on the same diagnosis for the same case roughly 55% of the time. Diagnostic accuracy against a reference standard averaged about 66%, and schizophrenia cases were the hardest to pin down, with correct diagnosis rates dropping to 37-54%. Importantly, the disagreements weren't random scatter — doctors tended to confuse the same categories in predictable, systematic ways, suggesting real gaps or ambiguities in how current diagnostic categories are defined rather than simple carelessness. A limitation: because the cases were written vignettes rather than real patients, real-world diagnosis (which includes interviews, history, and follow-up) could look somewhat different, though vignette studies are a standard, controlled way to isolate how clinicians interpret diagnostic criteria themselves.