Monitoring the performance of surgeons
15 May 2015 NUS statisticians developed a highly improved risk-adjusted procedure for monitoring the performance of surgeons.
In order to monitor the performance of a surgeon and to take a patient's existing health condition into account, Lovegrove et al. (1997) developed a variable life-adjusted display (VLAD) obtained by plotting the difference between the expected and actual cumulative mortality. The VLAD lacks a proper rule for signaling deterioration or improvement. Steiner et al. (2000) developed a cumulative sum (CUSUM) charting procedure that has a proper signaling rule.
Both procedures are based on a crude binary classification of surgical outcomes: 1 for death within 30 days of an operation and 0 otherwise. Such a classification of surgical outcome is naive. It is unreasonable for example, to regard a patient who has a full recovery, the same quality outcome as another patient who survived but remained bed-ridden for life.
A team led by Prof GAN Fah Fatt and his PhD student Mr Tang XU from the Department of Statistics and Applied Probability in NUS and a Statistician from New Zealand, Dr ZHANG Lingyun jointly developed a CUSUM charting procedure based on more than 2 outcomes. With a more refined set of surgical outcomes, the new procedure will be able to measure the performance of a surgeon accurately. The figure shows the effect of partial recoveries on the VLAD plots of two surgeons [see Figure].
The monitoring of medical practice is an important component in improving the quality of health care.
In 1999, the UK General Medical Council found three doctors possibly guilty of professional misconduct over the quality of their heart surgeries. This misconduct led to 29 mortalities out of 53 children who were operated at the Bristol Royal Infirmary. Some of these deaths could be avoided if a CUSUM procedure were in place.
The VLAD has since gained world-wide attention and it is used in many hospitals all over the world. The CUSUM procedure has a proper signaling rule and it is useful to hospitals that want to have a proper monitoring system to improve health care.
The figure compares the performances of two cardiac surgeons. The two plots on top are based on binary-outcome data and the two plots at the bottom are based on three-outcome data (death, partial recovery [shown by dots] and full recovery). Based on the crude binary-outcome data, Surgeon A seems to perform better than Surgeon B. Based on the more refined three-outcome data, Surgeon B performs better than Surgeon A because for those patients who survived, there are more patients who recovered fully. [Image credit: Gan FF]
- Levogrove, J, Valencia O, Treasure T, Sherlaw-Johnson C, Gallivan S. “Monitoring the Results of Cardiac Surgery By Variable Life-adjusted Display.” Lancet 350 (1997) 1128.
- Steiner SH, Cook RJ, Farewell VT, Treasure T. “Monitoring Surgical Performance Using Risk-Adjusted Cumulative Sum Charts.” Biostatistics 1 (2000) 441.
- Tang X, Gan FF, Zhang L. “Risk-Adjusted Cumulative Sum Charting Procedure Based on Multi-Responses.” Journal of the American Statistical Association. 110 (2015) 16.