ACS-NSQIP less accurate for gauging risk in emergency surgery patients

Last Updated: 2016-01-21

By Anne Harding

NEW YORK (Reuters Health) - A widely used tool for predicting surgical outcomes, the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP), is less accurate when used in emergency surgical patients than in elective cases, new findings show.

"When we do risk adjustment or comparison of outcomes for surgical procedures, it looks like it's hard to adjust for emergency surgery," Dr. Joseph Hyder of the Mayo Clinic in Rochester, Minnesota, the first author of the new study, told Reuters Health.

Based on the findings, Dr. Hyder said, emergency and elective surgical cases should be looked at separately when estimating risk, rather than included together.

Emergency surgery patients tend to be sicker than elective surgery patients, and these cases are complex and heterogeneous, Dr. Hyder and his team note in their report, online December 31 in the Annals of Surgery. However, current risk models, including the ACS-NSQIP, don't distinguish between emergency and elective cases.

To compare the accuracy of ACS-NSQIP for emergency and elective cases, the researchers looked at three types of surgical cases that are common in both elective and emergency situations: gastrointestinal, vascular, and hepato-biliary-pancreatic.

They identified nearly 57,000 emergency and more than 136,000 elective patients from the NSQIP 2011-2012 files, and used a sample of more than 37,000 patients matched by Common Procedure Terminology and year for their analysis.

Observed-to-expected (O:E) mortality ratios were different for the emergency and elective patients (1.031 vs. 0.79, p<0.0001), and c-statistics were also significantly different.

The Brier score, used to measure the inaccuracy of probability forecasts, found no significant differences when mortality rates were 6.5% and 9%, but did find differences when mortality rate was 1.7% (emergency 0.034 vs. elective 0.016, p=0.0005).

ACS-NSQIP "may not be as well tuned for emergency surgery as it is for elective surgery," Dr. Hyder said. To be able to use the tool "meaningfully" to benchmark outcomes and compare surgeons and hospitals, he added, "we may need to do it really differently for elective cases and emergency cases."

The current study could not determine whether the differences would affect hospital ratings and rankings, he added.

"What's really going to matter for physicians and surgeons is the performance of their hospital or of themselves, and whether taking care of more emergency patients is affecting how they themselves or their hospitals are ranked," Dr. Hyder said.

He and his colleagues conclude: "These findings with high-quality clinical data and sophisticated models of the ACS-NSQIP suggest that all risk-prediction and risk-adjustment enterprises that include both emergency and elective patients deserve formal evaluation of differential risk prediction, including hospital-level data. This is particularly important when such tools are used to inform patients and rate, rank, and reimburse providers."

SOURCE: bit.ly/1PqBWfQ

Ann Surg 2015.