Relationship between compliance to quality indicators , volume , and outcomes in pancreatic cancer resection in a community hospital setting

Objective: To calculate and compare adherence to pancreatic cancer care quality indicators and related outcomes at three hospitals within our health system and determine their relatedness. Design: Retrospective review of cancer registry data and patient charts from January 1, 2000 December 31, 2010. Setting: Three hospitals performing pancreatic resections within a single health system. Patients: All patients undergoing resection for the treatment of pancreatic cancer. Main Outcome Measures: Adherence to published quality indicators, patient survival, and surgical and institutional variables related to mortality. Results: Complication, morbidity and mortality, and overall survival rates were similar between facilities and were comparable to previously reported values of similar volume facilities. Overall adherence to quality indicators was approximately 73%; compliance to indicators in the “outcome” and “efficiency” domains was highest, but adherence was not associated with clinical outcomes. Patient mortality was lowest at the highest volume hospital in our health system; however, surgeon volume was not predictive of outcomes. Conclusion: Use of quality indicators is valuable in determining quality of care and identifying weaknesses, allowing for a targeted approach to improve quality of care and the associated processes. However, adherence to these quality indicators is not necessarily indicative of outcomes.

P ancreatic cancer is one of the leading causes of death from can- cer in the United States.Unfortunately, the prognosis for patients with pancreatic cancer is typically very poor.Resection of the tumor is the only possible curative treatment, though many patients present with advanced disease which does not allow for surgical intervention.While the overall 5 year survival of patients with pancreatic cancer is less than 5%, outcomes are variable and based on a number of factors (1)(2)(3).In the climate of pay-for-performance, it becomes increasingly important to identify the variables which contribute to outcomes and can be controlled (4).
Current literature suggests that outcomes are dependent on the volume of surgeries performed at the center, and that the volume of resections performed by the particular surgeon may have an impact on patient mortality (5).High volume centers report more favorable outcomes in many procedures, and the variability of care and outcomes has consistently been reported as being greatest in pancreatic cancer (5)(6)(7)(8).
Though high volume centers report the best outcomes, the majority of pancreatic resections are performed in small, low-volume community hospital settings (6,9).Regionalization of care for pancreatic cancer has been proposed, though critics assert that the inconvenience and cost to patients, among other factors, make this strategy unfeasible (10).As such, it is necessary to standardize practices and care in centers treating different volumes of patients, the overall goal being better and more consistent outcomes between low, moderate, and high volume centers.In order to standardize care, quality indicators have been developed which take into account many factors which may influence outcomes on different levels (11).Evaluation of an institution's adherence to these indicators may help to identify areas of weakness and reduce variability in quality of care (12)(13)(14)(15).

Methods
The ProMedica Institutional Review Board provided a letter of exemption for this evaluation.The electronic databases within the health system and the cancer registry were queried for all patients undergoing pancreatic cancer resection from January 1, 2000 to December 31, 2010.In total, 91 patient charts from three hospitals in the system were identified and reviewed.Variables collected included patient demographics (age, gender, comorbid conditions), tumor characteristics (location, stage, histology, grade, size), procedural information (date of surgery, surgeon, procedure type, operative time, operative blood loss, lymph nodes examined, margin status), hospital course details (length of stay, consults received, complications), as well as patient outcomes, including readmissions which may have occurred as a result of their surgical care.In order to construct a survival curve, date of death was also collected using the United States Death Index.
To assess compliance, quality indicators applying to surgical treatment of pancreatic cancer patients were identified from those put forth by Bilimoria et al in 2009 (11).In total, 37 of the 43 high-tomoderate quality indicators applied to pancreatic surgical treatment.Each quality indicator belongs to one of five domains which include structure, process, appropriateness, efficiency, and outcome.Compliance with these quality indicators is considered to be reached if the institution is concordant with the quality indicator in at least 90% of the patients undergoing resection; these overall rates of compliance are reported for each facility (11).In addition, each facility's compliance to individual quality indicators was calculated to evaluate differences between each hospital performing pancreatic resections within the health system; these are presented as percent of cases considered compliant to allow for comparisons between each facility.
Structure.To evaluate structural compliance, institution volume and surgeon volume were calculated for each year.Patient charts were reviewed to ensure that all patients were treated in a multidisciplinary effort with a surgeon, medical oncologist, and a radiation oncologist.Hospital services were also evaluated to ensure that the institution was able to provide interventional radiation services, intensive care services, endoscopic ultrasonography services, radiation and chemotherapy services, as well as endoscopic retrograde cholangiopancreatography services.All surgeons underwent verification of certification by the American Board of Surgery or equivalent international institution.Each facility was also evaluated on their participation in clinical trials.
Process.In order to evaluate process compliance, patient charts were reviewed for proper treatment protocol, including a history and physical with thorough preoperative risk assessment, a triple-phase, multislice CT or MRI, suspicious adenopathy evaluated by frozen section, use of the College of American Pathologists checklist, macroscopically clear margins, and proper documentation of disease and treatment in the patient chart.
Appropriateness.Treatment appropriateness compliance was assessed by reviewing all charts to ensure that following resection, all patients underwent adjuvant chemotherapy or chemoradiation; if they did not, a reason must be documented.It was also important to ensure that, of the patients resected, none had documented stage IV clinical disease.
Efficiency.Elapsed time from diagnosis to treatment was calculated to ensure that not more than two months had passed in order to evaluate efficiency compliance; time to diagnosis was abstracted from the cancer registry and is defined by the Commission on Cancer as the "first date diagnosis was clinically or histologically established".Additionally, patient charts were reviewed for operative times and readmissions within 30 days of their surgical procedures.
Outcome.Institution margin-negative resection rate, 30-, 60-, and 90-day perioperative mortality, as well as 2-year and 5-year survival rates were calculated to objectively evaluate surgical outcomes.

Statistical Methods
Survival curves were constructed using the Kaplan Meier method.Student t-tests and analysis of variance (ANOVA) tests were used for mean comparison, while Fisher's exact tests were used for comparing proportions.P-values less than 0.05 were considered statistically significant.All analyses were completed using R version 2.15.0 (16).

Results
Patient Characteristics.91 patients underwent pancreatic resection at one of the three hospitals in our health system offering this treatment between the years of 2000 and 2010.The average age at diagnosis of all patients was 66.6±11.5 years, 52.7% were female.There were no statistically significant differences between the cohorts of patients treated at each of the separate hospitals with regard to age, gender, number of coexisting comorbidities, or tumor characteristics.Demographic and baseline patient characteristics are reported in Table 1.
Adherence.The quality indicators evaluated are stratified into five separate domains: structure, process, appropriateness, efficiency, and outcome.Evaluation of the structure-related quality indicators reveals that each of our included facilities are adherent to 8 of the 11 (72.7%)defined indicators.Because there is no variability in overall adherence to the majority of domains, the percent of cases in which provided care was concordant with defined indicators is reported in Table 2. Compliance to individual indicators ranges from 0% to 100%; all patients were treated in concordance with all indicators for which the facilities were compliant.Surgical volume and lack of endoscopic ultrasonography services on-site accounted for 2 of the 3 structure-related indicators to which the facility did not adhere.Finally, assessment of "efficiency" and "outcome" domains indicated adherence to all 7 relevant quality indicators in each facility; concordance with prescribed care ranged from 95.6% to 100%.
Outcomes.Overall 60 day perioperative mortality rate for the entire study period was 13.2%, which is comparable to other low-tomoderate volume facilities.Additionally, 5-year overall survival and 5-year conditional survival were comparable to those rates reported for high volume centers (Table 3) (6).Observed annual survival rates as a function of stage of disease were compared to data abstracted from the American College of Surgeons Commission on Cancer; the survival rates of our facilities were comparable to the national data for the study period (Table 4).Furthermore, the complication rate and number of readmissions within 30 days were recorded.The overall complication rate for all three facilities was 45.1%, ranging from 36.2% to 70.0%, while the 30-day readmission rate was 24.2% overall.
The number of resections occurring each year varied; as few as 5 were performed one year, while 12 were performed the next.Furthermore, the 91 resections occurred in 3 separate hospitals; over the study period the three hospitals performed 47, 34, and 10 resections, respectively.Mortality rates varied significantly between the three facilities (Figure 1); however, we found no statistically significant differences in the number of readmissions or complication rates between facilities.Furthermore, no significant differences were uncovered in the rate of compliance between the three facilities, despite the fact that the volumes of surgeries performed at each hospital were significantly different from one another.While 30-, 60-, and 90-day survival rates were positively associated with increased surgical volume in our three facilities (p = 0.001, p= 0.004, and p= 0.002, respectively, Table 5), these rates were not associated with overall compliance to the quality indicators evaluated.Over the ten year span of data collected, no temporal trends were identified with respect to adherence or outcomes In order to determine whether any variables might be associated with 30-and 60-day mortality rates, student t-tests and chi square analyses were completed.The number of pre-existing comorbidities was not associated with 30-day mortality rates (p=0.12),but was significantly associated with 60-day mortality rates (p=0.007).Furthermore, blood loss and surgeon volume were not associated with 30-or 60-day mortality rates (Table 6).However, operative time was significantly associated with 30-and 60-day mortality (Table 6).Surgeries were approximately 100 minutes longer in those patients who were alive after 60 days when compared to those who had passed during that time, suggesting that operative time may be related to 30-and 60-day outcomes.

Discussion
Our data indicate that our institution adheres well to validated quality indicators for pancreatic cancer surgical care.Of interest, while all three facilities examined adhered very similarly to all prescribed quality indicators, the outcomes at each facility were significantly different from one another.Each facility investigated had very different surgical volumes, such that 30-, 60-, and 90-day mortality rates at our higher volume facilities were significantly better than those at our lower volume facilities.However, all facilities in our health system are considered low to low-moderate volume hospitals in pancreatic resection.Importantly, outcomes at all facilities are on par with reported outcomes at moderate volume hospitals, and many of our reported outcomes meet benchmarks for high volume centers (7).
Based on our analysis, adherence to quality indicators is not necessarily predictive of outcomes, as our adherence rates did not vary between facilities while the outcomes did significantly.In fact, the hospital with highest overall level of adherence (30 of 37 indicators, 81.1%) exhibited the lowest survival rates.Though these quality indicators may help improve processes and quality of care provided, it cannot be expected that adhering perfectly to all indicators will significantly improve outcomes, in part due to the nature of pancreatic cancer in which cure is quite rare and survival rates are expected to be low.
Because our adherence to quality indicators was not strongly associated with outcomes at each facility, other factors that may be predictive of survival were investigated (17)(18)(19)(20)(21); surgeon volume, number of pre-existing comorbidities, intraoperative blood loss, and even stage were not predictive of 30-day mortality.However, we did uncover an interesting trend; operative time was associated with mortality, such that patients who died within 90 days postoperatively had significantly shorter operative times than those patients who survived this postoperative period.
While our data support previous claims that higher volume surgical centers boast better outcomes, we found no evidence to support the hypothesis that surgeons who perform a larger volume of pancreatic surgeries in our institution can necessarily do the same.When compared to all other surgeons, the surgeon with the largest volume and lowest mean operative times actually had worse outcomes than all other surgeons combined; alternately, the second highest volume surgeon had better outcomes than the collective.This suggests that not only surgeon volume, but other surgical characteristics may be predictive of survival rates.
There are a number of lessons which can be learned from our assessment going forward.First, we learned that documentation omissions and errors are more common than we believed initially, and can be more detrimental than assumed.Efforts to correct this on all levels is being pursued, but it is important to realize that until such an audit is completed, the extent to which documentatin errors occur is likely grossly underestimated.As such, the authors recommend that cancer centers providing pancreatic cancer care engage in full evaluation of their processes and care provided using the validated indicators published by Bilimoria et al., as described herein (11).
Additionally, it must be recognized that outcomes may be due to factors outside of the process of care offered on an institutional level.Surgeon characteristics, particularly operative times, may be suggestive of mortality.Our investigation also suggested the possibility that pre-operative assessment and patient screening/selection may have a significant impact on outcomes.It is possible that not screening patients rigorously enough could contribute to variable outcomes in spite of adhering consistently to a number of care-related quality indicators.Importantly, during our audit, we recognized that 6 of our pancreatic resections were documented to have occurred in stage IV tumors according to AJCC 6th edition guidelines (22).One of these cases was a clear documentation error, two involved metastases to the spleen and would be considered stage III disease according to current AJCC 7th edition staging guideline (23).Of the remaining three cases in question, one was an emergent procedure, one was an endocrine neoplasm, and the final case involved metastases that were thought to be benign prior to surgery but were determined to be malignant postoperatively.In order to address our institution's shortcomings, we have developed an Oncology Grand Rounds program, with invited pancreatic cancer expert presenters, to better educate our clinicians.Furthermore, to address documentation errors and omissions, a preoperative AJCC template has been added to the operative notes, as a reminder to fully document disease staging.Finally, 36% of our patients did not receive adjuvant chemoradiation therapy, with no documented reason.In order to correct for the deficient number of patients receiving postoperative adjuvant chemoradiation therapy, our institution recognizes and is considering adopting the current paradigm shift to neoadjuvant chemoradiation therapy in patients with resectable pancreatic cancer.Evidence suggests that chemoradiation prior to surgery gives better chances of achieving negative margins; more importantly, restaging after this type of treatment will allow physicians to identify patients whose disease has progressed, making surgical resection ineffectual (24).The cost effectiveness of this strategy cannot be understated.

Study Limitations
The retrospective nature of this evaluation is an inherent weakness, as non-adherence rates may be overinflated due to inability to locate documentation; our analysis is also limited by the small sample size available.As such, though we included ten years of data, logistic regression was not possible to investigate some variables as small subgroups lead to unstable statistical estimates.Due to the small sample size available for analysis, our findings may not be generalizable to other settings.Additionally, it may also be due to this small sample size that we could not discern significant differences in the investigation of some variables, owing to type 2 error.Further investigation of how these variables relate to outcomes and quality indicators is warranted.Furthermore, we found that in order to appropriately compare all mortality data to previously reported data in the literature, investigators should calculate risk-adjusted mortality.However, this requires detailed data, which were not available in all of our patient charts.We suggest that moving forward, even community hospitals such as ours begin prospectively collecting all data necessary for risk-adjusted mortality calculations.To this end, the decision has been made within our institution to begin participation in the American College of Surgeons' National Surgical Quality Improvement Program (NSQIP).Participation on a community level is recommended, as most pancreatic surgeries occur in this setting.
Our data indicate that adherence to validated quality indicators in our institution was not directly associated with survival rates, which may call in to question the clinical value of those quality indicators.Though our results do not support clinical integration of all of these indicators, we advocate using these indicators to assess the process of care provided as well as the quality of documentation, as the information gained by our cancer center during this assessment was found to be of great value.This type of evaluation allows identification of areas of weakness which might not be realized without careful examination, allowing any shortfalls to be addressed appropriately.Further study of the clinical relevance of pancreatic cancer quality indicators is warranted.

Table 1 :
Patient demographics and baseline disease characteristics.

Table 2 :
Rates of adherence to quality indicators by domain and overall survival.
Three of the process-related indicators were excluded for evaluation due to unavailability of data or inapplicability of the defined

Table 3 :
(6)parison of survival in pancreatic cancer patients in high and low volume mortality rates(6)to those within our health system.60-dayunadjusted perioperative mortality rate b 5 year survival for patients surviving the perioperative period (excludes in-hospital deaths).indicator,as we included only patients who underwent pancreatic resection.Evaluation of documentation within patient charts revealed that our institution's care was concordant with 11 (64.7%), 10 (58.8%), and 14 (82.4%) of the 17 defined indicators in hospitals 1, 2, and 3, respectively.This was the only domain to exhibit variability in overall adherence between facilities.Compliance with processrelated indicators ranged from 20.9% to 100% in individual cases; documentation errors and omissions were the primary cause of nonadherence to indicators in this category.Similarly, evaluation of the concordance with indicators in the "appropriateness" domain demonstrates adherence to 1 of the 2 (50%) indicators assessed in each facility; non-adherence was related to documentation errors or omissions. a

Table 4 :
Observed (obs) survival rates in our health system compared to national (natl) rates, as reported by the American College of Surgeons Commission on Cancer, during the same time period.
b at least one patient censored due to diagnosis date after 10/11/2010 c at least one patient censored due to surgery date after 10/11/20 d at least one patient censored due to surgery date after 10/11/2008