Can an End-to-End Telesepsis Solution Improve the Severe Sepsis and Septic Shock Management Bundle-1 Metrics for Sepsis Patients Admitted From the Emergency Department to the Hospital?
IMPORTANCE: Early detection and treatment for sepsis patients are key components to improving sepsis care delivery and increased The Severe Sepsis and Septic Shock Management Bundle (SEP-1) compliance may correlate with improved outcomes.
OBJECTIVES: We assessed the impact of implementing a partially automated end-to-end sepsis solution including electronic medical record-linked automated monitoring, early detection, around-the-clock nurse navigators, and teleconsultation, on SEP-1 compliance in patients with primary sepsis, present at admission, admitted through the emergency department (ER).
DESIGN, SETTING AND PARTICIPANTS: After a “surveillance only” training period between September 3, 2020, and October 5, 2020, the automated endto-end sepsis solution intervention period occurred from October 6, 2020, to January 1, 2021 in five ERs in an academic health system. Patients who screened positive for greater than or equal to 3 sepsis screening criteria (systemic inflammatory response syndrome, quick Sequential Organ Failure Assessment, pulse oximetry), had evidence of infection and acute organ dysfunction, and were receiving treatment consistent with infection or sepsis were included.
MAIN OUTCOMES AND MEASURES: SEP-1 compliance during the “surveillance only” period compared to the intervention period.
RESULTS: During the intervention period, 56,713 patients presented to the five ERs; 20,213 (35.6%) met electronic screening criteria for potential sepsis; 1,233 patients had a primary diagnosis of sepsis, present at admission, and were captured by the nurse navigators. Median age of the cohort was 68 years (interquartile range, 57–79 yr); 55.3% were male; 63.5% were White/Caucasian, 26.3% Black/African-American; was 16.7%, and 879 patients (71.3%) were presumed bacterial sepsis, nonviral etiology, and SEP-1 bundle eligible. Nurse navigator real-time classification of this group increased from 51.7% during the “surveillance only” period to 71.8% during the intervention period (p = 0.0002). Five hospital SEP-1 compliance for the period leading into the study period (July 1, 2020–August 31, 2020) was 62% (p < 0.0001), during the “surveillance only” period, it was 68.4% and during the intervention period it was 78.3% (p = 0.002).
CONCLUSIONS AND RELEVANCE: During an 11-week period of sepsis screening, monitoring, and teleconsultation in 5 EDs, SEP-1 compliance improved significantly compared with institutional SEP-1 reporting metrics and to a “surveillance only” training period.
KEY WORDS: identification; resuscitation; The Severe Sepsis and Septic Shock Management Bundle-1; sepsis; telemedicine; time-sensitive illness
Sepsis, the syndrome of the body’s pathophysiologic response to infection, is common and deadly (1–3). Early identification and timely initiation of treatment for patients with sepsis (4–7) have resulted in improvements in outcomes (8–11). Sepsis Quality Improvement initiatives, including the Centers for Medicare & Medicaid Services (CMS) the SEP-1 3- and 6-hour bundles (12), have relied on the 1991 American College of Chest Physicians/Society of Critical Care Medicine Consensus Conference definition of severe sepsis, which was subsequently modified in 2001 at the 2nd International Sepsis Definitions Conference (13– 15). CMS SEP-1 captures patients retrospectively based on ICD-10 codes for severe sepsis and then applies inclusion and exclusion criteria to identify a sample of an institution’s sepsis patients. Conflicting studies exist about the relationship between SEP-1 compliance and sepsis outcomes, some suggesting increased compliance is associated with improved outcomes, others not supporting this relationship (16–19). Current national compliance rate with SEP-1 metrics is approximately 57% (20).
Thomas Jefferson University Hospitals (TJUH) has undertaken multiple process improvement initiatives to address the timeliness of sepsis recognition, optimize treatment, and improve outcomes. We have participated in the Vizient (www.vizientinc.com; Irving, TX) sepsis registry and reported our sepsis data to the CMS SEP-1 since its inception. Our Observed:Expected mortality ratio for severe sepsis cases admitted to the hospital from our two emergency departments (EDs) was 1.21 for the period from July 1, 2019, to June 30, 2020. Compliance with SEP-1 metrics is approximately 67% (20).
To address opportunities for improvement, we designed a telesepsis pilot integrating automated data capture, live nurse navigator monitoring, and remote “sepsis experts” available for telesepsis consultations by video telemedicine consult carts (Teladoc, Purchase, NY) (Fig. 1). We hypothesized, that the multiple, iterative assessments would provide the best chance to optimize sepsis care, as measured by SEP-1 compliance. The study’s primary hypothesis is that an endto-end sepsis solution will improve compliance with individual SEP-1 bundle elements and overall SEP-1 compliance. The study’s secondary hypotheses include that the sepsis solution will: 1) produce greater than or equal to 0.5 contacts/patient encounter, defined as a discrete interaction between the nurse navigators and the clinical staff and 2) accurately identify the sickest cohort of patients—those with an initial lactate greater than 4 mmol/L; persistent hypotension (systolic blood pressure [SBP] < 90mm Hg or mean arterial pressure [MAP] < 65mm Hg) after a 30 cc/kg fluid bolus; or at least 3 AODs—real-time and arrange telesepsis consultation on the majority of eligible patients.
METHODS
Study Population and Design
This was a prospective, observational cohort study conducted between September 3, 2020, and January 1, 2021, of consecutive adult (≥ 18 yr old) patients who presented to the ED and were triaged. Patients who were registered in the ED but left prior to triage were excluded. Patients were enrolled at five, urban and suburban, community and teaching hospitals, with annual ED volumes ranging from 30,000 to 80,000 visits, inpatient bed capacity ranging from 230 to 750 beds, institution-specific sepsis protocols, an enterprise-wide sepsis definition, and ICU capacity ranging from 24 to 150 ICU beds. A “surveillance only” period occurred between September 3, 2020, and October 5, 2020, during which nurse navigator coverage ranged from 8 to 16 hours a day, Monday to Friday, with offsite supervision from one researcher (D.F.G.). We used Ambient Clinical Analytics Sepsis DART (referred to as “DART” in the article), which is a healthcare information system integrating electronic medical record (EMR) data with live healthcare navigators via smart alerts, producing actionable information about sepsis patients. Nurse navigators were trained to monitor “Under Surveillance” and “Potential Sepsis” patients in the DART system and classify them as “Sepsis Monitoring,” where the patient meets criteria for sepsis and is actively being treated, “Alternative Diagnosis” or “Silence” the patient for 15 minutes until more data are available. After the 15-minute silence period, the patients were reassessed to see if they qualify.
All nurse navigators were hired through a temporary staffing agency, were licensed nurses with at least 5 years of nursing experience, and relevant ED, ICU, or medical-surgical experience. They were trained during didactic sessions where the clinical workflow was learned, utilizing five mock cases of potential sepsis patients, and during the “surveillance only” period, where real patients were monitored but interventions were not performed. Clinical workflows were developed for each SEP-1 metric, addressing whether the metric was not ordered (e.g., first lactate), ordered but not completed (e.g., antibiotics), or potentially completed but not documented (e.g., perfusion reassessment) and delineating whom the nurse navigator should contact first to address the issue and next steps if unsuccessful (see Supplementary Material, Clinical Workflows, http://links.lww.com/CCX/B62).
A staggered “go-live” period occurred from October 5, 2020, to October 21, 2020, with all five EDs activated by October 21, 2020. This investigation was performed as a quality improvement project and approved by TJUH’s administration. Prior to data analysis and submission for publication, the study was approved by the TJUH Institutional Review Board and granted a waiver of informed consent (protocol number 21P.021; approval date January 21, 2021). All research was performed in accordance with the ethical standards of the TJUH Institutional Review Board and the Helsinki Declaration of 1975.
All triaged adult patients who were either in the waiting room or in a treatment area, were captured by the DART tracking system, labeled as “Under Surveillance,” tracked with real-time data monitoring, and included in the database. Additional data including vital signs, laboratories, antibiotics, fluids, and vasopressors given to a patient from ED presentation until physical departure from the ED were automatically downloaded into DART, which was monitored by the trained nurse navigators, working assigned shifts in a sepsis-monitoring bunker, and interpreting the streaming data.
When a patient met at least three screening criteria—consisting of a chief complaint consistent with a suspected or confirmed diagnosis of infection, any of the four systemic inflammatory response syndrome criteria (13), any of the three quick Sequential Organ Failure Assessment criteria (1), and any of the acute organ dysfunction (AOD) criteria recognized in the 2001 2nd International Sepsis Definitions (14)— DART automatically changed the patient’s designation from “Under Surveillance” to “Potential Sepsis.” All “Potential Sepsis” patients were screened by the trained nurse navigators real-time in an iterative fashion, repeatedly looking for evidence of confirmed or suspected infection and AOD (see Supplementary Material, http://links.lww.com/CCX/B62).
When a patient was confirmed as “Sepsis Monitoring,” the nurse navigator set Time Zero— the time of the first documented AOD—and SEP-1 3- and 6-hour metric compliance was tracked. When the time remaining to complete a bundle element was less than or equal to 90 minutes, DART automatically notified the nurse navigators who followed predetermined workflows designed to maximize the percentage of SEP-1 metrics successfully completed (see Supplemental Material, http://links.lww.com/ CCX/B62). When the nurse navigators reached out to the clinical staff, they could address one or more outstanding issues per contact. Further, the bedside clinical staff caring for patients who met predetermined “telesepsis consult eligible” criteria (initial lactate > 4 mmol/L; persistent hypotension [SBP < 90mm Hg or MAP < 65mm Hg] after a 30 cc/kg fluid bolus; or ≥ 3 sepsis-related AODs), were contacted by the nurse navigators to arrange a telesepsis consultation, including instructions for movement of the telemedicine consult cart into the patient’s room (Fig. 1).
The telesepsis consultants were available around-the-clock during the intervention period. The 17 consultants, were on call for 12-hour shifts (7 am–7 pm; 7 pm–7 am), and were board certified either in pulmonary and critical care medicine (10) or Emergency Medicine (7). The consultants were considered sepsis experts based on their training, involvement in enterprise sepsis improvement projects, and expressed interest in the project. DART automatically notified the nurse navigators when all SEP-1 metrics were complete and the patient was automatically classified as SEP-1 compliant. Because of limitations in the Epic Systems Corporation-DART interface, repeat lactate values, vasopressor administration, and perfusion reassessments performed after the patient left the ED but within the mandated SEP-1 time frame were not available real time and had to be retrospectively obtained.
Data Collection
Data were electronically transferred from the EMR into DART and then transferred into Excel for statistical analysis. Severe sepsis patients presenting to the ED during the “surveillance only” period comprised the “surveillance only” convenience cohort used to establish baseline SEP-1 compliance. We also collected ED length of stay (LOS), based on the total time physically in the ED whether as an ED patient or a boarding in patient, hospital LOS, ICU LOS, discharge diagnoses, and discharge location. These additional data were either manually abstracted from the patient EMR by one author (D.F.G.) or automatically extracted from Epic Systems Corporation, using Qlik, an embedded data analytics tool. Qlik captured all patients admitted to the five hospitals during the study period who were assigned a severe sepsis (R65.20) or septic shock (R65.21) diagnostic code, present at admission (POA), at discharge. For calculation of SEP-1 compliance, the cohort was limited to presumed bacterial sepsis patients since viral sepsis is an exclusion criterion in SEP-1 reporting. Viral sepsis was defined as a positive rapid polymerase chain reaction test for influenza, severe acute respiratory syndrome coronavirus 2, or respiratory syncytial virus without evidence of concomitant bacterial infection (no antibiotics being administered in the ED or over the first 2 d of hospitalization). All data were stored on a secure, password-protected server at TJUH.
Statistical Analysis
For descriptive analysis, continuous data were expressed as means (± sds) if normally distributed and as medians (interquartile range [IQR]) if not normally distributed. Differences were tested using analysis of variance testing. Categorical variables were presented as percentages and analyzed using the chi-square test or Fisher exact method.
RESULTS
Step-Wise Identification of Patients Presenting to the ED With Severe Sepsis
During the study period, 55,714 adult patients were triaged in the five EDs and entered into surveillance in DART; 20,218 (36.3%) of these met greater than or equal to 3 sepsis screening criteria; 1,332 of these patients (6.6%) were identified by the nurse navigators as having severe sepsis, eligible for SEP-1 bundled care, and 1,233 of these (92.6%) were confirmed by one of the sepsis experts (D.F.G.) (Supplemental Fig. 1, http://links.lww.com/CCX/B63).
Demographics and Outcomes
Patients in the “surveillance only” cohort had a median age of 66 years (IQR, 55–79 yr); 52.2% were male; 64.7% were White and 28% Black. Patients in the intervention period cohort had a median age of 68 years (IQR, 57–79 yr); 55.3% were male; 63.5% were White and 26.3% Black (Table 1). During the “surveillance only” period, 90.7% (234/258) had presumed bacterial sepsis, while 9.3% (24/258) had confirmed viral sepsis. During the intervention period, 71.3% (879/1,233) had presumed bacterial sepsis, while 28.7% (354/1,233) had confirmed viral sepsis (Table 2). In-hospital mortality was 8.5% (22/258) during the “surveillance only” period and 15.7% (194/1,233) during the intervention period. During the “surveillance only” period, 91.5% (236/258) were discharged to home and 52.3% (136/258) were discharged to home; during the intervention period; 84.3% (1,039/1,233) were discharged and 57.8% (600/1,039) were discharged to home (Table 1). Three-hundred four patients were admitted to the five hospitals during the study period and assigned a severe sepsis (R65.20) or septic shock (R65.21) diagnostic code, POA, at discharge; 100% (304/304) of these were captured by the end-to-end solution.
Primary Hypothesis: SEP-1 Metric Compliance in Historic Controls Versus End-to-End Sepsis Solution Study Period
During the period prior to the study (July 1, 2020– August 31, 2021), the five hospitals’ reported SEP-1 compliance was 62% and the New Jersey, Pennsylvania, and national averages were 65%, 55%, and 57%, respectively. During the “surveillance only” and intervention periods, real-time SEP-1 compliance for presumed bacterial sepsis cases as adjudicated by the nurse navigators was 51.7% and 71.8%, respectively (p = 0.0002); after review by a sepsis expert (D.F.G.) for data points not available real time, the SEP-1 compliance for the “surveillance only” and intervention periods was 68.4%, and 78.3%, respectively (p = 0.002) (Supplemental Table 1, http://links.lww.com/CCX/B61). For individual items, during the “surveillance only” period versus intervention period, after review, the individual item pass rates included: first lactate 93.4% versus 99% (p < 0001); blood cultures before antibiotics, 92.5% versus 98.5% (p < 0001); appropriate antibiotics within 3 hours of time zero, 92.3% versus 97.9% (p < 0001) (Table 2).
Secondary Hypotheses Regarding the End-to-End Sepsis Solution
Confirmation of Sepsis Patients. During the “surveillance only” period, the nurse navigators and the sepsis expert confirmed 258 patients with severe sepsis; during the intervention period, the nurse navigators confirmed 1,332 patients and started the sepsis timer; 1,233 of these patients (92.6%) were confirmed as having a primary diagnosis of sepsis, POA, with a mean of 14 patients confirmed/d. Ninety-nine patients (7.4%) classified as sepsis by the nurse navigators were false positives, yielding a positive predictive value of 93% for a nurse navigator categorizing a patient as primary sepsis, POA, preliminarily eligible for SEP-1 bundle compliance (Supplemental Fig. 1, http://links.lww. com/CCX/B63).
Contacts With Clinical Staff. The nurse navigators made discrete 820 contacts with the ED staff over the intervention period (0.67/patient), a mean of 10 contacts/d (Fig. 2). These contacts included calls placed primarily to clarify whether a patient was septic (13%) and for completion of individual aspects of bundle compliance including first lactate (12%), antibiotic administration (15%), and obtaining a second lactate (15%). In addition, 21% of the calls were to request a telesepsis consultation on consult eligible patients (Supplemental Fig. 2, http://links.lww.com/ CCX/B63).
Consultations. Two-hundred forty-five patients met telesepsis consultation criteria. The nurse navigators identified 204 of these patients (sensitivity 83.3%). The 204 calls to ED staff requesting telesepsis consultation resulted in 71 consults (34.8%) being performed. The average time from identification of a patient’s meeting consult criteria to connection with the telemedicine consult cart was 25 minutes; the average length of a consult was 11 minutes. The 71 telesepsis consultations yielded 308 clinical recommendations, an average of 4.3 recommendations/patient (Supplemental Fig. 3, http://links.lww.com/CCX/B63).
DISCUSSION
In this study, we found that a partially automated endto-end sepsis solution, produced a statistically significant increase in overall SEP-1 metric compliance. For presumed bacterial sepsis patients, overall SEP-1 compliance increased from 68.4% in the “surveillance only” period to 78.3% in the intervention period (p = 0.002). We saw improvements in all SEP-1 reporting metrics except for vasopressor administration from the “surveillance only” to the intervention period. These findings support the ability of a combination of automated monitoring and remote, live nurse navigators to identify SEP-1 eligible patients and interact with clinical staff to increase SEP-1 compliance. Because of the relatively short intervention period in our study and the complexities presented by the second wave of the COVID pandemic, we did not compare differences between the “surveillance only” period and the intervention period for clinical outcomes such as inhospital mortality (IHM), ED LOS, hospital LOS, or discharge location.
Similar to the study by Barbash et al (16), we found that our intervention produced variable changes in the processes of sepsis resuscitation care articulated in SEP1. They conclude that SEP-1 appears to have improved metrics indirectly associated (e.g., first lactate and repeat lactate) more than those directly associated (e.g., early appropriate antibiotics) with improved clinical outcomes. Our findings are different in that our results demonstrated a statistically significant increase in administration of antibiotics, a measure directly associated with improved clinical outcomes (16, 21–23). It is difficult to make further comparisons between the two studies, however, because of significant differences in the study population and metrics of care. For example, the cohort by Barbash et al (16) had first lactate measured within 3 hours in 70.2% of the patients versus 99% in our intervention period and first antibiotics administered within 3 hours in 49.8% of patients versus 98.5% in our intervention period. Further, IHM for all patients in the cohort by Barbash et al (16) was 4.2% in the SEP-1 period versus 15.7% in our cohort (16).
A central issue to optimal sepsis care is early detection. Patients who present to the ED with unstable vital signs are identified early and prioritized for management. Less obviously unstable patients can have multiple AODs yet be assigned a lower triage class and wait extended periods for initiation of management. The automated sepsis screening system used in this pilot was able to identify the vast majority of primary sepsis, POA, patients real-time during triage, waiting room stay, or after placement in an ED room. Often patients alarmed as “Potential Sepsis” and the nurse navigators silenced the patient for 15 minutes waiting for the clinical team to assess the patient, develop a differential diagnosis, and initiate sepsis management. After the training period, the nurse navigators were highly accurate in their classification with a positive predictive value of 93%.
Despite conflicting data on whether improved SEP-1 compliance correlates with improved outcomes, increased attention and review of complex patients makes intuitive sense and, similar to Mohr et al (24), we demonstrated that this can be done remotely, augmented by teleconsultation. Further, in the cross-checking to reduce adverse events resulting from medical errors in the emergency department study, systematic cross-checking of care plans between emergency physicians reduced medical errors with the greatest reduction being for errors in sepsis care, including time to antibiotic administration, fluid therapy, identification of source of infection, and choice of antibiotics (25). These are exactly the same items that are reviewed during telesepsis consultations, providing the possibility of reviewing complex patients prior to admission.
This study has several limitations. First, it was performed in a health system with above-average SEP-1 compliance and the results of our study may not generalize to other hospitals or health systems with different resources and higher or lower baseline SEP-1 compliance. Second, the 11-week-long intervention period was too short to measure differences in hard outcomes such as IHM or discharge location. Third, our study was conducted during an upswing in COVID cases, which can impact metrics of sepsis care because of crowding, boarding, and ED volume. Fourth, our definitions of SEP-1 reporting qualification and compliance do not include all of the exclusion criteria in the complex SEP-1 reporting rules. Therefore, our compliance numbers may be higher or lower than “real” compliance as adjudicated by our SEP-1 abstractors. However, this real-time approach to probable SEP-1 patients is more realistic and captures patients who should be managed in a prospective, time-sensitive fashion. Fifth, there is debate about the relationship of SEP-1 compliance to Figure 2. Contacts with emergency department (ED) staff. Observational Study Critical Care Explorations www.ccejournal.org 9 outcomes for sepsis care; the clinical significance of improved SEP-1 metrics is not addressed in this study. Sixth, except in the case of contacts requesting telesepsis consultation, we did not document whether an individual contact from the nurse navigators led to the suggested action. Seventh, because the interventions stopped at the end of the study period, we have no way of knowing if the improvements in SEP-1 compliance would have been maintained and do not have data to examine whether they returned to the surveillance only baseline. Finally, the realities of staffing a novel nurse navigator role during a nationwide nursing shortage may pose serious challenges to reproducing the described workflow.
CONCLUSIONS
During an 11-week period, a partially automated endto-end sepsis solution improved SEP-1 compliance in patients with primary sepsis, POA, admitted through the ED.
ACKNOWLEDGMENTS
We would like to acknowledge the nurse navigators hired through JeffTemps specifically for this project. We would also like to acknowledge the participation of the bedside clinical staff and the on call sepsis teleconsultants who participated in this performance improvement initiative. Without them, we would not have been able to complete this project.
REFERENCES
1. Singer M, Deutschman CS, Seymour CW, et al: The third in ternational consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016; 315:801–810
2. Gaieski DF, Edwards JM, Kallan MJ, et al: Benchmarking the incidence and mortality of severe sepsis in the United States. Crit Care Med 2013; 41:1167–1174
3. Kaukonen KM, Bailey M, Suzuki S, et al: Mortality related to severe sepsis and septic shock among critically ill patients in Australia and New Zealand, 2000-2012. JAMA 2014; 311:1308–1316
4. Mikkelsen ME, Miltiades AN, Gaieski DF, et al: Serum lac tate is associated with mortality in severe sepsis inde pendent of organ failure and shock. Crit Care Med 2009; 37:1670–1677
5. Howell MD, Donnino MW, Talmor D, et al: Performance of severity of illness scoring systems in emergency depart ment patients with infection. Acad Emerg Med 2007; 14:709–714
6. Shapiro NI, Howell MD, Talmor D, et al: Implementation and outcomes of the Multiple Urgent Sepsis Therapies (MUST) protocol. Crit Care Med 2006; 34:1025–1032
7. Trzeciak S, Dellinger RP, Chansky ME, et al: Serum lactate as a predictor of mortality in patients with infection. Intensive Care Med 2007; 33:970–977
8. Rivers E, Nguyen B, Havstad S, et al; Early Goal-Directed Therapy Collaborative Group: Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med 2001; 345:1368–1377
9. Yealy DM, Kellum JA, Huang DT, et al; The ProCESS Investigators: A randomized trial of protocol-based care for early septic shock. N Engl J Med 2014;370:1683–1693
10. Peake SL, Delaney A, Bailey M, et al; The ARISE Investigators and the ANZICS Clinical Trials Group: Goal-directed resus citation for patients with early septic shock. N Engl J Med 2014;371:1496–1506
11. Mouncey PR, Osborn TM, Power GS, et al; The ProMISe Trial Investigators: Trial of early, goal-directed resuscitation for septic shock. N Engl J Med 2015; 372:1301–1311
12. Centers for Medicare & Medicaid Services: SEP-1 Bundles. 2022. Available at: https://cmit.cms.gov/CMIT_public/ ViewMeasure?MeasureId=1017. Accessed August 1, 2022
13. Bone RC, Balk RA, Cerra FB, et al: Definitions for sepsis and organ failure and guidelines for the use of innovative thera pies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992; 101:1644–1655
14. Levy MM, Fink MP, Marshall JC, et al; International Sepsis Definitions Conference: 2001 SCCM/ESICM/ACCP/ATS/ SIS international sepsis definitions conference. Intensive Care Med 2003; 29:530–538 Gaieski et al 10 www.ccejournal.org October 2022 • Volume 4 • Number 10
15. Kaukonen KM, Bailey M, Pilcher D, et al: Systemic inflamma tory response syndrome criteria in defining severe sepsis. N Engl J Med 2015; 372:1629–1638
16. Barbash IJ, Davis BS, Yabes JG, et al: Treatment patterns and clinical outcomes after the introduction of the Medicare Sepsis Performance Measure (SEP-1). Ann Intern Med 2021; 174:927–935
17. Baghdadi JD, Brook RH, Uslan DZ, et al: Association of a care bundle for early sepsis management with mortality among patients with hospital-onset or community-onset sepsis. JAMA Intern Med 2020; 180:707–716
18. Rhee C, Filbin MR, Massaro AF, et al; Centers for Disease Control and Prevention (CDC) Prevention Epicenters Program: Compliance with the national SEP-1 quality measure and association with sepsis outcomes: A mul ticenter retrospective cohort study. Crit Care Med 2018; 46:1585–1591
19. Venkatesh AK, Slesinger T, Whittle J, et al: Preliminary perfor mance on the new CMS Sepsis-1 National Quality Measure: Early insights from the Emergency Quality Network (E-QUAL). Ann Emerg Med 2018; 71:10–15.e1
20. Centers for Medicare & Medicaid Services: SEP-1 Bundles. 2022. Available at: https://www.medicare.gov/care-compare/ details/hospital//. Accessed August 1, 2022
21. Kumar A, Roberts D, Wood KE, et al: Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 2006; 34:1589–1596
22. Gaieski DF, Mikkelsen ME, Band RA, et al: Impact of time to antibiotics on survival in patients with severe sepsis or septic shock in whom early goal-directed therapy was initiated in the emergency department. Crit Care Med 2010; 38:1045–1053
23. Seymour CW, Gesten F, Prescott HC, et al: Time to treatment and mortality during mandated emergency care for sepsis. N Engl J Med 2017; 376:2235–2244
24. Mohr NM, Campbell KD, Swanson MB, et al: Provider-to provider telemedicine improves adherence to sepsis bundle care in community emergency departments. J Telemed Telecare 2021; 27:518–526
25. Freund Y, Goulet H, Leblanc J, et al: Effect of systematic physician cross-checking on reducing adverse events in the emergency department: The CHARMED cluster randomized trial. JAMA Intern Med 2018; 178:812–819
Sepsis, the syndrome of the body’s pathophysiologic response to infection, is common and deadly (1–3). Early identification and timely initiation of treatment for patients with sepsis (4–7) have resulted in improvements in outcomes (8–11). Sepsis Quality Improvement initiatives, including the Centers for Medicare & Medicaid Services (CMS) the SEP-1 3- and 6-hour bundles (12), have relied on the 1991 American College of Chest Physicians/Society of Critical Care Medicine Consensus Conference definition of severe sepsis, which was subsequently modified in 2001 at the 2nd International Sepsis Definitions Conference (13– 15). CMS SEP-1 captures patients retrospectively based on ICD-10 codes for severe sepsis and then applies inclusion and exclusion criteria to identify a sample of an institution’s sepsis patients. Conflicting studies exist about the relationship between SEP-1 compliance and sepsis outcomes, some suggesting increased compliance is associated with improved outcomes, others not supporting this relationship (16–19). Current national compliance rate with SEP-1 metrics is approximately 57% (20).
Thomas Jefferson University Hospitals (TJUH) has undertaken multiple process improvement initiatives to address the timeliness of sepsis recognition, optimize treatment, and improve outcomes. We have participated in the Vizient (www.vizientinc.com; Irving, TX) sepsis registry and reported our sepsis data to the CMS SEP-1 since its inception. Our Observed:Expected mortality ratio for severe sepsis cases admitted to the hospital from our two emergency departments (EDs) was 1.21 for the period from July 1, 2019, to June 30, 2020. Compliance with SEP-1 metrics is approximately 67% (20).
To address opportunities for improvement, we designed a telesepsis pilot integrating automated data capture, live nurse navigator monitoring, and remote “sepsis experts” available for telesepsis consultations by video telemedicine consult carts (Teladoc, Purchase, NY) (Fig. 1). We hypothesized, that the multiple, iterative assessments would provide the best chance to optimize sepsis care, as measured by SEP-1 compliance. The study’s primary hypothesis is that an endto-end sepsis solution will improve compliance with individual SEP-1 bundle elements and overall SEP-1 compliance. The study’s secondary hypotheses include that the sepsis solution will: 1) produce greater than or equal to 0.5 contacts/patient encounter, defined as a discrete interaction between the nurse navigators and the clinical staff and 2) accurately identify the sickest cohort of patients—those with an initial lactate greater than 4 mmol/L; persistent hypotension (systolic blood pressure [SBP] < 90mm Hg or mean arterial pressure [MAP] < 65mm Hg) after a 30 cc/kg fluid bolus; or at least 3 AODs—real-time and arrange telesepsis consultation on the majority of eligible patients.
METHODS
Study Population and Design
This was a prospective, observational cohort study conducted between September 3, 2020, and January 1, 2021, of consecutive adult (≥ 18 yr old) patients who presented to the ED and were triaged. Patients who were registered in the ED but left prior to triage were excluded. Patients were enrolled at five, urban and suburban, community and teaching hospitals, with annual ED volumes ranging from 30,000 to 80,000 visits, inpatient bed capacity ranging from 230 to 750 beds, institution-specific sepsis protocols, an enterprise-wide sepsis definition, and ICU capacity ranging from 24 to 150 ICU beds. A “surveillance only” period occurred between September 3, 2020, and October 5, 2020, during which nurse navigator coverage ranged from 8 to 16 hours a day, Monday to Friday, with offsite supervision from one researcher (D.F.G.). We used Ambient Clinical Analytics Sepsis DART (referred to as “DART” in the article), which is a healthcare information system integrating electronic medical record (EMR) data with live healthcare navigators via smart alerts, producing actionable information about sepsis patients. Nurse navigators were trained to monitor “Under Surveillance” and “Potential Sepsis” patients in the DART system and classify them as “Sepsis Monitoring,” where the patient meets criteria for sepsis and is actively being treated, “Alternative Diagnosis” or “Silence” the patient for 15 minutes until more data are available. After the 15-minute silence period, the patients were reassessed to see if they qualify.
All nurse navigators were hired through a temporary staffing agency, were licensed nurses with at least 5 years of nursing experience, and relevant ED, ICU, or medical-surgical experience. They were trained during didactic sessions where the clinical workflow was learned, utilizing five mock cases of potential sepsis patients, and during the “surveillance only” period, where real patients were monitored but interventions were not performed. Clinical workflows were developed for each SEP-1 metric, addressing whether the metric was not ordered (e.g., first lactate), ordered but not completed (e.g., antibiotics), or potentially completed but not documented (e.g., perfusion reassessment) and delineating whom the nurse navigator should contact first to address the issue and next steps if unsuccessful (see Supplementary Material, Clinical Workflows, http://links.lww.com/CCX/B62).
A staggered “go-live” period occurred from October 5, 2020, to October 21, 2020, with all five EDs activated by October 21, 2020. This investigation was performed as a quality improvement project and approved by TJUH’s administration. Prior to data analysis and submission for publication, the study was approved by the TJUH Institutional Review Board and granted a waiver of informed consent (protocol number 21P.021; approval date January 21, 2021). All research was performed in accordance with the ethical standards of the TJUH Institutional Review Board and the Helsinki Declaration of 1975.
All triaged adult patients who were either in the waiting room or in a treatment area, were captured by the DART tracking system, labeled as “Under Surveillance,” tracked with real-time data monitoring, and included in the database. Additional data including vital signs, laboratories, antibiotics, fluids, and vasopressors given to a patient from ED presentation until physical departure from the ED were automatically downloaded into DART, which was monitored by the trained nurse navigators, working assigned shifts in a sepsis-monitoring bunker, and interpreting the streaming data.
When a patient met at least three screening criteria—consisting of a chief complaint consistent with a suspected or confirmed diagnosis of infection, any of the four systemic inflammatory response syndrome criteria (13), any of the three quick Sequential Organ Failure Assessment criteria (1), and any of the acute organ dysfunction (AOD) criteria recognized in the 2001 2nd International Sepsis Definitions (14)— DART automatically changed the patient’s designation from “Under Surveillance” to “Potential Sepsis.” All “Potential Sepsis” patients were screened by the trained nurse navigators real-time in an iterative fashion, repeatedly looking for evidence of confirmed or suspected infection and AOD (see Supplementary Material, http://links.lww.com/CCX/B62).
When a patient was confirmed as “Sepsis Monitoring,” the nurse navigator set Time Zero— the time of the first documented AOD—and SEP-1 3- and 6-hour metric compliance was tracked. When the time remaining to complete a bundle element was less than or equal to 90 minutes, DART automatically notified the nurse navigators who followed predetermined workflows designed to maximize the percentage of SEP-1 metrics successfully completed (see Supplemental Material, http://links.lww.com/ CCX/B62). When the nurse navigators reached out to the clinical staff, they could address one or more outstanding issues per contact. Further, the bedside clinical staff caring for patients who met predetermined “telesepsis consult eligible” criteria (initial lactate > 4 mmol/L; persistent hypotension [SBP < 90mm Hg or MAP < 65mm Hg] after a 30 cc/kg fluid bolus; or ≥ 3 sepsis-related AODs), were contacted by the nurse navigators to arrange a telesepsis consultation, including instructions for movement of the telemedicine consult cart into the patient’s room (Fig. 1).
The telesepsis consultants were available around-the-clock during the intervention period. The 17 consultants, were on call for 12-hour shifts (7 am–7 pm; 7 pm–7 am), and were board certified either in pulmonary and critical care medicine (10) or Emergency Medicine (7). The consultants were considered sepsis experts based on their training, involvement in enterprise sepsis improvement projects, and expressed interest in the project. DART automatically notified the nurse navigators when all SEP-1 metrics were complete and the patient was automatically classified as SEP-1 compliant. Because of limitations in the Epic Systems Corporation-DART interface, repeat lactate values, vasopressor administration, and perfusion reassessments performed after the patient left the ED but within the mandated SEP-1 time frame were not available real time and had to be retrospectively obtained.
Data Collection
Data were electronically transferred from the EMR into DART and then transferred into Excel for statistical analysis. Severe sepsis patients presenting to the ED during the “surveillance only” period comprised the “surveillance only” convenience cohort used to establish baseline SEP-1 compliance. We also collected ED length of stay (LOS), based on the total time physically in the ED whether as an ED patient or a boarding in patient, hospital LOS, ICU LOS, discharge diagnoses, and discharge location. These additional data were either manually abstracted from the patient EMR by one author (D.F.G.) or automatically extracted from Epic Systems Corporation, using Qlik, an embedded data analytics tool. Qlik captured all patients admitted to the five hospitals during the study period who were assigned a severe sepsis (R65.20) or septic shock (R65.21) diagnostic code, present at admission (POA), at discharge. For calculation of SEP-1 compliance, the cohort was limited to presumed bacterial sepsis patients since viral sepsis is an exclusion criterion in SEP-1 reporting. Viral sepsis was defined as a positive rapid polymerase chain reaction test for influenza, severe acute respiratory syndrome coronavirus 2, or respiratory syncytial virus without evidence of concomitant bacterial infection (no antibiotics being administered in the ED or over the first 2 d of hospitalization). All data were stored on a secure, password-protected server at TJUH.
Statistical Analysis
For descriptive analysis, continuous data were expressed as means (± sds) if normally distributed and as medians (interquartile range [IQR]) if not normally distributed. Differences were tested using analysis of variance testing. Categorical variables were presented as percentages and analyzed using the chi-square test or Fisher exact method.
RESULTS
Step-Wise Identification of Patients Presenting to the ED With Severe Sepsis
During the study period, 55,714 adult patients were triaged in the five EDs and entered into surveillance in DART; 20,218 (36.3%) of these met greater than or equal to 3 sepsis screening criteria; 1,332 of these patients (6.6%) were identified by the nurse navigators as having severe sepsis, eligible for SEP-1 bundled care, and 1,233 of these (92.6%) were confirmed by one of the sepsis experts (D.F.G.) (Supplemental Fig. 1, http://links.lww.com/CCX/B63).
Demographics and Outcomes
Patients in the “surveillance only” cohort had a median age of 66 years (IQR, 55–79 yr); 52.2% were male; 64.7% were White and 28% Black. Patients in the intervention period cohort had a median age of 68 years (IQR, 57–79 yr); 55.3% were male; 63.5% were White and 26.3% Black (Table 1). During the “surveillance only” period, 90.7% (234/258) had presumed bacterial sepsis, while 9.3% (24/258) had confirmed viral sepsis. During the intervention period, 71.3% (879/1,233) had presumed bacterial sepsis, while 28.7% (354/1,233) had confirmed viral sepsis (Table 2). In-hospital mortality was 8.5% (22/258) during the “surveillance only” period and 15.7% (194/1,233) during the intervention period. During the “surveillance only” period, 91.5% (236/258) were discharged to home and 52.3% (136/258) were discharged to home; during the intervention period; 84.3% (1,039/1,233) were discharged and 57.8% (600/1,039) were discharged to home (Table 1). Three-hundred four patients were admitted to the five hospitals during the study period and assigned a severe sepsis (R65.20) or septic shock (R65.21) diagnostic code, POA, at discharge; 100% (304/304) of these were captured by the end-to-end solution.
Primary Hypothesis: SEP-1 Metric Compliance in Historic Controls Versus End-to-End Sepsis Solution Study Period
During the period prior to the study (July 1, 2020– August 31, 2021), the five hospitals’ reported SEP-1 compliance was 62% and the New Jersey, Pennsylvania, and national averages were 65%, 55%, and 57%, respectively. During the “surveillance only” and intervention periods, real-time SEP-1 compliance for presumed bacterial sepsis cases as adjudicated by the nurse navigators was 51.7% and 71.8%, respectively (p = 0.0002); after review by a sepsis expert (D.F.G.) for data points not available real time, the SEP-1 compliance for the “surveillance only” and intervention periods was 68.4%, and 78.3%, respectively (p = 0.002) (Supplemental Table 1, http://links.lww.com/CCX/B61). For individual items, during the “surveillance only” period versus intervention period, after review, the individual item pass rates included: first lactate 93.4% versus 99% (p < 0001); blood cultures before antibiotics, 92.5% versus 98.5% (p < 0001); appropriate antibiotics within 3 hours of time zero, 92.3% versus 97.9% (p < 0001) (Table 2).
Secondary Hypotheses Regarding the End-to-End Sepsis Solution
Confirmation of Sepsis Patients. During the “surveillance only” period, the nurse navigators and the sepsis expert confirmed 258 patients with severe sepsis; during the intervention period, the nurse navigators confirmed 1,332 patients and started the sepsis timer; 1,233 of these patients (92.6%) were confirmed as having a primary diagnosis of sepsis, POA, with a mean of 14 patients confirmed/d. Ninety-nine patients (7.4%) classified as sepsis by the nurse navigators were false positives, yielding a positive predictive value of 93% for a nurse navigator categorizing a patient as primary sepsis, POA, preliminarily eligible for SEP-1 bundle compliance (Supplemental Fig. 1, http://links.lww. com/CCX/B63).
Contacts With Clinical Staff. The nurse navigators made discrete 820 contacts with the ED staff over the intervention period (0.67/patient), a mean of 10 contacts/d (Fig. 2). These contacts included calls placed primarily to clarify whether a patient was septic (13%) and for completion of individual aspects of bundle compliance including first lactate (12%), antibiotic administration (15%), and obtaining a second lactate (15%). In addition, 21% of the calls were to request a telesepsis consultation on consult eligible patients (Supplemental Fig. 2, http://links.lww.com/ CCX/B63).
Consultations. Two-hundred forty-five patients met telesepsis consultation criteria. The nurse navigators identified 204 of these patients (sensitivity 83.3%). The 204 calls to ED staff requesting telesepsis consultation resulted in 71 consults (34.8%) being performed. The average time from identification of a patient’s meeting consult criteria to connection with the telemedicine consult cart was 25 minutes; the average length of a consult was 11 minutes. The 71 telesepsis consultations yielded 308 clinical recommendations, an average of 4.3 recommendations/patient (Supplemental Fig. 3, http://links.lww.com/CCX/B63).
DISCUSSION
In this study, we found that a partially automated endto-end sepsis solution, produced a statistically significant increase in overall SEP-1 metric compliance. For presumed bacterial sepsis patients, overall SEP-1 compliance increased from 68.4% in the “surveillance only” period to 78.3% in the intervention period (p = 0.002). We saw improvements in all SEP-1 reporting metrics except for vasopressor administration from the “surveillance only” to the intervention period. These findings support the ability of a combination of automated monitoring and remote, live nurse navigators to identify SEP-1 eligible patients and interact with clinical staff to increase SEP-1 compliance. Because of the relatively short intervention period in our study and the complexities presented by the second wave of the COVID pandemic, we did not compare differences between the “surveillance only” period and the intervention period for clinical outcomes such as inhospital mortality (IHM), ED LOS, hospital LOS, or discharge location.
Similar to the study by Barbash et al (16), we found that our intervention produced variable changes in the processes of sepsis resuscitation care articulated in SEP1. They conclude that SEP-1 appears to have improved metrics indirectly associated (e.g., first lactate and repeat lactate) more than those directly associated (e.g., early appropriate antibiotics) with improved clinical outcomes. Our findings are different in that our results demonstrated a statistically significant increase in administration of antibiotics, a measure directly associated with improved clinical outcomes (16, 21–23). It is difficult to make further comparisons between the two studies, however, because of significant differences in the study population and metrics of care. For example, the cohort by Barbash et al (16) had first lactate measured within 3 hours in 70.2% of the patients versus 99% in our intervention period and first antibiotics administered within 3 hours in 49.8% of patients versus 98.5% in our intervention period. Further, IHM for all patients in the cohort by Barbash et al (16) was 4.2% in the SEP-1 period versus 15.7% in our cohort (16).
A central issue to optimal sepsis care is early detection. Patients who present to the ED with unstable vital signs are identified early and prioritized for management. Less obviously unstable patients can have multiple AODs yet be assigned a lower triage class and wait extended periods for initiation of management. The automated sepsis screening system used in this pilot was able to identify the vast majority of primary sepsis, POA, patients real-time during triage, waiting room stay, or after placement in an ED room. Often patients alarmed as “Potential Sepsis” and the nurse navigators silenced the patient for 15 minutes waiting for the clinical team to assess the patient, develop a differential diagnosis, and initiate sepsis management. After the training period, the nurse navigators were highly accurate in their classification with a positive predictive value of 93%.
Despite conflicting data on whether improved SEP-1 compliance correlates with improved outcomes, increased attention and review of complex patients makes intuitive sense and, similar to Mohr et al (24), we demonstrated that this can be done remotely, augmented by teleconsultation. Further, in the cross-checking to reduce adverse events resulting from medical errors in the emergency department study, systematic cross-checking of care plans between emergency physicians reduced medical errors with the greatest reduction being for errors in sepsis care, including time to antibiotic administration, fluid therapy, identification of source of infection, and choice of antibiotics (25). These are exactly the same items that are reviewed during telesepsis consultations, providing the possibility of reviewing complex patients prior to admission.
This study has several limitations. First, it was performed in a health system with above-average SEP-1 compliance and the results of our study may not generalize to other hospitals or health systems with different resources and higher or lower baseline SEP-1 compliance. Second, the 11-week-long intervention period was too short to measure differences in hard outcomes such as IHM or discharge location. Third, our study was conducted during an upswing in COVID cases, which can impact metrics of sepsis care because of crowding, boarding, and ED volume. Fourth, our definitions of SEP-1 reporting qualification and compliance do not include all of the exclusion criteria in the complex SEP-1 reporting rules. Therefore, our compliance numbers may be higher or lower than “real” compliance as adjudicated by our SEP-1 abstractors. However, this real-time approach to probable SEP-1 patients is more realistic and captures patients who should be managed in a prospective, time-sensitive fashion. Fifth, there is debate about the relationship of SEP-1 compliance to Figure 2. Contacts with emergency department (ED) staff. Observational Study Critical Care Explorations www.ccejournal.org 9 outcomes for sepsis care; the clinical significance of improved SEP-1 metrics is not addressed in this study. Sixth, except in the case of contacts requesting telesepsis consultation, we did not document whether an individual contact from the nurse navigators led to the suggested action. Seventh, because the interventions stopped at the end of the study period, we have no way of knowing if the improvements in SEP-1 compliance would have been maintained and do not have data to examine whether they returned to the surveillance only baseline. Finally, the realities of staffing a novel nurse navigator role during a nationwide nursing shortage may pose serious challenges to reproducing the described workflow.
CONCLUSIONS
During an 11-week period, a partially automated endto-end sepsis solution improved SEP-1 compliance in patients with primary sepsis, POA, admitted through the ED.
ACKNOWLEDGMENTS
We would like to acknowledge the nurse navigators hired through JeffTemps specifically for this project. We would also like to acknowledge the participation of the bedside clinical staff and the on call sepsis teleconsultants who participated in this performance improvement initiative. Without them, we would not have been able to complete this project.
REFERENCES
1. Singer M, Deutschman CS, Seymour CW, et al: The third in ternational consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016; 315:801–810
2. Gaieski DF, Edwards JM, Kallan MJ, et al: Benchmarking the incidence and mortality of severe sepsis in the United States. Crit Care Med 2013; 41:1167–1174
3. Kaukonen KM, Bailey M, Suzuki S, et al: Mortality related to severe sepsis and septic shock among critically ill patients in Australia and New Zealand, 2000-2012. JAMA 2014; 311:1308–1316
4. Mikkelsen ME, Miltiades AN, Gaieski DF, et al: Serum lac tate is associated with mortality in severe sepsis inde pendent of organ failure and shock. Crit Care Med 2009; 37:1670–1677
5. Howell MD, Donnino MW, Talmor D, et al: Performance of severity of illness scoring systems in emergency depart ment patients with infection. Acad Emerg Med 2007; 14:709–714
6. Shapiro NI, Howell MD, Talmor D, et al: Implementation and outcomes of the Multiple Urgent Sepsis Therapies (MUST) protocol. Crit Care Med 2006; 34:1025–1032
7. Trzeciak S, Dellinger RP, Chansky ME, et al: Serum lactate as a predictor of mortality in patients with infection. Intensive Care Med 2007; 33:970–977
8. Rivers E, Nguyen B, Havstad S, et al; Early Goal-Directed Therapy Collaborative Group: Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med 2001; 345:1368–1377
9. Yealy DM, Kellum JA, Huang DT, et al; The ProCESS Investigators: A randomized trial of protocol-based care for early septic shock. N Engl J Med 2014;370:1683–1693
10. Peake SL, Delaney A, Bailey M, et al; The ARISE Investigators and the ANZICS Clinical Trials Group: Goal-directed resus citation for patients with early septic shock. N Engl J Med 2014;371:1496–1506
11. Mouncey PR, Osborn TM, Power GS, et al; The ProMISe Trial Investigators: Trial of early, goal-directed resuscitation for septic shock. N Engl J Med 2015; 372:1301–1311
12. Centers for Medicare & Medicaid Services: SEP-1 Bundles. 2022. Available at: https://cmit.cms.gov/CMIT_public/ ViewMeasure?MeasureId=1017. Accessed August 1, 2022
13. Bone RC, Balk RA, Cerra FB, et al: Definitions for sepsis and organ failure and guidelines for the use of innovative thera pies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992; 101:1644–1655
14. Levy MM, Fink MP, Marshall JC, et al; International Sepsis Definitions Conference: 2001 SCCM/ESICM/ACCP/ATS/ SIS international sepsis definitions conference. Intensive Care Med 2003; 29:530–538 Gaieski et al 10 www.ccejournal.org October 2022 • Volume 4 • Number 10
15. Kaukonen KM, Bailey M, Pilcher D, et al: Systemic inflamma tory response syndrome criteria in defining severe sepsis. N Engl J Med 2015; 372:1629–1638
16. Barbash IJ, Davis BS, Yabes JG, et al: Treatment patterns and clinical outcomes after the introduction of the Medicare Sepsis Performance Measure (SEP-1). Ann Intern Med 2021; 174:927–935
17. Baghdadi JD, Brook RH, Uslan DZ, et al: Association of a care bundle for early sepsis management with mortality among patients with hospital-onset or community-onset sepsis. JAMA Intern Med 2020; 180:707–716
18. Rhee C, Filbin MR, Massaro AF, et al; Centers for Disease Control and Prevention (CDC) Prevention Epicenters Program: Compliance with the national SEP-1 quality measure and association with sepsis outcomes: A mul ticenter retrospective cohort study. Crit Care Med 2018; 46:1585–1591
19. Venkatesh AK, Slesinger T, Whittle J, et al: Preliminary perfor mance on the new CMS Sepsis-1 National Quality Measure: Early insights from the Emergency Quality Network (E-QUAL). Ann Emerg Med 2018; 71:10–15.e1
20. Centers for Medicare & Medicaid Services: SEP-1 Bundles. 2022. Available at: https://www.medicare.gov/care-compare/ details/hospital//. Accessed August 1, 2022
21. Kumar A, Roberts D, Wood KE, et al: Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 2006; 34:1589–1596
22. Gaieski DF, Mikkelsen ME, Band RA, et al: Impact of time to antibiotics on survival in patients with severe sepsis or septic shock in whom early goal-directed therapy was initiated in the emergency department. Crit Care Med 2010; 38:1045–1053
23. Seymour CW, Gesten F, Prescott HC, et al: Time to treatment and mortality during mandated emergency care for sepsis. N Engl J Med 2017; 376:2235–2244
24. Mohr NM, Campbell KD, Swanson MB, et al: Provider-to provider telemedicine improves adherence to sepsis bundle care in community emergency departments. J Telemed Telecare 2021; 27:518–526
25. Freund Y, Goulet H, Leblanc J, et al: Effect of systematic physician cross-checking on reducing adverse events in the emergency department: The CHARMED cluster randomized trial. JAMA Intern Med 2018; 178:812–819