Revising the Canadian Standards of Critical Care Nursing Practice 6th Edition: Pt 2: A Modified Delphi Process

Abstract

Background:

The Canadian Association of Critical Care Nurses (CACCN) establishes the national standards of practice to guide safe, competent, and evidence-informed critical care nursing practice, which are reviewed regularly to ensure they remain current and responsive to emerging evidence. Previous revisions lacked a standardized methodology, prompting this study to employ a Modified Delphi process to revise the Standards for Critical Care Nursing Practice (6th edition).

Methods:

An expert panel of Canadian critical care nurses (≥5 years’ experience) was recruited to ensure representation across provinces, care settings, and patient populations. Three Delphi rounds, with a fourth for peer review, were conducted using secure online surveys. The overarching standard of practice statements along with sub-standard statements included were generated from a previously conducted scoping review. Standards and sub-statements were rated on a 5-point Likert scale, with consensus defined a priori as ≥75%.

Results:

Attrition occurred across rounds, but consensus was achieved on eight overarching standards and 42 sub-statements, emphasizing accountability, evidence-informed practice, ethical care, leadership, collaboration, specialized education, and high-quality care across the illness trajectory. Protocol deviations, including stricter inclusion criteria for later rounds, were implemented to maintain rigor.

Conclusion:

This structured, reproducible approach strengthens the validity of the revised standards and provides a framework for future updates. The findings underscore the importance of systematic, collaborative processes in defining national practice standards for critical care nursing in Canada.

Background

The Canadian Association of Critical Care Nurses (CACCN) is the national association representing critical care nurses (CCN) in Canada.  Since 1992, the CACCN has set the standards of practice for Canadian CCNs. Standards of care provide legal guidance for what constitutes “safe and appropriate patient care” (Shapiro, 2019, p.102); nurses “are obligated to provide knowledgeable, competent, and safe care and act in the best interests of their patients” (Shapiro, 2019, p.101). Standards of practice also “delineate the scope, function, and role of the nurse in practice” (Shapiro, 2019, p.102).  The Standards for Critical Care Nursing Practice (CACCN, 2017) are used to guide and to provide a resource for CCNs to ensure the best practices are utilized in critical care units. The Standards provide broad, overarching guidance that is then individualized at the unit level to ensure high quality care is provided to all patients, families and substitute decision makers accessing critical care in Canada.

The most recent Standards were revised in 2017, after undergoing the fifth review since inception. Historically, the Standards were developed utilizing literature reviews and expert nurse input in the past (Kidd et al., 1987). Although previous revisions constituted a robust review of literature and contemporary practice, with the exception of the first published standards, there has not been a defined, standardized methodological process applied to their development, review or revision. Therefore, the aim of this 6^th revision of the Standards, was to use a Modified Delphi process to provide a structured, anonymous collaboration among expert Canadian CCNs. . Given the volunteer nature of the Association whereby there is turnover of the National Board and committees, this approach was devised to provide CACCN National a process to move forward and modify as necessary overtime. The associated protocol was published in the Canadian Journal of Critical Care Nurses (Vanderspank-Wright et al., 2023).  The following describes the Modified Delphi process and outcomes in detail.  Where appropriate, protocol deviations are identified, described and an associated rationale provided.

Design and Methods

Design

A Modified Delphi (Keeney et al., 2011) was conducted and guided by the study protocol (Vanderspank-Wright et al., 2023).  

Sample – Expert Panel Composition

To ensure adequate representation of critical care nursing practice experience and expertise, we anticipated the recruitment of an expert panel of approximately 107 participants. Our original sample size estimation included an effort to ensure maximum variation (e.g., nursing roles); provincial and territorial representation; representation from rural, remote, community and academic centres; as well as adult, pediatric, and neonatal clinical contexts.  We also anticipated attrition during each round.  Further, we aimed to ensure CACCN National Board of Directors members representation in the Delphi sample but excluded the primary authors. All participants who expressed interest were invited to participate. 

Recruiting Expert Panel Members

To recruit expert panel members, CACCN National Office sent an email to current CACCN members (as of November 2022) with a link to express an interest in participating.  The link corresponded to a secure Survey Monkey platform where the following information was requested: 1) confirmation of CACCN membership; 2) confirmation of a minimum of five years of critical care experience; 3) an expression (yes/no) of interest in participating in the revision of the Standards; 4) an email address for contact; 5) creation of a four-digit code that would be used for future survey participation; along with pertinent demographic information (province or territory of current practice; description of hospital setting; patient population; role in critical care; range of years of experience).

For a participant to continue to partake in subsequent rounds they had to have completed the previous round (this would apply to all but Round 1).  We tracked participation based on the 4-digit code created by participants.  This resulted in the following process: expressions of interest were received, from the expressions – an invitation was sent to participate in Round 1.  Those who participated in Round 1 were invited to Round 2.  This same process continued, albeit with attrition, through to Round 4.

Data Collection – Delphi Rounds

Using the online survey platform, Survey Monkey (secured and licensed by the University of Ottawa), we aimed to complete three rounds of consultations with participants with a fourth round being conducted only if necessary.  Each round was 6 weeks in duration and followed Dillman’s (1978) Total Design for Survey Research for survey distribution and reminders.  An email with the electronic survey link was sent to all participants who had completed a round, and reminders were sent in weeks 2 and 4. Other than the original email sent by CACCN for expressions of interest, all further email communications were sent by the study leads (BVW, SC). Attrition was captured for each round. 

All standards of practice statements were rated for the purpose of achieving consensus, using a 5-point Likert scale defined as follows: 1 – Not Applicable, 2 – Not At All Important, 3 – Somewhat Important, 4 – Important and 5 – Essential.  Consensus was established a priori as 75% based on recommendations from Foth et al. (2016). 

In Round 1, we focused on the overarching standards statements and themes that were retrieved from the scoping review of literature conducted (Crowe et al., 2026).  In this process, all overarching statements were extracted from the included manuscripts/standards documents and using a process of aggregation, like statements were combined into a theme (See Figure 2). In Round 1, overarching statements that were ranked at 4 and 5 with a minimum of 75% consensus (Important and Essential) were moved forward to Round 2. Participants were also given the opportunity to identify missing elements that should be included/considered for inclusion in the revised Standards through an open-ended question format.

In Round 2, all sub-standards associated with the overarching statements were included from those categories that were rated 4-5 with a 75% consensus in Round 1.  Each sub-standard was rated. In this round, any substandard with a rate of 5 and a corresponding 75% consensus was moved forward to Round 3.  This represented a protocol deviation and is explained in the corresponding section of this manuscript. 

In Round 3, only sub-standards rated at 5 with a 75% consensus from the previous round were included.  For this round only sub-standards rated at 5 with a consensus of 75% were included in the final list of sub-standards that would constitute the draft Standards.

Ultimately, we completed a fourth round of consultation with expert panel participants.  While this was a protocol deviation, this was done to ensure that the draft of the standards received an element of peer-review.  For this final round we prepared, via Survey Monkey, a final questionnaire including the eight standards and their sub-standards.  The expert panel was asked to determine whether or not the proposed sub-standards were appropriate, inappropriate, or not needed for the overarching statement. The intent of this questioning was not to determine whether the sub-standard would or would not be included because this was determined through the consensus process.  Rather, we wanted to ensure a best-fit of the sub-standard statement to the overarching standard statement it was listed under. If deemed inappropriate or not needed, a corresponding textbox was included where participants were asked to provide an associated rationale for their recommendation. To manage the feedback and maintain a consensus orientated process, only instances where >75% of expert panel members identified a concern with the same statement were addressed.  While the authors have reported this section under methods,– it yielded no change and the eight overarching statements and accompanying sub-standard statements were accepted.

Data Analysis

Data collected in each Delphi round were analyzed using descriptive statistical methods (N, %).  Open ended Reponses from Round 1 were carefully reviewed and cross-referenced with standard and sub-standard statements to ensure they were reflected in the final Delphi round.

Ethical Considerations

Implied consent was obtained from all participants at the beginning of each Delphi Round as part of the electronic survey. A copy of the implied consent text is available on request to the authors or to CACCN National. Participants created a 4-digit identifier that was used as their unique participant code to track participation across the Delphi rounds. A master list of participants and codes has been kept separately from the data and is housed on the uOttawa secure SharePoint server.  Only the principal investigators (BVW and SC) will retain access to the master list.

Data will be kept until the next CACCN Standards revision.  Following publication of the 7^th edition of the Standards, the data will be safely destroyed.

Results

Expert Panel Composition

From the initial expressions of interest in participating, a total of 76 participants accessed the corresponding link and ultimately n = 72 individuals met the inclusion criteria required of the expert panel and n=69 provided complete demographic information.  Figure 1 represents the outcomes of the expressions of interest and Table 1 represents the initial demographics of the sample.

Results from Round 1

An invitation to participate was sent to all those who expressed an interest in participating in the modified Delphi.  Of the 72 emails received, a total of 69 individuals consented to participate in Round 1; however, of the 69 consented, only 42 provided ratings on the overarching standard statements.  Table 2 represents the findings from Round 1.  As evidenced from the data provided, all overarching statements moved forward to Round 2 given ratings of “Very Important” or “Essential” at greater than 75%.  No statement was identified as Not Applicable by participants.  Data from the open-ended response in Round 1 were incorporated into the final peer-reviewed round and are discussed therein.

Table 2

Results from Round 2

As indicated, all overarching statements included in Round 1 met criteria for Round 2 inclusion (≥75% consensus at a rating of 3 or 4).  This resulted in all related sub-standards being incorporated into Round 2 of consensus.  As a result, this round was much more involved for participants.  At the outset, 21 overarching statements and a total of 270 sub-standards required a rating. It is presumed that due to the length of the survey, we saw participant attrition. Initially, there was an N = 32 who completed assessment of statement 1.  At statement 2, there was attrition of five participants (N = 27), with a final N = 26 (attrition of 1 other participant) beginning at statement 3 and through all remaining statements in the survey. The results of Round 2 are available as a supplemental document on request to the authors or CACCN. At the conclusion of the Round only 27 sub-statements did not meet the threshold of ≥75%. Given this finding, a protocol deviation was established for Round 3 (reported under Protocol Deviations).

Results from Round 3

In Round 3, a total of 21 overarching statements and accompanying 243 sub-statements were rated by expert panel participants.  Attrition during this round was as follows: N = 18 participants began the survey, N = 16 participants rated the overarching statements 1 and 2, N = 15 participants continued to rate to the remaining statements.

From Round 3, all statements identified at “Essential” and over 75% consensus were moved forward into a final fourth round and peer-review component.. In total, 8 overarching statements and corresponding sub-statements (N = 41) were assessed. Statements that moved forward are provided in Table 4. In this last consensus round, it is possible that deviations are noted from wording in Table 4 and from the final Standards document.  This is possible for the following reasons: a verbatim statement reflected in a published document moved forward, or a substandard statement was moved to another standard that was more reflective of its meaning.  In this next-to-final version, we also reflected on open-ended feedback received during the first round to assess whether the feedback was captured or could meaningfully be incorporated into a statement already in use. Examples included but are not limited to principles of patient autonomy, staff-wellbeing, equity, diversity and inclusion, and mentorship.

Table 3

Table 4

Protocol Deviations with Associated Rationales

In the recruitment phase of the Delphi, rather than make use of a generic email to receive expressions of interest, we proceeded an email sent via CACCN National with a link to an online questionnaire (Interest in Participate Questionnaire).  All responses were kept confidential. Between Rounds 2 and 3, we made a decision to only include statements rated at 5 (Essential) with a >75% consensus to move forward into the final round of peer-review.  This decision was made due to responses received in Round 2 whereby all statements received a rating of 4 (Very Important) or 5 (Essential) at a >75% consensus.  We maintained this approach through the final peer-reviewed round.  The final, Round 4, was in itself, a protocol deviation.  This was not conceived in the original protocol; however, it had utility.  It provided a final consensus process and helped to ensure that the Standards were acceptable to the expert panel.  These Standards were later presented to the membership of the CACCN at the 2024 Canadian Critical Care Nursing Conference.

Strengths and Limitations

The methods used to revision the CACCN Standards of Critical Care Nursing Practice have resulted in a reproducible approach for subsequent revision.  Using a systematic approach for retrieving published, peer-reviewed international critical care nursing standards in addition to the grey literature included (see Crowe & Vanderspank-Wright for CACCN, Under Review), has facilitated both national and international perspectives specific to critical care nursing standards of practice offering a broad but also Canadian centric lens.  Limitations include attrition that is associated with the Delphi process.  The latter may have resulted in lack of representation from provinces as well as the territories.  Potential mitigation strategies might include consultation with the National BOD and active chapters to provide suggestions on yielding a diverse and meaningful representation from all provinces and territories, including critical care nursing roles in their respective contexts.  There was no neonatal representation in the expert panel group from the time of expressions of interest and a very limited number of pediatric representatives (N=3 in the expressions of interest). Consultations with this subset of CACCN membership and perhaps their like Associations may be of benefit.  Further, as Artificial Intelligence and technological advancement continues, these tools may be considered for utility in the standards revisions processes.

Conclusion

This manuscript presents the results of a modified Delphi process that was used to revise the CACCN Standards for Critical Care Nursing Practice.  It provides a systematic and reproducible approach for subsequent standards revision for the CACCN.

References

Return to Vol 37, Special Issue Journal