29. Accreditation (CanERA)

Print

Author: Tim Dalseg, MD, MMEd, FRCPC

Co-Author: Sarah Taber, HBA, MHA

Co-Author: Joan Binnendyk, M.Ad.Ed.

Objectives

At the end of this chapter you will be able to:

  • outline the purpose and key components of the CanERA accreditation system
  • list accreditation cycle activities, timelines, and decision categories
  • address common areas for improvement and accreditation myths
  • identify resources available to successfully prepare for and complete an accreditation review

Introduction

“What is accreditation and why am I doing it?”

Accreditation is, “the process of formal evaluation of an educational program, institution, or system against defined standards by an external body for the purpose of quality assurance and enhancement.”1  This process allows you as Program Director to really examine the program in detail from every aspect and ensure input from a broad range of stakeholders as you strive to make the program better.


CanRAC Statement on Equity, Diversity and Inclusion

The three CanRAC colleges have collectively embarked on a process to recognize and address issues related to equity, diversity and inclusion through the PGME accreditation standards. The primary objective of this work is ensuring that learning, and ultimately, care environments are inclusive, psychologically and culturally safe, and free from systemic bias. As an important part of these efforts, CanRAC understands that it must examine and help address systemic racism, which creates inequities in health and wellbeing, health care and social outcomes. Although the experiences of Black and Indigenous Peoples in Canada are distinct, both groups have experienced the severe and ongoing effects of colonial practices and of racism. While there are currently no expectations specific to the health of Indigenous, Black, and other vulnerable peoples and groups at the level of the CanERA general standards, some faculties and programs have begun implementing measures to address health inequities and advance cultural safety and anti-racist practice during residency training. For example, all three Colleges are committed to fulfilling the Truth and Reconciliation Commission’s Calls to Action), particularly those that focus on ensuring health care providers nurture and demonstrate cultural safety in medical education and practice. Recognizing that any proposed changes to the CanERA general standards must be driven by Indigenous Peoples and informed by extensive consultation with distinct Indigenous communities, CanRAC will rely on Indigenous partners and those with the historical, social, and cultural expertise to put forward recommendations that will result in meaningful change for Indigenous communities. Furthermore, the Colleges are committed to the integration of an anti-racism lens into medical education, research and clinical care. Our shared commitment to support self-determination will influence the health system and medical education to address the ongoing health inequities and racism faced Black and Indigenous Peoples, as well as other marginalized populations. To address the inequities experienced by all marginalized peoples and groups, this important work will continue to be developed by the CFPC, CMQ and Royal College. The three Colleges will actively explore how accreditation standards and processes can most appropriately and effectively define, evaluate, and implement clear expectations in postgraduate medical education. Ultimately, this will lead to improved therapeutic relationships and a culturally safe, anti-racist approach to medical care.

CanERA

CanERA (Canadian Excellence in Residency Accreditation) is the conjoint system of accreditation for residency education in Canada and includes stakeholders from the Collège des médecins du Québec, the College of Family Physicians of Canada, and the Royal College of Physicians and Surgeons of Canada. It is an outcomes-based accreditation design that is able to accommodate both time-based and outcomes-based educational training programs with the evolution to Competence by Design.

The aim of CanERA is to:

  • ensure the quality of residency education provided across Canada;
  • objectively evaluate residency programs and institutions to ensure compliance with required expectations;
  • facilitate and contribute to the continuous quality improvement of residency programs and institutions;
  • ensure that residency education adequately prepares residents to meet the health care needs of their patient populations upon completion of traini

For Program Directors, although it is a requirement, and can seem like a lot of additional work, accreditation will provide you with confirmation of everything that is going well in your program, as well as a detailed roadmap of things to work on going forward.

The Accreditation Cycle

“How does this work?”

As a Program Director, you are probably wondering how accreditation works. After we unpack it, you’ll see how manageable it is!

The CanERA accreditation process takes place within a continuous, eight-year accreditation cycle, with expectations for ongoing quality improvement to the residency program throughout. Within this cycle, the regular accreditation survey is conducted every eight years for the institution as well as for each of its residency programs. The eight-year process is, however, punctuated with key accreditation activities at select, predictable points across the cycle as illustrated in the figure below. Further descriptions of the accreditation decision categories, and their associated follow-up and timelines can be found in section 5 of this chapter.

During each regular accreditation survey, there are a number of components that contribute important pieces of information to the accreditation review and decision.

  • CanAMS: The Canadian Accreditation Management System is a digital cloud-based application that has been developed to not only support and streamline accreditation activities, but to provide postgraduate offices and residency programs with a tool to support continuous quality improvement. Once populated, CanAMS will contain all program profile instruments and required pieces of evidence that can be easily validated or updated on a regular basis. This will help you keep your program information up-to-date, so it isn’t as much “new” work each time your program comes up for review! Accreditation surveyors will use information from CanAMS to assist with the accreditation review.
  • Data integration: Throughout the eight-year accreditation cycle, continuous program evaluation data will be collected by CanERA. Currently, data integration consists of annual resident and faculty surveys administered and analyzed by the accrediting colleges. This aggregate anonymized information will be returned directly to program directors to help inform a program’s own continuous quality improvement. This is a good thing for programs! By continuously integrating this information into a continuous quality improvement process, programs will improve their ability to maintain compliance with the required accreditation standards and processes at all times, instead of dealing with issues in “boluses” as they prepare for their accreditation cycle. Additional sources of data integration information will be added to this process in the future, in consultation with postgraduate deans, program directors and other stakeholders.
  • Onsite visit: Interviews and document reviews are conducted in order to evaluate the accreditation standards and in some cases, validate information submitted through CanAMS.
  • Previous accreditation outcomes: Results from the most recent accreditation review ensure follow-up of any documented areas for improvement. So be sure to look at your program’s last review!

Accreditation cycle

Image

Fig. 26-1 Accreditation cycle

 

The timeline

“What do I need to do, and when do I need to start?”

Although the CanERA accreditation system is a continuous model of accreditation with expectations of ongoing program evaluation and improvement throughout the cycle, there is some expected work prior to any onsite accreditation review. Detailed preparations for the regular accreditation review begin about two years in advance, with the identification of the dates for the week that the review will take place; this is a collaborative process between the Royal College, CFPC (and CMQ, in Quebec) and the university. Residency programs are then expected to begin actively preparing for the specific tasks associated with the regular accreditation review. The volume of preparation will be reduced for programs that have kept their information up-to-date throughout the accreditation cycle, so it’s a good idea to review things regularly with your Residency Program Committee(RPC)!

Image

Fig. 26-2 CanERA

 

Key tasks and timelines for program directors

12-18 months in advance:

  • Review CanERA general and specialty-specific accreditation standards (see section 4)
  • Complete CanAMS training modules
  • Access CanAMS program profile instrument, review questions, and ensure narrative responses and uploaded documents are current
  • Ensure responses to previously identified areas for improvement (AFIs) are up-to-date and reflective of the most recent QI efforts

6-12 months in advance:

  • Discuss program strengths and areas for improvement with your RPC and your residents to ensure their views are accurately captured in the CanAMS answers
  • Attend the pre-accreditation review workshop delivered by the Royal College and the CFPC (and the CMQ, in Quebec), to learn process details, clarify questions, and receive key templates (e.g., program schedules)

1-4 months in advance:

  • Work with your program administrator to build the program review schedule, using the Royal College’s template, and ensure availability of key program stakeholders, and meeting rooms, as required
  • Submit the program profile instrument and response to previous AFIs through CanAMS, and work with your postgraduate office to finalize the submission

Day of review:

  • Prepare any required documentation for the onsite accreditation review
  • Respond to questions or clarifications requested by the relevant Royal College specialty committee
  • Greet the surveyors the day of the accreditation review

The Accreditation Standards

“What are the accreditation standards?”

The CanERA residency accreditation standards are organized according to an established hierarchical framework: domain, standards, elements, requirements, and indicators. This structure allows the standards to cover both overarching outcomes of residency education quality, and concrete, measurable components for residency programs. During accreditation reviews, residency programs and institutions are evaluated against each requirement, based whether or not the program meets the associated indicators.

Image

Fig. 26-3 The Accreditation Standards

Surveyors will evaluate whether each indicator is met or not met. If all required indicators in a requirement are met, the requirement is also met. If one or more required indicators is ‘not met’, the requirement becomes an Area for Improvement (AFI). Surveyors will recommend whether the AFI must be addressed within two years (AFI-2Y), or whether the AFI can be something the program can work towards over the eight-year cycle, in time for the next regular review (AFI-RR).

There are different CanERA general standards that apply to institutions and residency programs. The General standards of accreditation for institutions with residency programs include requirements applicable to all faculties of medicine, postgraduate offices, and learning sites. Your PGME office will take care of these standards. They ensure residency programs are supported in their delivery of residency education and in their continuous quality improvement efforts. These standards are organized according to three domains: Institutional governance; Learners, teachers, and administrative personnel; and Continuous improvement.

The General standards of accreditation for residency programs include requirements that apply to all residency programs including their respective learning sites. These standards are those for which you, as Program Director, will be responsible.  The purpose is to ensure that Canadian residency programs adhere to a set of high, uniform standards in their design and delivery of residency education. The program standards are organized according to five domains: Program organization; Education program; Resources; Learners, teachers, and administrative personnel; and Continuous improvement.

In addition to the CanERA general standards, each residency program also has accreditation standards specific to the content of your specialty or subspecialty area. The standards of accreditation for each discipline build upon and complement the General Standards of Accreditation for Residency Programs. They provide additional discipline-specific expectations regarding educational experiences and content, assessment, and resources.

Here is an example of one of the general standards, with the CanERA hierarchical framework:

DOMAIN: PROGRAM ORGANIZATION

The Program Organization domain includes standards focused on the structural and functional aspects of the residency program.

STANDARD 1: There is an appropriate organizational structure, with leadership and administrative personnel to support the residency program, teachers, and residents effectively.

Element 1.1: The program director leads the residency program effectively.

Requirement 1.1.1: The program director is available to oversee and advance the residency program.

Indicators

1.1.1.1: The program director has adequate protected time to oversee and advance the residency program consistent with the postgraduate office guidelines and in consideration of the size and complexity of the program.

1.1.1.2: The program director is accessible and responsive to the input, needs, and concerns of residents.

1.1.1.3: The program director is accessible and responsive to the input, needs, and concerns of teachers and members of the residency program committee.

Accreditation decision categories and follow-up

“What does my AC status mean?”

Accreditation StatusDefinitionFollow-up
Accredited ProgramThe residency program has demonstrated acceptable compliance with standards.

In practice, this means that the program may have some AFIs, but they are all AFI-RR.
Next Regular Accreditation Review: Follow-up occurs at the next regular review established in the accreditation cycle, with expectations of ongoing continuous quality improvement throughout the cycle.
Accredited ProgramThere is one (or more) significant area(s) for improvement impacting the overall quality of the program, best evaluated via submission of evidence from the program.

This means that surveyors have identified at least one AFI-2Y that requires attention before the next regular review.
Action Plan Outcomes Report (APOR): Follow-up occurs prior to the next regular accreditation review, aligning with the predictable two-year timeline established in the accreditation cycle.

The process to address the area(s) for improvement is at the discretion of the institution; however, evidence submitted via an APOR, must be sufficient to demonstrate compliance with the associated requirement(s), and be limited to information related to the identified area(s) for improvement.
Accredited ProgramThere is one (or more) significant area(s) for improvement impacting the overall quality of the program, best evaluated by external peer reviewers. Again, this means that surveyors have identified at least one AFI-2Y that requires attention before the next regular review.

Factors that may suggest the need for follow-up by external review may include areas for improvement that are:
● persistent
● best evaluated onsite by a reviewer from outside of the university, and/or
● require a specialist from the discipline.

Concerns with the program’s and/or institution’s continuous quality improvement of the residency program may also warrant an external review. 		External Review: Follow-up occurs prior to the next regular accreditation review, aligning with the predictable two-year timeline established in the accreditation cycle.
Accredited Program on Notice of Intent to Withdraw AccreditationThere are major and/or continuing concerns which call into question the educational environment or integrity of the residency program and its ability to deliver high quality residency education.

In practice, this means there are several, often repeat AFIs that require significant work to address within two years, for the program to continue to be accredited.

Notice may also be awarded when, despite notifications and reminders, the program has failed to complete and submit the required accreditation follow-up by the deadline.		External Review: Follow-up occurs prior to the next regular accreditation review, aligning with the predictable two-year timeline established in the accreditation cycle.
Withdrawal of AccreditationA program with the status of “notice of intent to withdraw accreditation” has failed to demonstrate acceptable compliance with the general and discipline-specific standards of accreditation.

Accreditation may also be withdrawn when, despite notifications and reminders, a program on notice of intent to withdraw accreditation has failed to complete and submit the required accreditation follow-up by the deadline.

Institutions may also voluntarily withdraw a residency program at any time.		Withdrawal of accreditation is effective immediately; unless there are residents in the residency program, in which case the status becomes effective at the end of the current academic year.

An application for new accreditation of a residency program may be submitted following a waiting period of 12 months from the effective date of withdrawal.

Accreditation tips

“I’ve heard some things…”

Common Myths:

MYTH:A program’s accreditation can get taken away the day of the review without a chance to improve.

FACT: A program’s accreditation cannot be taken away on the day of the site visit. Only a program already on Notice of Intent to Withdraw Accreditation may lose accreditation after review by the Residency Accreditation Committee.

MYTH: Accreditation is a process that only takes place every 8 years when the college comes for a site visit.

FACT: A follow-up next regular accreditation review does not imply that a residency program can go eight years without interacting with the accreditation system: standards detail expectations of continuous quality improvement efforts between accreditation reviews; programs may receive data integration information throughout the cycle which may require action; and programs are expected to address any areas for improvement identified at the time of the regular accreditation review. This means that your program will maintain evidence of meeting the standards, and this ongoing CQI will make the next accreditation cycle much less work!

MYTH: If a resident representative is present at my onsite accreditation visit, it means that there has been a problem flagged within my program.

FACT: A resident representative at an accreditation visit does not mean there has been a problem identified within your program. Resident representatives self-select which programs they would like to visit and they are encouraged to participate in the accreditation process for a variety of programs.

MYTH: Accreditation is so labour-intensive, most Program Directors retire immediately afterwards.

FACT: While accreditation can be a lot of work, in many cases it can be a ‘labour of love’, resulting in recognition of great work, as well as helpful suggestions and a plan for improvement going forward. It’s true that some PDs turn over after accreditation, but no more than any other year – the Program Director role is demanding, whether it is accreditation time or not!

Common Areas for Improvement (AFI) and their associated accreditation requirements

Continuous Quality Improvement: process to review and improve the resident program (9.1.1); data availability for identification of strengths; action plans for areas of improvement (9.1.3).

Educational Plan: program competencies and objectives guide educational experiences, providing increasing responsibility (3.2.1, 3.2.4); comprehensive curriculum plan addressing CanMEDS/CanMEDS-FM (3.22)

Program of Assessment: planned, defined, and implemented system of assessment (3.4.1); regular review of resident performance and progression (3.4.2)

Teaching and Learning: safe learning environment with promotion of wellness (5.1.2, 5.1.3);  learning needs guide teaching (3.3.1);  regular assessment of clinical teachers; faculty development  support (7.1.1)

Program Support: protected time and resources for Program Director to oversee and advance the program (1.1.1, 1.1.2);  adequate financial, physical, and technical resources (4.1.4)

Further reading

References

  1. International Health Professions Accreditation Outcomes Consortium (IHPAOC); publication pending 2020, BMC Med Ed.