23. Understanding Competence Committees

Introduction

In Competence by Design (CBD), outcomes of training and processes for assessing performance have been defined for each discipline.^1,2 To evaluate individual learners’ progress in achieving these outcomes, programs must collect a large number of assessment data points from multiple sources.³ These data must then be synthesized and interpreted to inform decisions about learner progress and promotion.

Research suggests that a group decision-making process leads to better judgments.^4,5 Accordingly, CBD introduced the important concept of a competence committee (CC) — a group of educators who review and discuss data integrated from a variety of sources and observations to make informed and transparent decisions about a learner’s performance as well as their progression toward competence. A subcommittee of the residency program committee (RPC), the competence committee has the goal of ensuring that all learners achieve the training outcomes of the discipline.

The competence committee is a key element of a program’s system of assessment, and it is crucial to your work as program director in monitoring the progress of each of your residents. In CBD, promotion decisions are made by the competence committee away from the individual supervisor–learner interactions. By shifting these summative discussions to the competence committee, CBD enables interactions between front-line supervisors and individual residents to focus on formative assessment through coaching and feedback to enhance resident performance.

Through a process of regular systematic review of quantitative and qualitative assessment data, and guided by the national specialty competency framework, competence committees track the progress of each resident and identify patterns of performance that inform judgments of competence. The competence committee makes recommendations related to:

• individual learning plans to address areas for improvement (e.g., modified rotations, specific readings, clinical coaching),
• achievement of entrustable professional activities (EPAs),
• the status of the learner’s progress (e.g., progressing as expected),
• promotion to the next stage of training,
• readiness to challenge the Royal College examination and
• readiness for independent practice.

There is no single way to set up and run a competence committee. Programs and institutions have different contexts and cultures, which means there is no one-size-fits-all approach. As a program director, you will be asked to help lead the process to ensure proper functioning of the committee and its members. You will work closely with the competence committee chair to do this work. The following sections outline some guiding principles and best practices to help you to establish and maintain your competence committee.

The role of the program director in the competence committee

If your program does not yet have a competence committee, you will play a critical role in establishing it. The first task is to select the competence committee chair  (you will be a member of the competence committee but in most cases you should assign the chair position to someone else). It is important to choose your competence committee chair carefully, as this person will hold a leadership position that can bring great value to your program if they do the job well: in addition to overseeing periodic reviews of residents, they can identify faculty development needs pertaining to assessment and can contribute information to ongoing program evaluation. Together with the competence committee chair, you will select committee members, design competence committee processes and workflows and lead faculty development. When the competence committee is up and running, you will be there, as a member of the committee, to advocate for residents and provide context to resident data where appropriate. Given your intimate knowledge and understanding of the workings of the program, you will be able to help to clarify issues related to curriculum and culture and to champion quality improvement initiatives with your CBD lead and competence committee chair.

Terms of reference and processes and procedures

New competence committees should begin their work by developing their terms of reference (TOR) and processes and procedures (P&P), which will outline the who, what, where, when, why and how of the committee. As program director, you will be very involved in these tasks. Developing these important documents will create a shared mental model among team members of the committee’s purpose, practices and decision-making process.^6,7 Clear and detailed TOR and P&P create transparency and serve as a reference point for ongoing development of faculty, residents and the committee itself.⁸ Given that competence committees are a relatively new concept, these documents should be widely shared among all program stakeholders. Residents, in particular, are keen to understand the competence committee’s processes. It’s important to cover the concepts in resident orientation sessions and throughout their training.

Membership

Whether you are recruiting a complete slate of members for your program’s first competence committee or selecting candidates to replace departing members from an existing CC, consider what mix of people will give your competence committee broad expertise.^5,9 You will probably achieve the breadth of expertise you need by selecting a combination of individuals from your training sites who can contribute diverse opinions, skills and experiences to committee decision-making. Faculty with lived experience and expertise in equity, diversity and inclusion (EDI) are crucial to provide this perspective on a competence committee. You may also consider inviting a resident representative, a faculty member from outside your program or a member of the public (e.g. a member of a local Indigenous community or organization that your program interfaces with) to sit on your competence committee to contribute different perspectives. The size of the committee should reflect the number of residents in the program, with a minimum size of 3 members for smaller programs. Committee members are expected to give of their time and energy outside of meetings (e.g., to review resident data). Therefore, the members’ statement of work in the TOR should outline these expectations and state what supports are available including consideration for remuneration, especially for community members.

New members to the committee may feel overwhelmed in their new role. To ensure that your competence committee works effectively, make sure that new members are onboarded. Faculty development initiatives should orient new members to the role and purpose of the competence committee as well as the processes and procedures for data review, meetings and group decision-making.

Meetings of the competence committee

Before the meeting

Scheduling meetings

Your office will play an integral role in implementing the processes of the competence committee. To ensure that residents each have an equal opportunity for timely periodic progress updates, each learner should be reviewed at least biannually. Depending on how the committee functions to meet this goal, meetings may need to be scheduled quarterly, or even monthly if you have a larger program.¹² It is important to consider the timing of meetings relative to the timing of regularly scheduled specific summative assessments (e.g., in-training examinations) and stage promotions to ensure that the necessary data are available to make progress decisions. Reviewing the curriculum map annually can provide an overview of all of the assessment data that are routinely collected and can help you to schedule competence committee meetings at opportune times throughout the academic year.¹³

Setting the agenda

The competence committee meeting agenda should outline which residents are to be reviewed at the meeting and their assigned primary reviewer. A trainee may be selected for competence committee review on the basis of any of the following criteria:

• is it the trainee’s turn for a regularly timed review,
• a concern has been flagged on 1 or more of the trainee’s completed assessments,
• there is a requirement to assess the trainee’s completion of stage requirements and eligibility for promotion or completion of training,
• there is a requirement to determine the trainee’s readiness for the Royal College exam,
• there appears to be a significant delay in the trainee’s progress or academic performance or
• there appears to be a significant acceleration in the trainee’s progress.

When setting the agenda, allow sufficient time to address the most critical items, such as stage promotions and any flagged concerns. Once the agenda items have been determined and a time allotment has been assigned to each one, think about how to order the agenda so that the meeting has a logical flow from one item to the next. Consider rotating between reviewers to keep members actively engaged in the discussions. Many competence committees also use their meeting time to engage in program enhancement activities, including member faculty development as well as evaluation and optimization of the competence committee’s processes and practices.

Assigning primary reviews

Each trainee scheduled for review at a competence committee meeting should be assigned to a designated primary reviewer. The primary reviewer is responsible for completing a detailed review of the progress of their assigned trainee(s) on the basis of evidence from completed observations and other assessments and data included in their academic portfolio.^14,15 The primary reviewer considers the trainee’s recent progress relative to the national specialty competency framework and identifies patterns of performance from quantitative and qualitative workplace assessment data as well as any other valid sources of data (e.g., performance on in-training examinations, objective structured clinical examinations).

Collating data

To improve the quality of decision-making at competence committee meetings, all assessment data points need to be collected and collated in a way that facilitates review and synthesis. Given the finite time and resources of the committee members, not all data points can be reviewed during a meeting; ask the committee to decide what data are most valuable to them.

Electronic portfolios and dashboards are useful tools that house data for easy reference during meetings and facilitate visualization of trends over time. Collaborate with your postgraduate medical education office, your program administrator and your faculty and residents to ensure that everyone builds a strong working knowledge of your portfolio’s capabilities.¹⁶ Pay special attention to ensuring that resident assessment data are stored and shared securely in compliance with the standard practices and regulations of your educational institution.

Dashboards have other uses beyond providing benchmark data and options for data visualization that can inform resident progress decisions. They can also enable you and your competence committee to identify gaps in training experiences and facilitate continuous quality improvement for all teaching activities through periodic review of your program’s assessment data.

During the meeting

Facilitating the meeting

The competence committee will be responsible for the logistics of the meeting and facilitating the discussion to ensure that all items on the agenda are addressed. This is an important role, and it requires the chair to have strong facilitation skills to ensure that all voices are heard. The chair should urge members to use a structured approach, outlined in the P&P document, as they share information and should ensure that all members are actively contributing to the discussion.¹⁰

During the meeting, the chair should closely monitor and address any group dynamics that may hinder teamwork or bias decisions. Some bias may be related to process and team functioning, such as dominant voices, groupthink, or anchoring bias.¹¹ However, also it is important to also recognize forms of discrimination against residents. Implicit in discussions on competence are ‘code words’ that can signal bias against a resident’s performance – e.g. an IMG may be scrutinized differently than a CMG due to unfounded assumptions about their previous training. Being able to recognize these patterns in assessment and during discussion is crucial to equitably evaluate a resident, and faculty with EDI expertise on a competence committee may help with this challenge.

Making decisions

A hallmark of high-performing organizations is their ability to make quality decisions through structured group processes.¹⁷ The most important step in the competence committee decision-making process is to clearly define roles and responsibilities.¹⁷ Committees need to identify how information will be presented, who will make the initial recommendation, how input will be sought from other members, how a final decision will be made and who will be accountable for follow-through. The functioning and practices of the competence committee, including its decision-making processes, will be a specific focus of accreditation reviews in the future.

There are a number of approaches the competence committee might use to clarify its decision-making process. Here is an example of a clearly articulated process:

• Each trainee is considered in turn, with the primary reviewer presenting their synthesis of the data, displaying relevant reports from the portfolio and sharing exemplar quotes from any observational comments about the trainee.
• The primary reviewer concludes by proposing a status for the trainee going forward in the program.
• If the recommendation proposed by the primary reviewer is seconded by another committee member, all members are invited to discuss the motion.
• During the discussion and where appropriate, the program director may share information about the resident to provide context to the data. This information may be particularly helpful when the competence committee is developing recommendations regarding the resident’s learning plan.
• Following the discussion, the chair will call a vote on the recommendation proposed by the primary reviewer.
• If the recommendation of the primary reviewer is not seconded or the motion does not achieve a majority of votes, the chair will then request an alternative motion regarding the trainee.
• This will continue until a majority of competence committee members support a status motion.

Documenting decisions

During the meeting, someone will need to capture the key findings and the decisions that result from the discussions. The rationale for any recommendations made by the competence committee must be clearly documented and will be an area of attention in future accreditation reviews. Documenting key decisions and the rationale for these recommendations contributes to transparency and ensures that any outcomes that lead to changes in training experiences or learning progress are defensible.

After the meeting

As soon as possible after the competence committee makes a decision about a resident and the decision is ratified by the RPC, you, the resident’s academic advisor or an appropriate delegate should discuss the recommendation of the competence committee with the trainee. Programs may take slightly different approaches to competence committee follow-up. Whatever approach your program uses, make sure to establish how information will be relayed to residents and within what time frame, and then communicate these details clearly to your residents. Informing the resident of the rationale for the recommendation will facilitate the co-development of a learning plan that is feasible, achievable and more acceptable to them. It is important that the discussion with the resident includes a comprehensive review of all of their assessment data and a mutually developed learning plan for continued growth and acquisition of competencies. Programs need to have an appeals mechanism in place for situations in which a resident does not agree with the decision of the competence committee. This process should conform with your local institutional policies and will involve communication with the postgraduate dean.

Conclusion

Competence committees play a critical role in CBD by supporting individuals’ paths toward competence. They can fulfill their mandate of making transparent and defensible resident progress and promotion decisions by incorporating standardized procedures and being mindful to mitigate potential bias in group decision-making. As program director and a member of the competence committee you will work closely with the competence committee chair to establish these processes, contribute to member faculty development and engage in program enhancement activities.

References

1. Frank JR, Snell LS, ten Cate O, Holmboe ES, Carraccio C, Swing SR, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638–645.
2. Englander R, Frank JR, Carraccio C, Sherbino J, Ross S, Snell S, et al. Toward a shared language for competency-based medical education. Med Teach. 2017;39(6):582–587.
3. Gruppen LD, Ten Cate O, Lingard LA, Teunissen PW, Kogan JR. Enhanced requirements for assessment in a competency-based, time-variable medical education system. Acad Med. 2018;93(3S Competency-Based, Time-Variable Education in the Health Professions):S17–S21.
4. Lockyer J, Carraccio C, Chan M-K, Hart D, Smee S, Touchie C, et al. Core principles of assessment in competency-based medical education. Med Teach. 2017;39(6):609–616.
5. Hauer K, ten Cate O, Boscardin C, Iobst W, Holmboe ES, Chesluk B, et al. Ensuring resident competence: a narrative review of the literature on group decision making to inform the work of clinical competency committees. J Gr Med Educ. 2016;8(2):156–164.
6. Royal College of Physicians and Surgeons of Canada. Competence committees: guidelines for the terms of reference. Ottawa: Royal College of Physicians and Surgeons of Canada; 2017. Available from: https://www.royalcollege.ca/rcsite/documents/cbd/competence-committees-guidelines-for-terms-of-reference-e
7. Royal College of Physicians and Surgeons of Canada. Competence committee: process and procedures in decision making: a framework. Ottawa: Royal College of Physicians and Surgeons of Canada; 2017. Available from: https://www.royalcollege.ca/rcsite/documents/cbd/competence-committees-process-procedures-e
8. Pack R, Lingard L, Watling C, Cristancho S. Beyond summative decision‐making: illuminating the broader roles of competence committees. Med Educ. 2020;54(6):517–527.
9. Chahine S, Cristancho S, Padgett J, Lingard L. How do small groups make decisions? Perspect Med Educ. 2017;6(3):192–198.
10. Kinnear B, Warm EJ, Hauer KE. Twelve tips to maximize the value of a clinical competency committee in postgraduate medical education. Med Teach. 2018;40(11):1110–1115.
11. Dickey CC, Thomas C, Feroze U, Nakshabandi F, Cannon B. Cognitive demands and bias: challenges facing clinical competency committees. J Grad Med Educ. 2017;9(2):162–164.
12. Acker A, Hawksby E. Competence committee: how meeting frequency impacts committee function and learner-centered assessment. Med Teach. June 2020;42(4):472–473.
13. Harden RM. AMEE Guide No. 21: Curriculum mapping: a tool for transparent and authentic teaching and learning. Med Teach. 2001;23(2):123–137.
14. Rich JV, Fostaty Young S, Donnelly C, Hall AK, Dagnone JD, Weersink K, et al. Competency‐based education calls for programmatic assessment: But what does this look like in practice? J Eval Clin Pract. 2020;26(4):1087–1095.
15. Duitsman ME, Fluit CRMG, van Alfen-van der Velden JAEM, de Visser M, Ten Kate-Booij M, Dolmans DHJM, et al. Design and evaluation of a clinical competency committee. Perspect Med Educ. 2019;8(1):1-8.
16. Thoma B, Bandi V, Carey R, et al. Developing a dashboard to meet Competence Committee needs: a design-based research project. Can Med Educ J. 2020;11(1):e16–e34.
17. Rogers P, Blenko M. How clear decision roles enhance organizational performance. In: Harvard Business Review 10 Must Reads on Strategy. Boston: Harvard Business Review Press; 2011:229–250.