Molly Estes, MD; Puja Gopal, MD; Jeffrey N. Siegelman, MD; John Bailitz, MD; Michael Gottlieb, MD
Over the last several years, there has been increasing interest in transitioning a portion of residency education from traditional, lecture-based format to more learner-centered asynchronous opportunities. These asynchronous learning activities were renamed in 2012 by the Accreditation Council of Graduate Medical Education (ACGME) as Individualized Interactive Instruction (III). The effectiveness and applicability of III in residency education has been proven by multiple studies, and its routine use has been made officially acceptable as per the ACGME. This article provides a review of the current literature on the implementation and utilization of III in emergency medicine residency education. It provides examples of currently implemented and studied III curricula, identifies those III learning modalities which can be considered best practice, and provides suggestions for program directors to consider when choosing how to incorporate III into their residency teaching.
One of the most recent trends in medical education is the transition from traditional didactics (ie, lecture-based classroom teaching) to online learning modules, collectively referred to as asynchronous learning. Over the last several years, asynchronous learning has been shown to be a successful learning style for many learners. For example, Liu and colleagues performed a meta-analysis of what the authors termed “blended learning” (ie, the combination of traditional teaching methods with asynchronous learning) throughout all health professional learners. Their review found that blended learning consistently performed better than no intervention and that it did not perform inferiorly to traditional “non-blended” learning.1 A host of additional data exists, demonstrating that learners prefer smaller learning environments2 and that these methods can address the challenge of teaching physician self-assessment and fostering the practice of lifelong learning.3
It can be seen that there is a broad range of the use of asynchronous learning across the field of medicine. Looking specifically at resident training, a survey of internal medicine program directors revealed that out of the 214 responding programs, 71.5% used III sometimes, somewhat often, or very often.4 Examples of asynchronous learning curricula can be found in nearly every medical area and specialty, from a pediatrics gastroenterology subspecialty rotation,5 microsurgery competencies in plastic surgery,6 radiology residents receiving more real-time feedback on radiographic reads,7 to journal club for general surgery.8 There are examples for the training of fellows9 and faculty.10 There are even examples of all learners, laypeople and medical professionals, participating in a basic life support class11 and for interprofessional learners from all levels of training and fields participating in teamwork training.12
The early 2000s to 2010s saw a unique challenge to residency programs specifically as Free Open Access Medical Education (FOAM) resources saw an exponential rise.13 Anecdotal evidence at that time suggested that residents were utilizing these resources for their own asynchronous education, with or without residency program oversight. Programs were faced with the decision to either begin vetting and incorporating these resources into their curricula or to maintain a more traditional didactic approach. Questions were raised whether time spent in asynchronous learning could even be counted as part of ACGME required didactic time.
In 2008, The Council of Emergency Medicine Residency Directors (CORD) in conjunction with a task force from the Residency Review Committee for Emergency Medicine (RRC-EM) set out to critically evaluate the Accreditation Council of Graduate Medical Education (ACGME) Emergency Medicine (EM) Program Requirements specifically pertaining to educational conferences. One of the suggestions to come from that task force was for residency programs to actively consider incorporating asynchronous learning as an educational tool.3 Not long after the task force’s recommendations, the RRC-EM published criteria allowing up to twenty percent of conference didactic time to be spent in asynchronous learning, which was renamed as Individualized Interactive Instruction (III).14 A subsequent publication from the same group further defined specific requirements of a valid III program (see section on “Cautions of Implementation”).15
Since then, there has been increasing research into how and which aspects of EM residency teaching can be transitioned to III.16 Some programs have applauded it as the way of the future,17-18 while others have advised caution in implementation.19-20 Multiple ideas have been published on how to incorporate III such as flipped classroom,21 journal article discussion boards,22 or a series of varied online learning tasks.23 Comprehensive databases have emerged offering vetted sources, centralized information, and access to experts.24
Surveys have shown extensive utilization of III among residents,25 as well as extensive incorporation into EM training programs.26-27 A survey by Waxman and colleagues in 2014 showed that 63% of programs were incorporating III into residency training; however, they noted there were significant variations in the structure of the curricula. Of the 37% who were not using III, 71% had concerns related to the understanding and implementation of III within the ACGME/RRC-EM criteria.26 The purpose of this article is to provide a review of the current literature on III and best practices recommendations for programs to consider as they refine their already existing III curricula or implement a curriculum for the first time.
APPRAISAL OF THE LITERATURE
This article is the second in a series of best practice reviews from the CORD Best Practices Subcommittee. The first three authors performed a search of PubMed for articles published from inception to March 31, 2018 using the same keywords “asynchronous learning” and “interactive individualized instruction.” Bibliographies of all relevant articles were reviewed for additional studies. Articles were screened by the search authors to evaluate for any articles addressing the specific topics of implementation and utilization of III curricula within the field of emergency medicine.
The search yielded a total of 664 articles, of which 19 articles were deemed to be directly relevant to the field of emergency medicine and for inclusion in this review. When supporting data was not available, recommendations were made based upon the authors’ combined experience and consensus opinion. Prior to submission, the manuscript was reviewed by the entire CORD Best Practices Subcommittee.
CURRENT USES OF III IN EMERGENCY MEDICINE
In 2015, the CORD III Task Force performed an updated survey of program directors of ACGME-accredited emergency medicine residency programs on their current use of III (unpublished data). Of the 77 unique programs which responded (approximately 46% response rate), 74% reported incorporating III into their programs. More four-year format programs utilized III than 3-year programs (91% compared to 67%). Programs implementing III were divided among those who offered either four or five hours of synchronized didactics weekly, or some variation thereof. Of those who reported not using III, the most cited rationale was an unclear definition of what constituted III. Other programs were concerned about compliance or the resources required for implementation. Offerings for III credit were quite diverse. Many programs offered online learning modules, FOAM resources, and board review sessions for III credit. Some utilized simulation, journal club, and attendance at national or regional meetings. This survey shows that although there is a high rate of utilization of III amongst programs, there still remains a wide variation in qualifying activities.
While there is a significant amount of literature on the importance and acceptance of III as a learning tool, there is no standard or consensus method of implementation in emergency medicine. In addition, there is a dearth of information (only the single survey as described above) in the published literature as to how individual EM residency programs specifically implement III and significant variation amongst programs based on qualitative preliminary surveys. Some research even suggests that III may not be an adequate replacement for all of the didactics in a traditional curriculum, specifically for novice learners, concerns namely being their ability to identify specific knowledge gaps and their need to have adequate expert oversight to ensure true knowledge acquisition and retention.20 Several publications in recent years highlight examples of how EM residency programs nationwide have and are utilizing III; some selected examples are discussed below.
Wray and colleagues implemented an III curriculum in 2013 and measured the effect on in-training exam (ITE) scores. There were four modules per month created by faculty and chief residents, each designed to be completed in less than 1 hour. Educational content included journal articles, audio and video lectures, podcasts, links to FOAMed resources, and modules linked to quizzes. Residents were required to complete these modules and their progress was monitored in addition to ITE exam scores. The group found that despite the decrease in traditional conference hours, time now allotted to III, there was no negative impact on resident ITE scores.28
Pensa and colleagues created a digital course for residents in 2014 and surveyed residents to assess satisfaction. The program educational material was curated by faculty from various FOAMed/digital resources and participation was optional. The modules included an assignment page with the content; a discussion page, which was a mandatory component of the module, and allowed for learners to post queries and for faculty members to answer questions; and a multiple choice quiz page for assessment. Thirty three of 48 residents participated in the survey in the first year and appeared to overall find the course useful, although there were significant variations in time spent participating in the course both among residents as well as faculty. The biggest barrier to participation residents identified was lack of time.29
Kornegay and colleagues developed an III curriculum implemented during the 2011-2012 academic year. Faculty members identified gaps in the pre-existing synchronous curriculum and topics better suited for independent learning then developed a web-based platform consisting of curated content and an evaluation component, namely a reflective writing assignment or quiz. Of responding residents, about eighty percent were satisfied, very satisfied or extremely satisfied with the new modality. The group also analyzed conference attendance and ITE scores and found that PGY-1 resident attendance rate significantly improved from the prior year (85% vs 62% mean) though other curricular changes in the program (e.g. small group-based learning, interactive case-based conferences and changes in off-service rotations) may have also enhanced participation. There was no statistically significant difference in mean ITE scores pre- and post-intervention. Faculty reported a time commitment of about 4-8 hours per month, which was comparable to the time spent to prepare 1 hour of instruction for weekly conference pre-intervention.30
Kothari and colleagues designed an III curriculum based on the Academic in Life Emergency Medicine (ALiEM) popular Approved Instructional Resources (AIR) series. The AIR series curates FOAM content from the top 50 open-access EM and critical care blog and podcast sites, provides associated core teaching points and multiple-choice questions for residents, and tracks resident participation to provide residency program directors with resident progress.24 Kothari and colleagues then implemented a second component to their III curriculum, which consisted of two high-impact journal articles selected by faculty on a monthly basis. The group found that introduction of the III did not negatively affect residency educational conference; attendance across all PGY levels was comparable to the year before.31
Other innovative strategies and formats to implement III in EM have been centered upon discrete, focused topic areas within the larger EM curriculum, such as pediatrics,32-33 palliative and end-of-life care,34 and disaster medicine35 to mention a few examples. Commonalities exist amongst these examples, namely facilitators’ deliberate choosing of either a specific asynchronous learning program or a specific topic to be taught using asynchronous learning depending on their program’s needs.
BEST PRACTICE RECOMMENDATIONS:
CAUTIONS OF IMPLEMENTATION
The ACGME policy statement on the use of III within emergency medicine residency education is very strict as to the criteria that must be met in order for an activity to be considered III. Given that up to 20% (one out of every five hours) of previously considered core curriculum time can now be spent as III, there may be a natural inclination amongst programs to begin to cut back on planned traditional educational activities. This is a fallacy, and there exist several ways that implementation of what might be considered III can go wrong (see Table 1).15 Below are listed some common pitfalls encountered when implementing III.
Table 1: ACGME Criteria for III15
|1. The Program Director must monitor resident participation.|
|2. There must be an evaluation component.|
|3. There must be faculty oversight.|
|4. The activity must be monitored for effectiveness.|
Independent reading and use of question banks: The ACGME places particular emphasis on any potential III being a planned activity that is tailored for the individual’s level of learning. Resident-directed reading is not considered to be a planned activity. Additionally, independent use of a question bank is not directed to the individual’s particular needs, even if the astute resident is choosing specific topics to review. Faculty may perhaps choose a specific reading or set of questions to include as a part of III, but these by themselves do not qualify.
Resident attestations of completion: An attestation of completion of an III activity is not considered to be adequate enough to prove resident participation. There must be a separate, tangible source of evaluation. Tracking quiz completion/participation after an online module or required reading would provide ample proof of activity completion just as a sign-in sheet before a simulation does the same.
Audio, video, or podcasts: These learning methods are considered to be passive learning, and use of them alone does not qualify as III. However, they can be combined with other learning modalities, such as a particular question set from an online question bank, to include an active component.
Monitoring for effectiveness: At the time of implementation of the chosen curricula, program directors must have a plan for how they will go about tracking the effectiveness of the III program. This can take many different forms: use of periodic review quizzes, objective clinical performance, test scores on the in-training exam, etc. However, this type of evaluation must be planned over several generations of residents to account for individual class variation and ensure the III program itself is not causing knowledge gaps. Regular check-ins with residents to additionally ensure their continued perspective of the curricula as beneficial are also recommended.
BEST PRACTICE RECOMMENDATIONS:
OPTIONS FOR III ACTIVITIES
Several best practices have emerged from surveying EM program directors implementing III, both with respect to high quality, effective educational programming as well as compliance with RRC regulations.27
Simulation: Simulation activities easily satisfy the requirements of III. They can provide an individual resident the opportunity for self-directed work on a particular area of improvement with direct faculty supervision and immediate feedback. These work best when a resident identifies a particular case, topic, or procedure on which he or she would like to focus.
Online Resources: A wealth of freely accessible material is available for III learning via podcasts, blogs, and online modules. Program directors need to creatively consider how they will allow for the use of such material for III while maintain compliance with RRC regulations. Additionally, care must be taken to appropriately vet all resources to ensure credibility and academic rigor.36-37 Perhaps the most widely adopted single resource is the ALIEM-AIR Series,24 which as of that group’s publication in 2016 had been implemented in 65 programs. This group rigorously selects the highest quality online resources, as judged by EM faculty, provides a quiz for an evaluative component, and allows for online discussion. Individual program directors are able to monitor both the modules as well as their residents’ participation. Other best practices include discussion sessions with a faculty lead about a particular podcast or blog post.
National/Regional Conferences: Attendance at specialty society meetings offers many learning opportunities. To rise to the level of III, and to meet the criteria set forth by the RRC, programs have instituted a number of policies for such activities. Monitoring participation and faculty oversight are key areas of concern, and can be addressed by checking in with faculty who are also attending or presenting at a particular session. Some programs require discussion or written assignments following the session or conference.
Question Banks: Multiple question banks are available online and in print for residents’ use in preparing for standardized tests. While answering questions alone does not meet criteria for III (see “Cautions of Implementation” above), reviewing specific questions missed or themes with a faculty member certainly would be acceptable.
Other Opportunities: Multiple other activities are in use in emergency medicine programs for III including journal clubs, research and teaching activities, oral boards practice, and many others.
BEST PRACTICE RECOMMENDATIONS:
While all attempts have been made to create an inclusive review of the current use of III in emergency medicine residency education, limitations must be acknowledged. In the identification of pertinent articles for inclusion, although multiple search terms were used and bibliographies cross-referenced, it is possible that some articles may not have been identified by the current review. Articles were chosen based on their primary relevance to the field of emergency medicine, not as an expansive review of the history of the use of III or its current use in other medical fields or specialties. The primary limitation to this data analysis is the relative paucity of data available on the direct implementation or utilization of full III curricula within emergency medicine residency programs. Multiple sources have supplied information pertaining to the use of specific topic-based curricula, but few show analysis of a more extensive use of III as might pertain to what can be considered a core curriculum.
This article provides a review of the literature currently available on the implementation and utilization of III in emergency medicine residency education. It can be said conclusively that III has been proven to be an accepted part of modern residency education. Preliminary data suggests III may very well augment resident learning without negatively affecting standardized testing scores or resident participation in other traditional didactics. Care must be given to choose the appropriate learning level of the resident and ensure ACGME compliance with curricular activities. However, despite multiple sources of curricula options, there remains a paucity of information as to the effectiveness of specific III as pertaining to resident knowledge acquisition and retention. More research is needed into this area to further refine what we determine to be gold standard III modalities. Until then, it is the authors’ intention that readers will be more aware of the ACGME guidelines and the III options that exist in order to avoid the potential pitfalls of implementation at their home institutions.