Medical resident’s pursuing specialty and differences in clinical proficiency among medical residents in Japan: a nationwide cross-sectional study

Abstract

Importance

Standardized examinations assess both learners and training programs within the medical training system in Japan. However, it is unknown if there is an association between clinical proficiency as assessed by the General Medicine In-Training Examination (GM-ITE) and pursuing specialty.

Objective

To determine the relative achievement of fundamental skills as assessed by the standardized GM-ITE based on pursuing career specialty among residents in the Japanese training system.

Design

Nationwide cross-sectional study.

Setting

Medical residents in Japan who attempted the GM-ITE in their first or second year were surveyed.

Participants

A total of 4,363 postgraduate years 1 and 2 residents who completed the GM-ITE were surveyed between January 18 and March 31, 2021.

Main measures

GM-ITE total scores and individual scores in each of four domains assessing clinical knowledge: 1) medical interview and professionalism, 2) symptomatology and clinical reasoning, 3) physical examination and treatment, and 4) detailed disease knowledge.

Results

When compared to the most pursued specialty, internal medicine, only those residents who chose general medicine achieved higher GM-ITE scores (coefficient 1.38, 95% CI 0.08 to 2.68, p = 0.038). Conversely, the nine specialties and “Other/Not decided” groups scored significantly lower. Higher scores were noted among residents entering general medicine, emergency medicine, and internal medicine and among those who trained in community hospitals with higher numbers of beds, were more advanced in their training, spent more time working and studying, and cared for a moderate but not an extreme number of patients at a time.

Conclusions

Levels of basic skill achievement differed depending on respective chosen future specialties among residents in Japan. Scores were higher among those pursuing careers in general medical fields and lower among those pursuing highly specialized careers. Residents in training programs devoid of specialty-specific competition may not possess the same motivations as those in competitive systems.

Background

The quality of graduate medical education (GME) training received by resident physicians is challenging to assess [1,2,3]. It depends on a myriad of factors, including the number and type of clinical experiences, patient population, and the country, geographic region, and hospital site of the training program [3,4,5,6]. It also depends on individual residents’ intrinsic motivations, such as time spent acquiring medical knowledge and future career pathway [7]. Standardized examinations assessing knowledge, skills, and attitudes are one of the methods evaluating the GME training programs’ effectiveness, including their ability to equip residents with key clinical skills to provide safe, effective patient care [2, 8]. For example, the United States Medical Licensing Examination (USMLE) has historically been used to assess medical student knowledge and, in some GME programs, determine candidacy for competitive residency positions within different specialties. Indeed, one study demonstrated an association between residency specialty match and scores on the USMLE examinations in the USA [3, 9].

Important differences exist between training systems in different countries. In Japan, the current training program has only been in place since 2004 [10, 11]. After completing a six-year medical school, trainees in Japan are referred to as “residents” and spend the next two post-graduate years acquiring essential clinical skills in one of seven unique specialties (internal medicine, surgery, anesthesiology, pediatrics, psychiatry, obstetrics and gynecology, emergency medicine, and community medicine) and pursuing their future specialty [10, 11]. The purpose of the compulsory post clinical training program as stipulated by the Japanese government is to foster the development of a physician's character and to facilitate the acquisition of fundamental medical skills necessary for adequately addressing prevalent injuries and illnesses encountered in general practice. This is undertaken while acknowledging the social role that medicine and medical care ought to play, irrespective of the specific field they may choose to specialize in the future (Article 16–2, Sect. (1) of the Medical Practitioners Act). Thereafter, three to five additional years are spent training within the chosen field. Unlike in the USA, residents in the Japanese training system may choose their career pathway without specific external competition or need for a certain level of achievement on standardized examinations [7, 10, 11].

In addition to defining individual trainee progress, standardized examinations may provide critical feedback to programs that are participating in nascent training systems like the one in Japan, so that they may identify issues within the education system or allow adjustment of educational strategies [6, 12,13,14]. One such examination, the internationally validated General Medicine In-Training Examination (GM-ITE), has been incorporated as an annual assessment into many Japanese training centers [8]. The GM-ITE is the Essential Clinical Skills Assessment Test, aimed to facilitate an objective appraisal of the residents' comprehensive clinical competencies as mandated by the Japanese Ministry of Health, Labour and Welfare (MHLW), and is utilized with the intention of pinpointing focal areas and formulating training programs to garner general clinical skills, as well as for the assessment and enhancement of training programs at each medical institution. The Japan Association for the Advancement of Medical Education (JAMEP), a non-profit organization with experienced physicians and peer reviewers, administers the examination.

Prior studies has shown that GM-ITE scores are higher for residents training in rural settings, [6] those with more rotations in general medicine departments, [4] and those with at least a moderate or higher number of patients at any given time [15,16,17]. Given that residents in Japan may choose their future specialty without external competition (i.e., certain examination score attainment for specific specialties), [10, 18] their intrinsic motivations and perhaps the pursuing specialty may influence their respective levels of knowledge and skill acquisition much more than residents in other countries. However, it is unclear if there exists an association between clinical proficiency (as assessed by the GM-ITE) and their pursuing specialty.

This study’s primary objective was to examine the relative achievement of fundamental skills assessed by the GM-ITE based on pursuing career specialty among residents in the Japanese training system. The secondary objective was to explore hospital and training program characteristics as well as resident factors (e.g., work type and amount, study habits) that may contribute to proficiency in the GM-ITE using the lens of future career specialty.

Methods

Participants

This is a nationwide cross-sectional study of medical residents in Japan. First-year (PGY-1) and second-year (PGY-2) residents took the same GM-ITE exam, in which study participants answered the surveys consecutively immediately following the exam. The study was conducted between January 18, 2021 and March 31, 2021. This study followed the STROBE guidelines. Residents were included if they had just completed the GM-ITE, provided informed consent, and completed the electronic survey (Fig. 1). Residents were excluded if they did not respond to the survey (n = 1,514), did not provide consent (n = 588), indicated more than one preferred specialty (n = 1,106), or did not respond to specific survey questions, including the average number of hours worked (n = 74), the average number of patients (n = 10), emergency department shift amounts (n = 8), and time spent studying (n = 18).

Fig. 1

Flow chart of study participants. Note: First (PGY-1) and second (PGY-2) year residents who had completed the GM-ITE were surveyed between January 18, 2021 and March 31, 2021. The residents were included if they provided informed consent and completed the electronic survey

Main measures

The GM-ITE assesses general clinical knowledge and its application according to the core curriculum of the training programs of the Ministry of Health, Labour and Welfare (MHLW) [19]. It evaluates individual residents, residency programs, and the clinical training system [8]. The GM-ITE comprises 80 multiple-choice questions and is completed by over 50% of all Japanese medical residents every year (sample GM-ITE questions translated into English are shown in Supplement 1) [8]. In line with the Japanese MHLW’s goals for residents, the 2021 GM-ITE consisted of four domains of basic clinical knowledge: 1) medical interview and professionalism, 2) symptomatology and clinical reasoning, 3) physical examination and treatment, and 4) detailed disease knowledge. The examination assesses the most frequent skills in various fields acquired during initial training. Upon completion of the examination, residents receive feedback based on their relative scores and detailed explanations for each question. Our primary outcome was the total score and the scores in each domain described above.

Data collection

Immediately following the GM-ITE examination, resident participants provided consent and completed an electronic survey that assessed their work and educational environment, future pursuing specialty, and sociodemographic details. We also surveyed duration of internal medicine rotation, number of emergency department shifts per month which included both walk-in patient settings and cases where patients were transported by ambulance, average daily number of inpatients for whom they provided care, average resident duty hours worked per week, and average time spent studying per week. Characteristic information on each hospital was obtained from the Resident Electronic Information System website [20] and the Foundation for the Advancement of Medical Training [7].

Statistical analysis

We used the above GM-ITE total score and scores on four domains as the primary outcome (independent variable). All analyses were performed using Stata statistical software (Stata Corp. 2015, Stata 17 Base Reference Manual). Standard descriptive statistics were used to calculate each data set’s number, proportion, mean, median, and interquartile range (IQR). The chi-square, or Fisher’s exact test, was used to compare categorical data. Additionally, multivariate linear regression and multivariate logistic regression analyses were performed to examine factors associated with scores. Department-specific scores were adjusted based on clinical relevance and prior studies [4, 6, 8, 15,16,17,18, 21,22,23,24]. Furthermore, we assessed the multicollinearity of the multivariate model employing Variance Inflation Factor (VIF) to scrutinize the influence of the aforementioned variables on the GM-ITE score. Dunnett's post hoc tests run after a significant one-way analysis of variance (ANOVA), to determine which differences are significant. Finally, a sensitivity analysis was performed, incorporating several factors into a multivariate regression analysis. All tests were two-tailed, and the statistical significance was set at p < 0.05.

Results

A total of 4,363 residents were included in the analysis. Table 1 shows hospital- and resident-level variables as well as GM-ITE score details as they pertain to residents’ pursuing future specialty. Internal medicine garnered the most interest (n = 1,433, 32.8%), followed by surgery (n = 408, 9.4%), orthopedics (n = 305, 7.0%), pediatrics (n = 267, 6.1%), and obstetrics and gynecology (n = 214, 4.9%). Community hospitals accounted for 81.3%, hospitals in rural cities accounted for 67.7%, and hospitals overall contained an average of 559 beds. Among all participants, 69.0% were male, and 58.7% were PGY-2 residents. The largest group of residents (71.0%) had 3–5 shifts per month in the emergency department, and 9.1% had 6 or more. The most common number of patients assigned to a resident was 5–9 (55.7%), followed by 0–4 (29.6%) and 10–14 (9.2%). Residents worked an average of fewer than 59 h per week (40.5%), 60–79 h (35.6%), and more than 80 h (24.0%). Finally, 40.2% studied for less than 30 min per week, 39.7% for 31–60 min, 12.8% for 61–90 min, and 3.6% for 91 min or more.

Table 1 Background factors and residents’ characteristics, among future specialty

Table 2 presents the mean of the total scores and standard deviations of GM-ITE pertaining to each hospital-level, resident-level, and their pursuing future specialty variables. The average score demonstrated an upward trend proportional to the frequency of Emergency Department duties per month, and a similar positive correlation was observed with the average score as the weekly study hours escalated. Nevertheless, no substantial disparities were discernible in terms of gender, Postgraduate Year, or urban residency. Furthermore, Table 2 also shows the correlations between the GM-ITE total scores and the multivariate linear analysis of the future pursuing specialty and individual factors. The VIF estimations for all predictive variables were ascertained to reside within the permissible spectrum in our multivariate regression analysis (mean VIF = 1.42, all variables were less than 10). Residents training in community hospitals scored higher than those in university hospitals (coefficient 2.44, 95% CI 1.08 to 3.81; p < 0.0001), and there was a positive correlation for total scores as the number of beds increased. There was no difference in scores between residents training in urban hospitals and those training in rural hospitals. There was no difference in scores between male and female residents, but PGY-2 residents scored higher than PGY-1 residents (coefficient 0.90, 95% CI 0.49 to 1.32; p < 0.0001). When compared to the referent of zero shifts, there was no significant difference in scores between the number of emergency department (ED) shifts per month. When compared to the referent of 0–4 patients, the average number of patients assigned significantly correlated with higher scores for those assigned 5–9 patients (coefficient 0.80, 95% CI 0.33 to 1.27; p < 0.001) and 10–14 patients (coefficient 0.86, 95% CI 0.04 to 1.68; p = 0.039), but not for 15 or more patients or unknown. When compared to the referent of 59 or fewer hours, the average amount of resident duty hours worked per week significantly correlated with higher scores for those who worked 60–79 h (coefficient 1.18, 95% CI 0.73 to 1.64; p < 0.001) and 80 or more hours (coefficient 0.62, 95% CI 0.10 to 1.15; p = 0.020). When compared to the referent of 0–30 min, those residents who spent more than 30 min per week studying tended to have significantly higher scores. Internal medicine was chosen as the referent since it was the most commonly chosen future specialty. When compared to internal medicine, only those residents who chose general medicine achieved higher scores (coefficient 1.38, 95% CI 0.08 to 2.68, p = 0.038). Conversely, the nine specialties and “Other/Not decided” groups had significantly lower scores. Moreover, both the PGY-1 and the PGY-2 were segregated into their respective categories, with the distinctive characteristics of each being elucidated utilizing the identical methodology as depicted in Tables 1 and 2. The outcomes exhibited a semblance to the comprehensive table, with scores for both General Medicine and Emergency Medicine exhibiting a propensity towards higher values across both academic years. (Supplementary table 1–3.)

Table 2 The mean of the GM-ITE score among variables and the Multivariate Linear Regression Analysis

In addition, General Medicine scores in PGY-1 demonstrated a statistically significant escalation subsequent to adjustment for Internal Medicine as a benchmark. The residents of PGY-1 manifested a significantly superior performance in General Medicine (coefficient 3.51, 95% Confidence Interval 0.04 to 1.68; p = 0.039). Conversely, the disparity between General Medicine and PGY-2 no longer bore statistical significance in the aftermath of the PGY-2 analysis. (Refer to Supplementary table 4 and 5 for additional data). Figure 2 shows a heat map of the GM-ITE total score and scores for its four domains organized by residents’ chosen future specialties. Total scores were higher in general medicine, emergency medicine, and internal medicine, with residents who chose general medicine having higher achievement in basic clinical skills during the study period. Some areas of low performance were also identified among certain future specialties. For example, scores on the medical interview and professionalism tended to be lower in departments involving procedures, such as ophthalmology (6.07/8), orthopedics (6.09/8), surgery (6.15/8), and anesthesia (6.18/8). Scores in symptomatology and clinical reasoning tended to be lower in the clinical laboratory (9.67/18), ophthalmology (9.71/18), and rehabilitation medicine (9.87/18). Similarly, scores for physical examination/treatment were lower among residents whose chosen specialty lacks opportunities to examine the whole body, administer medications, and perform procedures, such as ophthalmology (8.64/18), rehabilitation (8.68/18), and dermatology (8.77/18). Scores for detailed disease knowledge were low in specialties lacking head-to-toe physical examinations, medication treatment, and procedures, such as ophthalmology (16.03/36) and plastic surgery (16.61/36). Next, interesting patterns emerged within individual specialties. In many specialties, all scores aligned (either all or most domain scores were high, as in general medicine, emergency medicine, and internal medicine; or all or most domain scores were low, as in ophthalmology, rehabilitation, dermatology, and orthopedics). However, in other specialties, scores in some domains were higher than in others. For instance, in surgery and anesthesiology, scores for medical interview/professionalism were lower, whereas those for detailed disease knowledge were higher. In urology, symptomatology/clinical reasoning scores were higher, whereas all other domains were lower. Finally, we elucidate the outcomes of Dunnett's test as a post-hoc appraisal to authenticate the divergence in scores corresponding to each medical specialty (Fig. 3). Within this illustration, upon scrutinizing the axis of medical specialties in the leftmost column, it is observed that general medicine, internal medicine, and emergency medicine manifest significantly augmented scores. Conversely, the scores of residents nurturing aspirations to delve into highly specialized domains such as ophthalmology, dermatology, orthopedics, and plastic surgery in the impending future demonstrated a propensity towards lower values in this examination, which gauges essential clinical training proficiency.

Fig. 2

Ranking heatmap of GM-ITE scores by chosen specialty. Note: A heat map of the GM-ITE total scores and scores in the four domains, organized according to the residents' chosen specialties, is presented. The colors ranging from red to blue on the rightmost side serve as indicators of the rank order of the scores, which are classified into 21 levels. The left side provides a detailed breakdown of the scores, with the overall score equating to 80 points (1 point per clinical question). It is further divided into medical interview/professionalism (8 points), symptomatology/clinical reasoning (18 points), physical examination/treatment (18 points), and detailed disease knowledge (36 points). Analysis of the data revealed that the attainment of basic clinical skills was generally higher in the specialties of general medicine, emergency medicine, and internal medicine. However, certain specialties exhibited areas of underperformance

Fig. 3

Univariate comparison of GM-ITE scores among medical resident’s pursuing specialty using ANOVA with Dunnett post hoc test. Note: For the departments delineated in the left-hand column, the disparity between their scores relative to those departments listed in the corresponding columns is indicated. Highlighted cells denote statistical significance, with an orange hue signifying a positive correlation and blue designating a negative one

Discussion

In this nationwide, cross-sectional study, we examined associations among clinical proficiency and skill as assessed by the GM-ITE, training program characteristics, individual resident factors, and pursuing career specialty among resident physicians in Japan. We found that scores on the standardized GM-ITE assessment were higher for residents who planned careers in general medicine, emergency medicine, and internal medicine and lower for residents who planned for highly specialized departments that do not offer general practice. We also discovered distinctive trends based on hospital and resident attributes. GM-ITE scores were higher for those who trained in community hospitals with higher numbers of beds, who were more advanced in their training (i.e., PGY-2), and who spent more time working and studying. Higher scores were noted for residents who cared for a moderate but not extreme number of patients at a time.

In Japan, general medicine is a new specialty certification, established recently in 2018 [23, 25]. General medicine training includes care in the outpatient clinic, inpatient ward, and emergency department settings. Moreover, a resident may thereafter choose a career pathway among family medicine, hospital medicine, or general internal medicine [25]. One potential explanation of our data is that Japanese general medicine physicians cover a wide range of settings and contexts; this breadth of experience facilitates the acquisition of basic clinical skills (knowledge, skills, and attitudes) needed for success on the GM-ITE. This is supported by the fact that previous GM-ITE studies have shown higher scores among those who have completed a general medicine rotation [4, 23, 24]. It is not surprising that those residents with more clinical experience (in the form of time spent in practice and exposure to those clinical conditions most likely to be tested on the GM-ITE) fare better on the examination. In other countries also, postgraduate training examinations have shown that extensive comprehensive training results in higher examination scores [3]. It has been noted that learners in Japan who are passionate about lifelong comprehensive and extensive learning tend to go on to become general medicine physicians [6, 17, 23].

Nearly all physicians in Japan choose their specialty in the second postgraduate year. As a comparative example, trainees in the USA often select a residency program in the specialty of their coice before graduating from medical school. This decision may be based partly on USMLE performance, although the transition to pass/fail assessment for USMLE Step 1 has changed this dynamic. Unlike in Japan, considerable examination competition exists in the USA [7, 9, 26]. Competitive fields such as dermatology, otolaryngology, plastic surgery, and ophthalmology generally require high USMLE Step 2 score scores to be considered [9]. In the USA, annual incomes vary by specialty, with significant differences noted [26]. This pay discrepancy may be an external motivating factor in future career, in that departments with high income may attract residents with good performance in consideration of repayment of large student loans [26]. In Japan, however, there is no competition for nor restrictions on pursuing specific medical specialties. Additionally, physician income in Japan does not vary significantly between departments, further lowering competition compared with the system in the USA [7, 9, 18, 26, 27]. In a large-scale survey in Japan, high remuneration was not a contributing factor in matching training hospitals [18]. Instead, junior career Japanese physicians often decide their career pathways based on their medical interests and curiosity, the scope of their practice, ease of work, expected duties, and alignment of work and abilities [11, 18, 27].

Does the training environment influence residents’ future pursuing specialty, or do they choose the training environment based on their defined future career pathway? We cannot explain any potential or real causal relationships. However, the differences in examination scores according to the chosen future specialty may be due to differences in motivation and the actual amount of active learning during the two years of mandatory rotational training although this was not measured in this study [16, 21]. In addition, university hospitals in Japan tend to focus on experimental research achievement [28, 29] and do not provide the same levels of training in primary and general health care as community hospitals [6, 23]. In our study, a large proportion of residents who seek careers in highly specialized fields like ophthalmology and dermatology are trained at university hospitals, where the number of work shifts, amount of time worked, and amount of clinical experience tend to be smaller. Furthermore, it has been shown that residents who work fewer hours have significantly less time for actual independent study, despite having relatively more time overall, [21] in which case less studying correlates with lower GM-ITE scores [16, 21]. Considering all of the above, it is possible that in Japan, highly specialized departments, such as ophthalmology, dermatology, and plastic surgery, are not expected to have a wide range of clinical skills or knowledge related to systemic conditions. Thus, those who wish to pursue these careers may not have the motivation and study time to devote to skills assessed by the GM-ITE.

Our findings must be interpreted in the context of several limitations. First, this is a cross-sectional study, and it is unknown whether PGY-1 residents accurately predict their actual future specialties. In fact, the commencement of the GM-ITE coincides with the culmination of the academic year, suggesting a greater probability that PGY-2 residents would have delineated their career trajectory by this juncture compared to PGY-1. Furthermore, the GM-ITE scores even before the initiation of the PGY-1 residency are unknown, and the contemplation of future assessments to ascertain this before mandatory clinical training commences is currently under investigation. Second, the presence of selection bias is inevitable. Despite the test being taken by more than a half of residents nationwide, we exclude data from respondents who indicated more than one division from this cohort. Inclusion of such data might engender alterations in the results. Third, the history of the general medicine specialty in Japan is young, and there is mutual overlap among the fields of general internal medicine, hospital medicine, and family medicine, all of which are often also responsible for general internal medicine ward and outpatient services [25]. On the other hand, there may be residents in internal medicine who wish to choose hospital medicine or general internal medicine after their residency program. This overlap can be a major misclassification bias [25, 29, 30]. Subsequently, our GM-ITE data exclusively displays the four categories of examination questions and the total score, as articulated above. This may be because certain questions may appraise the attitude towards physical examination and interview, even when the disease is distinctly categorized under obstetrics and gynecology in the medical department classification. Consequently, it was not feasible to demonstrate a correlation between which medical department applicants are more likely to encounter questions in specific medical department categories. Nonetheless, this is congruent with the MHLW's objective of training 'physicians who can provide comprehensive and wide-ranging medical care. Finally, the results of this examination may serve as either intrinsic or extrinsic motivation for resident aspirants. Even though this examination does not exert a direct influence on career progression or retention, the results are communicated to the program director. Consequently, there exists a potential bias, such as residents who harbor future intentions to apply to a broader array of departments, inclusive of general practice or emergency medicine, may approach the examination with heightened diligence and engage in more rigorous preparation.

Conclusion

In conclusion, this nationwide study is the first of its kind to reveal differing levels of attainment of basic clinical skills, as assessed by the GM-ITE, among residents in Japan according to their chosen specialty. In particular, higher scores were noted among those pursuing careers in general medicine and emergency medicine fields and lower scores among those pursuing highly subspecialized careers. Differences were also observed among future departments with respect to the amount of time spent caring for patients and studying and the number of assigned patients. Residents in medical training programs devoid of specialty-specific competition may not possess the same motivations as competitive systems, and countries may learn from these experiences. Further research is needed to address potential causative factors for these disparate levels of achievement and associations, including perhaps intrinsic motivations to learn within fields outside of the chosen career pathway during the two years of required training. Future studies should also seek to elucidate strategies to improve clinical skills for residents with low achievement levels.