|Ahead of print publication
Introduction of mini-Clinical Evaluation Exercise as a mode of assessment for postgraduate students in medicine for examination of sacroiliac joints
Anuj Singhal1, Shankar Subramanian1, Sonal Singh1, Arun Kumar Yadav2, Anup Hallapanavar1, B Anjali1
1 Department of Internal Medicine, AFMC, Pune, Maharashtra, India
2 Department of Preventive and Social Medicine, AFMC, Pune, Maharashtra, India
Department of Internal Medicine, Armed Forces Medical College, Solapur Road, Pune - 411 040, Maharashtra
Source of Support: None, Conflict of Interest: None
Background: Formative and subjective assessment of medical education is the need of the hour today. Mini-Clinical Evaluation Exercise (mini-CEX) and Directly Observed Procedural Skills (DOPS) are the commonly used workplace-based assessment (WPBA) tools which have been extensively studied in both undergraduate and postgraduate (PG) settings. In this study, we tried to assess the feasibility of mini-CEX and also its acceptability among PG students of internal medicine and teaching faculty for the examination of sacroiliac joints.
Methods: This is an interventional study which was carried out in a government medical college in Pune, Maharashtra, India, from January to June 2019. A total of 18 1st- and 2nd-year PG students and 9 teaching faculty from the department of internal medicine participated in this study. Each student underwent five mini-CEX evaluations over a period of 6 months under different teaching faculty. Feedback was taken from both the teaching faculty and students regarding the feasibility of mini-CEX as an assessment tool.
Results: A total of ninety mini-CEX exposures involving 18 PG students and nine faculty were analyzed. We found a statistically significant improvement in the domain of medical interviewing (P < 0.001), physical examination (P = 0.003), professionalism (P = 0.001), clinical judgment (P = 0.003), counseling skills (P < 0.001), and organizing efficiency (P < 0.001). Overall clinical competence improved from a scale of 5 (1.7) to 6.7 (0.8).
Conclusion: The results of this study show that overall mini-CEX is an acceptable and effective assessment tool. However, regular training of assessors through workshops on the provision of effective feedback is required. Modification of the assessment form based on the feedbacks provided by teachers and students will further facilitate the implementation of this teaching tool in the curriculum.
Keywords: Medical education, mini-Clinical Evaluation Exercise, qualitative study, sacroiliitis, workplace-based assessment
|How to cite this URL:|
Singhal A, Subramanian S, Singh S, Yadav AK, Hallapanavar A, Anjali B. Introduction of mini-Clinical Evaluation Exercise as a mode of assessment for postgraduate students in medicine for examination of sacroiliac joints. Indian J Rheumatol [Epub ahead of print] [cited 2020 Feb 22]. Available from: http://www.indianjrheumatol.com/preprintarticle.asp?id=277420
| Introduction|| |
Formative and subjective assessment of medical education is the need of the hour today. Traditional methods of assessment focused on the cognitive domain of learning and required students to demonstrate their medical competence through written and oral examinations, whereas new assessment tools evaluate students' skill with actual patients in workplace setting. Mini-Clinical Evaluation Exercise (mini-CEX) and Directly Observed Procedural Skills (DOPS) are the commonly used workplace-based assessment (WPBA) tools which have been extensively studied in both undergraduate (UG) and postgraduate (PG) settings.,,, These teaching techniques have also been evaluated in other fields such as midwifery and dentistry., Both the methods involve direct observation of the clinical skills of the student followed by immediate feedback from the teacher. However, in DOPS, the main emphasis lies in procedural skills, whereas the mini-CEX has a more holistic concept and covers several aspects of medical education. WPBA tools such as mini-CEX and DOPS have been found to be reasonably good for assessing clinical competence.
The mini-CEX was designed by the American Board of Internal Medicine in 1995 for PG students keeping into consideration both the clinical skills required by a resident during patient encounters and educational interaction between teaching faculty and residents during rounds. The mini-CEX evaluates residents on the following six core competencies – medical interviewing skills, physical examination skills, professionalism, clinical judgment, organization and efficiency, and counseling skills. The mini-CEX involves multiple exposures by different faculty over a period of time and can be carried out as a part of training curriculum. Medical professionalism is a multidimensional concept that encompasses several aspects regarding attitude, conduct, and professional responsibilities. It is a balance between patient centeredness and medical expertise while taking available resources and societal expectancies into account. Professionalism-Mini-Evaluation Exercise is used as an assessment tool for medical professionalism with good efficacy.
Despite its several advantages and increasing need, the mini-CEX has been sparingly used in India for both UG and PG assessments. There have also been concerns regarding its feasibility due to busy schedules and availability of teaching faculty for multiple assessments.
In this study, we tried to assess the feasibility of mini-CEX and also its acceptability among PG students and teaching faculty of internal medicine for the examination of sacroiliac joints.
Aims and objectives
The main aim of this study is to introduce mini-CEX as an assessment tool among PG students and to assess the feasibility of mini-CEX and its acceptability by the students and teaching faculty for the examination of sacroiliac joints.
| Methods|| |
This was an interventional study which was carried out in a government medical college in Pune, Maharashtra, India, from January 2019 to June 2019. A total of 18 1st and 2nd-year PG students undergoing training in internal medicine participated in this study. Written informed consent was obtained from both the students and the patients. A total of nine teaching faculty from the department of internal medicine volunteered to participate in the study as assessors and verbal consent was taken from them. Ethical clearance was taken from the institutional ethical committee. Both the faculty and the students were explained the nature and the goal of exercise in a small lecture at the beginning of the study. The faculty were also trained to give effective feedbacks balancing both the negative and positive aspects. The training and sensitization of faculty members was done by the first author who is trained in medical education.
The study was carried out in two phases. Both phases lasted for 3 months each. In the first phase, each student underwent five mini-CEX exposures under different assessors over a period of 3 months. These exposures were carried out as part of routine inpatient work and students were given diagnosed X-ray-proven cases of sacroiliitis. The standard mini-CEX form, to assess the students in six domains, namely interviewing skills, physical examination skills, professionalism, clinical judgment, organizing and efficiency, and counseling skills, was used. Modified Schober and FABER tests were used to examine the sacroiliac joint. Each exposure lasted for 25–30 min. Each evaluation form had a column for evaluator's feedback and the same was documented at the end of exercise. Direct feedback was given to the students and the completed mini-CEX forms were deposited in the department for statistical analysis.
During Phase II, feedback was taken from both the students and faculty regarding the effectiveness and feasibility of mini-CEX as an assessment tool. The feedback was based on a 5-point Likert scale with five options such as strongly agree, agree, neutral, disagree, and strongly disagree. The feedback form also had a column for suggestions on the improvement and shortcomings of this assessment tool.
The continuous variables were expressed as means and standard deviations. The qualitative variables were expressed as percentage and numbers. Likert-scale data were checked for normality using Shapiro–Wilk test. However, the data were analyzed using both parametric and nonparametric tests. To find the change over time in one variable, repeated-measurement ANOVA/Friedman ANOVA was used. P < 0.05 was taken as statistically significant. Data were analyzed using Stata Corp. 2013 (Stata Statistical Software, Stata Corp LP, Release 13, College Station, TX, USA).
| Results|| |
A total of ninety mini-CEX exposures involving 18 PG students were analyzed for the examination of sacroiliac joints. We found a statistically significant improvement in the domain of medical interviewing (P < 0.001), physical examination (P = 0.003), professionalism (P = 0.001), clinical judgment (P = 0.003), counseling skills (P < 0.001), and organizing efficiency (P < 0.001). The values are shown in [Table 1]. As opposed to previous studies on mini-CEX which showed the highest ratings for professionalism and lowest ratings for the physical examination, our study did not show any statistically significant difference of one domain over the other (P = 0.12, one-way ANOVA).
|Table 1: Table depicting the P in different domains evaluated under mini-CEX|
Click here to view
Overall clinical competence improved from a scale of 5 (1.7) to 6.7 (0.8), which is statistically significant (P = 0.0005) [Table 1]. We further did posthoc analysis for ceiling effect and found that there was no ceiling effect. The increase in one domain was compared with the corresponding increase in other domains, However, the same was not statistically significant (P = 0.12, one-way ANOVA). The results were same using nonparametric methods also.
The teachers' feedback and students' feedback were also taken after the exercise. All, that is, 18 (100%) students and 9 (100%) teaching faculty, agreed that mini-CEX is an effective learning tool [Figure 1] and [Figure 2].
The other short-term outcomes of this study were to increase the awareness among PG students and the faculty about the utility of mini-CEX as a learning and assessment tool in the examination of sacroiliac joints. More than 75% of the PG students and 80% of the faculty rated satisfaction levels as more than 3 on a 5-point Likert scale in this study. There was an overall improvement in the knowledge and confidence levels of PG students in the examination of sacroiliac joints. More than 90% of the students were satisfied with the use of mini-CEX for learning clinical skills.
A few suggestions and comments given by the teachers were – “Mini-CEX should be inculcated at UG level so that it becomes a norm in clinical skills assessment across all specialties.” Another student commented, “Immediate feedback given to the students was the highlight of the exercise.”
| Discussion|| |
In our study, we were able to show that mini-CEX is an effective tool for the assessment of students under various domains. This exercise provided an opportunity for the students to interact with the faculty, get a direct and focused feedback immediately after the exercise, and discuss future improvement strategies with the teacher. The overall clinical competence also improved, which is in concordance with other studies.
Students should be given an opportunity to self-reflect on their performance after an effective feedback. The social and educational environment of a teaching institute has a profound influence on the professionalism of students. The Mini-CEX is an effective tool to assess for professionalism as the students interact with the patients. Professionalism should be cultivated, promoted, and made a quintessential part of our current teaching program to foster both behavioral and cognitive changes.,
Our study demonstrated that mini-CEX is an effective tool to evaluate professional development of the students. As the number of exposures increased, students' performance also gradually improved over a period of time. This improvement was mainly attributed to the direct feedback provided to the students. Providing an effective feedback to the student is imperative for the success of mini-CEX as an effective assessment tool and depends on various factors such as teachers' perception of students' skill and potential, their own clinical competencies, interpersonal relationship with the students, stress levels, and contextual factors. Balancing positive and negative feedback is an art which can be developed during faculty development programs.
During these workshops, faculty should be evaluated on the basis of statistical models such as generalizability analysis or variance component analysis to assess their ability as assessors and also how much variance in scores is due to assessors.
Few shortcomings of the mini-CEX were also noticed. Although students appreciated the one-on-one interaction with the teachers, they also felt that such interactions also brought on anxiety due to the pressure of performance. This problem can be mitigated by implementing an effective faculty development program and making mini-CEX an assessment exercise, which is carried out regularly as a part of training curriculum. Whereas, challenges such as patients' schedule and privacy were manageable, multiple assessments were perceived as an additional workload, and managing faculty's time for these assessments was found to be cumbersome.
Our study had a few limitations. First, it was carried out on a small number of residents and only in one specialty, which may not be reflective of the overall effectiveness of the mini-CEX. Multiple such studies with large sample sizes and multiple exposures are required to assess the reliability and validity of mini-CEX in other specialties as an assessment tool. Second, the cases included in this study were of sacroiliitis with moderate difficulty. The spectrum of cases and their difficulty levels need to be broadened. And third, the written feedback may not always be reflective of the actual feedback given on ground.
| Conclusion|| |
Overall, the mini-CEX is an effective assessment tool for WPBA. It is a good tool to enhance the clinical skills and knowledge of PG students in sacroiliac joint examination. This also provides a structured assessment module and enhances improvement in the clinical competency of students. However, regular training of assessors through workshops on the provision of effective feedback is required. Modification of the assessment form based on the feedbacks provided by teachers and students will further facilitate the implementation of this teaching tool in the curriculum.
| Acknowledgment|| |
This project was done under the aegis of Foundation for Advancement of International Medical Education and Research (FAIMER) Christian Medical College, Ludhiana (CMCL) 2019 fellowship program.
Our team wholeheartedly thank Dr. Dinesh Badyal, convener, and his team from FAIMER CMCL fellowship program. We also thank Dr. Shaista Saiyad, faculty, for this project. We also would like to thank:
- Administration, UG Wing, Armed Forces Medical College (AFMC), for unstinted support
- HOD and Staff, Department of Medical Education, AFMC, for all resources and peer validation.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Crossley J, Jolly B. Making sense of work-based assessment: Ask the right questions, in the right way, about the right things, of the right people. Med Educ 2012;46:28-37.
Kogan JR, Holmboe ES, Hauer KE. Tools for direct observation and assessment of clinical skills of medical trainees: A systematic review. JAMA 2009;302:1316-26.
Frank JR, Snell LS, Cate OT, Holmboe ES, Carraccio C, Swing SR, et al
. Competency-based medical education: Theory to practice. Med Teach 2010;32:638-45.
Behere R. Introduction of Mini-CEX in undergraduate dental education in India. Educ Health (Abingdon) 2014;27:262-8.
Sweet LP, Glover P, Mc Phee T. The midwifery mini-CEX – A valuable clinical assessment tool for midwifery education. Nurse Educ Pract 2013;13:147-53.
Norcini JJ, Blank LL, Arnold GK, Kimball HR. The mini-CEX (Clinical Evaluation Exercise): A preliminary investigation. Ann Intern Med 1995;123:795-9.
Norcini JJ, Blank LL, Duffy FD, Fortna GS. The mini-CEX: A method for assessing clinical skills. Ann Intern Med 2003;138:476-81.
Kogan JR, Bellini LM, Shea JA. Feasibility, reliability, and validity of the mini-Clinical Evaluation Exercise (mCEX) in a medicine core clerkship. Acad Med 2003;78:S33-5.
Kogan JR, Conforti LN, Bernabeo EC, Durning SJ, Hauer KE, Holmboe ES. Faculty staff perceptions of feedback to residents after direct observation of clinical skills. Med Educ 2012;46:201-15.
Stern DT, Papadakis M. The developing physician – Becoming a professional. N Engl J Med 2006;355:1794-9.
Archer JC. State of the science in health professional education: Effective feedback. Med Educ 2010;44:101-8.
Cruess R, McIlroy JH, Cruess S, Ginsburg S, Steinert Y. The Professionalism Mini-Evaluation Exercise: A preliminary investigation. Acad Med 2006;81:S74-8.
Fernando N, Cleland J, McKenzie H, Cassar K. Identifying the factors that determine feedback given to undergraduate medical students following formative mini-CEX assessments. Med Educ 2008;42:89-95.
Sidhu RS, Hatala R, Barron S, Broudo M, Pachev G, Page G. Reliability and acceptance of the mini-Clinical Evaluation Exercise as a performance assessment of practicing physicians. Acad Med 2009;84:S113-5.
[Figure 1], [Figure 2]