Medical Education Team Develops New Instrument for Assessing Students’ Clinical Skills

To address a longstanding training challenge, the University of Miami Miller School of Medicine’s medical education team has developed a new instrument for assessing students’ clinical skills.

Dr. Chandra at podium with Drs. Deshpande and Agarwal applauding
Amar Deshpande, M.D.; Gauri Agarwal, M.D.; and Latha Chandran, M.D., M.P.H., M.B.A.

“Evaluating clinical skills is more challenging than grading classroom or laboratory assignments,” said Latha Chandran, M.D., M.P.H., M.B.A., executive dean for education, chair of the Department of Medical Education, and professor of medical education and pediatrics.

With the launch of the Miller School’s NextGenMD curriculum in 2020, the medical education team began examining opportunities to improve national standards for clinical assessments. The result was a competency-based approach that incorporates the 13 core entrustable professional activities (EPAs) for entering residency, published by the Association of American Medical Colleges in 2014.

“While some of these EPAs are very task-based, like generating a differential diagnosis, others are harder to observe and evaluate, such as contributing to a culture of patient safety,” said Gauri Agarwal, M.D., associate dean for curriculum, associate professor of medicine, and course director for the Senior Capstone Course at the Gordon Center for Simulation and Innovation and the Medical Education Senior Elective.

EPMO Framework for Assessing Medical Students’ Skills

The Miller School’s new assessment form incorporates the EPA framework, as well as the “Professionalism, Reporter, Interpreter, Manager, Educator” measures of student independence, and the Modified Ottawa co-activity appraisal scale, resulting in the EPMO acronym.

“Our goal in developing this framework was to find a better way for faculty members to give medical students feedback on their clinical rotations,” said Matthew Imm, M.D., assistant professor of clinical medicine, director of assessment and entrustment, and director of the practice of medicine clerkship. “The system at that time was challenging for the faculty and didn’t provide students with meaningful feedback for getting to the next level in their training.”

Smiling group holding NextGenMD sign in front of balloons
Members of the Miller School medical education team celebrate the success of NextGenMD.

After a 2018 pilot study of fourth-year students, the Miller School leaders used the new tool for assessing third-year medical students across multiple clerkships from July 2019 to March 2020, prior to the COVID pandemic closures, and reported the results in the journal article.

“This was an ideal time to experiment and develop a new assessment form,” Dr. Imm said.

Assessing the Increasing Independence of Students in Clinical Tasks

Dr. Agarwal said that the framework provides a foundation for assessing students’ independence in performing clinical tasks, rather than relying on subjective ratings — an approach that has been embraced by faculty members.

“Now we can ask ourselves questions like, ‘Do I need to be in the room watching the student or can I completely trust the student to handle the task?’” she said. “This new instrument also allows us to assess the development of clinical skills over time, as a third-year student should be able to handle more tasks in an independent manner than a first-year student.”

Medical student in scrubs performs procedure on the arm of a man as two people observe
A Miller School student gets hands-on clinical experience.

Dr. Imm added that the tool can also be applied nationally. “One of our field’s big struggles is to assess clinical skills,” he said. “Since the publication of our study, we have heard from other schools that are interested in collaborating in the future.  Most importantly, we believe it’s doing a really good job helping us graduate skilled and trusted physicians serving their patients and communities.”

Dr. Imm was the lead author of a study, “EPMO: A Novel Medical Student Assessment Tool That Integrates Entrustable Professional Activities, Prime, and the Modified Ottawa Coactivity Scale,” published recently in the journal Medical Teacher. Co-authors included Drs. Agarwal and Chandran; Chi Zhang, Ph.D., director of program evaluation and assistant professor of professional practice in the Department of Medical Education; Amar R. Deshpande, M.D., professor of medicine, associate dean for medical education and administration, and vice chair of education for the Department of Internal Medicine; and Barry Issenberg, M.D., professor of medicine, Michael S. Gordon Chair of Medical Education, senior associate dean for research in medical education, and director of the Gordon Center for Simulation and Innovation in Medical Education.

Tags: Department of Medical Education, Dr. Amar Deshpande, Dr. Barry Issenberg, Dr. Chi Zhang, Dr. Gauri Agarwal, Dr. Latha Chandran, Dr. Matthew Imm, Gordon Center for Simulation and Innovation, Medical Teacher journal, NextGenMD