Record Detail

Ill-structured problems, by definition, have multiple paths to a solution and aremultifaceted making automated assessment and feedback a difficult challenge. Diagnosticreasoning about medical cases meet the criteria of ill-structured problem solving sincethere are multiple solution paths. The goal of this study was to develop an adaptivefeedback mechanism that is capable of identifying and responding to novice physicianmisconceptions by mining the log trace data of user interactions in BioWorld, a computer-based learning environment designed to support medical students in regulating their owndiagnostic reasoning. We applied a series of text pre-processing algorithms to extract thelinguistic features that characterized symptoms identified by 30 medical students solvingthree endocrinology cases: diabetes mellitus (type 1), Pheochromocytoma, and Hyper-thyroidism. A subgroup discovery mining algorithm was applied in two steps. First, ruleswere induced to predict an incorrect diagnosis submitted as the final solution for a case onthe basis of symptoms highlighted by medical students as being pertinent, that were in factnot pertinent. Second, rules were induced to predict a distractor hypothesis (an incorrecthypothesis listed as the most probable) during the differential diagnosis process whilesolving the case. The rule set discovered through the subgroup discovery task was shown topredict both incorrect and distractor hypotheses set by novice physicians while solving thecase. We discuss the implications in terms of developing an adaptive feedback mechanismthat can detect physicians’ misconceptions and errors during problem-solving as a meansto deliver prompts and feedback that promote the acquisition of metacognitive monitoringand control skills.