A Theory Driven Supervised Learning Approach for Math Skill Proficiency Prediction Using Assessment and Practice Data

NA (site), NA (city),

Abstract:

From the abstract: "Skill proficiency prediction is an important problem in the personalized learning space. We describe a machine learning approach to skill proficiency prediction that is highly interpretable, efficient, and can be used to incorporate data from assessment, practice and potentially, other sources. We formulated the approach as a theory driven supervised learning problem - specifically, we used concepts from Item Response Theory to construct features that provide model interpretability. Results indicate that the most predictive features are those that are estimates of student ability, skill difficulty and ability-difficulty difference, thus aligning well with Item Response Theory. Model diagnostics and error analysis provided several insights into the model that align with learning theories. We incorporated data from both assessment and practice products in the prediction model. Results indicate that the model performs better when there is both prior assessment and practice student data as opposed to when there is just assessment or practice data. Finally, we proposed an approach that recommends the instructional category that best represents each student’s current level of skill development as determined by the skill proficiency prediction value. The proposed approach provides confidence metrics for the recommendations and optimizes for percentage of students placed in the categories."

The full report is available online.

Publication Date:
04/01/2026