Using Machine Learning to Predict Item Difficulty and Response Time in Medical Tests

[mehrdadyn]

Confirm that this is a metadata correction

• [X] I want to file corrections to make the metadata match the PDF file hosted on the ACL Anthology.

Anthology ID

2024.bea-1.48

Type of Paper Metadata Correction

• [ ] Paper Title
• [X] Paper Abstract
• [ ] Author Name(s)

Correction to Paper Title

No response

Correction to Paper Abstract

Prior knowledge of item characteristics, such as difficulty and response time, without pretesting items can substantially save time and cost in high-standard test development. Using a variety of machine learning (ML) algorithms, the present study explored several (non-)linguistic features (such as Coh-Metrix indices) along with MPNet word embeddings to predict the difficulty and response time of a sample of medical test items. In both prediction tasks, the contribution of embeddings to models already containing other features was found to be extremely limited. Moreover, a comparison of feature importance scores across the two prediction tasks revealed that cohesion-based features were the strongest predictors of difficulty, while the prediction of response time was primarily dependent on length-related features.

Correction to Author Name(s)

No response

[anthology-assist]

@mehrdadyn Please update the anthology id for the paper.