search

pykt-team / pykt-toolkit

pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models
https://pykt.org
MIT License
194 stars 53 forks source link

How to evaluate with question level? #94

Closed venzino-han closed 1 year ago

venzino-han commented 1 year ago

In the pykt paper, it seems to evaluate model with KC level.

How can we evalutate model in question level?

It is different with early fusion because, in the peper, DKT/GKT didn't evaluate with early fusion setting.

Do you have any plan to add question level evaluation metrics?

meta-tabchen commented 1 year ago

Hi, pyKT already has question-level evaluation metrics you can see our doc here.

In our paper, we use the question level to evaluate different models (section "3.3.1 Real-world Prediction Scenarios"), the KC level result just to show the label leakage problems (Table 2).

venzino-han commented 1 year ago

@meta-tabchen Thanks for your kind reply! Forgive my misunderstanding.