search

pykt-team / pykt-toolkit

pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models
https://pykt.org
MIT License
194 stars 53 forks source link

SAINT model has incorrect defaults #166

Closed Xemorr closed 2 months ago

Xemorr commented 5 months ago

According to the SAINT paper, the default dropout rate is 0.1. PyKT has it set to 0.2 by default

lycyhrc commented 5 months ago

According to the SAINT paper, the default dropout rate is 0.1. PyKT has it set to 0.2 by default

Because we need to search for the best superparameter in wandb, PYKT in yaml file (https://github.com/pykt-team/pykt-toolkit/blob/main/examples/seedwandb/saint.yaml) Lines 21-22 set different dropout values for search. Therefore, the dropout value of examples/wandb_saint_train.py can be freely adjusted to suit your needs. @Xemorr

Xemorr commented 5 months ago

Yes, I know it can be freely adjusted but I think the default should be the one in the paper.