search

pykt-team / pykt-toolkit

pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models
https://pykt.org
MIT License
194 stars 53 forks source link

关于如何复现Question Level和KC Level的疑问 #120

Open DDCY220 opened 11 months ago

DDCY220 commented 11 months ago

感谢pykt为知识追踪领域做出的巨大贡献。 在阅读论文后，关于如何复现Question Level和KC Level仍存有疑问。我运行example中wandb_xxxx_train.py再运行wandb_predict.py后，所示结果包含One-by-One的结果等。请问其他结果中对应的是Question Level还是KC Level的呢？

{'testauc': 0.9188002221943278, 'testacc': 0.8678735316393662, 'window_testauc': 0.9201199409709553, 'window_testacc': 0.868663798838086, 'oriaucconcepts': 0.807274877304899, 'oriauclate_mean': 0.807413805143088, 'oriauclate_vote': 0.8049668720325175, 'oriauclate_all': 0.8058672455891169, 'oriaccconcepts': 0.7851768050790059, 'oriacclate_mean': 0.807144009898327, 'oriacclate_vote': 0.8066383344988972, 'oriacclate_all': 0.8018075205766851, 'windowaucconcepts': 0.8093884777107377, 'windowauclate_mean': 0.8101170671331457, 'windowauclate_vote': 0.8078638216548034, 'windowauclate_all': 0.8085259538001138, 'windowaccconcepts': 0.7864040341303994, 'windowacclate_mean': 0.808367475743734, 'windowacclate_vote': 0.8078753971395257, 'windowacclate_all': 0.8028583348487928}

期待您的回答，感谢

sonyawong commented 11 months ago

感谢pykt为知识追踪领域做出的巨大贡献。 在阅读论文后，关于如何复现Question Level和KC Level仍存有疑问。我运行example中wandb_xxxx_train.py再运行wandb_predict.py后，所示结果包含One-by-One的结果等。请问其他结果中对应的是Question Level还是KC Level的呢？

{'testauc': 0.9188002221943278, 'testacc': 0.8678735316393662, 'window_testauc': 0.9201199409709553, 'window_testacc': 0.868663798838086, 'oriaucconcepts': 0.807274877304899, 'oriauclate_mean': 0.807413805143088, 'oriauclate_vote': 0.8049668720325175, 'oriauclate_all': 0.8058672455891169, 'oriaccconcepts': 0.7851768050790059, 'oriacclate_mean': 0.807144009898327, 'oriacclate_vote': 0.8066383344988972, 'oriacclate_all': 0.8018075205766851, 'windowaucconcepts': 0.8093884777107377, 'windowauclate_mean': 0.8101170671331457, 'windowauclate_vote': 0.8078638216548034, 'windowauclate_all': 0.8085259538001138, 'windowaccconcepts': 0.7864040341303994, 'windowacclate_mean': 0.808367475743734, 'windowacclate_vote': 0.8078753971395257, 'windowacclate_all': 0.8028583348487928}

期待您的回答，感谢

hi, testauc/window_testauc分别对应的one-by-one 非window和window的结果(即KC level), auclate_xx/windowauclate_xx是我们question level的结果

DDCY220 commented 8 months ago

感谢你的回答，我有进一步的问题。 1.oriaucconcepts是否对应了论文中KC Level(ALL-in-One) 的部分呢？ 2.带window的结果对应论文中的哪些部分呢？ 盼复。