Hi Luca, I am actually working on Knowledge Tracing for my thesis. I looked at your code to see if I could find a DKT model as baseline.
While trying it I found 2 errors:
1) distraction error in metrics.py in TruePositives function, which extends tensorflow TrueNegatives (instead of truePositives) resulting in wrong ACC and AUC.
2) using "get_target" function before computing loss function and metrics does not support the use of automatic masking.
I know you changed all -1 s to zeros, making it work, but this way the padded values will be considered in your metrics and loss as correct anwered values, modifying real Accuracy and AUC for example. I suggest you to try incorporating "get_target" function in a final custom layer of the model instead, giving as input the target skill to the model too. This way you can use directly a custom layer (remember to define compute_mask function too) to do the reduce_sum over the skill_true array. I have done it and values of AUC, ACC and loss change a lot in the case you have very different length of the sequences as input (due to large padding).
Despite this errors thanks for the code available, I used to understand better how to implement my own DKT as baseline
Bye ;)
Hi Luca, I am actually working on Knowledge Tracing for my thesis. I looked at your code to see if I could find a DKT model as baseline. While trying it I found 2 errors: 1) distraction error in metrics.py in TruePositives function, which extends tensorflow TrueNegatives (instead of truePositives) resulting in wrong ACC and AUC. 2) using "get_target" function before computing loss function and metrics does not support the use of automatic masking. I know you changed all -1 s to zeros, making it work, but this way the padded values will be considered in your metrics and loss as correct anwered values, modifying real Accuracy and AUC for example. I suggest you to try incorporating "get_target" function in a final custom layer of the model instead, giving as input the target skill to the model too. This way you can use directly a custom layer (remember to define compute_mask function too) to do the reduce_sum over the skill_true array. I have done it and values of AUC, ACC and loss change a lot in the case you have very different length of the sequences as input (due to large padding).
Despite this errors thanks for the code available, I used to understand better how to implement my own DKT as baseline Bye ;)