pykt-team / pykt-toolkit

pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models
https://pykt.org
MIT License
194 stars 53 forks source link

Saint model training issue #154

Open zncj2fdx opened 6 months ago

zncj2fdx commented 6 months ago

I modified the Saint model structure in the saint.py file. However, when I call the same command 'python wandb_saint_train.py --dataset_name=assist2009 --seed=3407 --dropout=0.3 --use_wandb=0 --add_uuid=0' before and after the modification, the accuracy and AUC remain exactly the same. Why is that?

zncj2fdx commented 6 months ago

python wandb_ssaint_train.py --dataset_name=assist2009 --model_name=ssaint --seed=3407 --dropout=0.3 --use_wandb=0 --add_uuid=0

Traceback (most recent call last): File "wandb_ssaint_train.py", line 25, in main(params) File "D:\xxx\xxx\xx\pykt-toolkit\examples\wandb_train.py", line 123, in main opt = Adam(model.parameters(), learning_rate) AttributeError: 'NoneType' object has no attribute 'parameters'

sonyawong commented 6 months ago

python wandb_ssaint_train.py --dataset_name=assist2009 --model_name=ssaint --seed=3407 --dropout=0.3 --use_wandb=0 --add_uuid=0

Traceback (most recent call last): File "wandb_ssaint_train.py", line 25, in main(params) File "D:\xxx\xxx\xx\pykt-toolkit\examples\wandb_train.py", line 123, in main opt = Adam(model.parameters(), learning_rate) AttributeError: 'NoneType' object has no attribute 'parameters'

It seems like you change the saint into ssaint, have u insert the ssaint into the pykt/init_model.py for defining a new model?

zncj2fdx commented 6 months ago

python wandb_ssaint_train.py --dataset_name=assist2009 --model_name=ssaint --seed=3407 --dropout=0.3 --use_wandb=0 --add_uuid=0 Traceback (most recent call last): File "wandb_ssaint_train.py", line 25, in main(params) File "D:\xxx\xxx\xx\pykt-toolkit\examples\wandb_train.py", line 123, in main opt = Adam(model.parameters(), learning_rate) AttributeError: 'NoneType' object has no attribute 'parameters'

It seems like you change the saint into ssaint, have u insert the ssaint into the pykt/init_model.py for defining a new model?

elif model_name == "ssaint": model = SSAINT(data_config["num_q"], data_config["num_c"], **model_config, emb_type=emb_type, emb_path=data_config["emb_path"]).to(device) Here is my initialization code, but the problem still occurs.

sonyawong commented 6 months ago

python wandb_ssaint_train.py --dataset_name=assist2009 --model_name=ssaint --seed=3407 --dropout=0.3 --use_wandb=0 --add_uuid=0 Traceback (most recent call last): File "wandb_ssaint_train.py", line 25, in main(params) File "D:\xxx\xxx\xx\pykt-toolkit\examples\wandb_train.py", line 123, in main opt = Adam(model.parameters(), learning_rate) AttributeError: 'NoneType' object has no attribute 'parameters'

It seems like you change the saint into ssaint, have u insert the ssaint into the pykt/init_model.py for defining a new model?

elif model_name == "ssaint": model = SSAINT(data_config["num_q"], data_config["num_c"], **model_config, emb_type=emb_type, emb_path=data_config["emb_path"]).to(device) Here is my initialization code, but the problem still occurs.

How about the print results of model_name (wandb_trian.py line 103) and model (wandb_trian.py line 105) ?

zncj2fdx commented 6 months ago

python wandb_ssaint_train.py --dataset_name=assist2009 --model_name=ssaint --seed=3407 --dropout=0.3 --use_wandb=0 --add_uuid=0 Traceback (most recent call last): File "wandb_ssaint_train.py", line 25, in main(params) File "D:\xxx\xxx\xx\pykt-toolkit\examples\wandb_train.py", line 123, in main opt = Adam(model.parameters(), learning_rate) AttributeError: 'NoneType' object has no attribute 'parameters'

It seems like you change the saint into ssaint, have u insert the ssaint into the pykt/init_model.py for defining a new model?

elif model_name == "ssaint": model = SSAINT(data_config["num_q"], data_config["num_c"], **model_config, emb_type=emb_type, emb_path=data_config["emb_path"]).to(device) Here is my initialization code, but the problem still occurs.

How about the print results of model_name (wandb_trian.py line 103) and model (wandb_trian.py line 105) ?

model_name:ssaint and model is None

sonyawong commented 6 months ago

python wandb_ssaint_train.py --dataset_name=assist2009 --model_name=ssaint --seed=3407 --dropout=0.3 --use_wandb=0 --add_uuid=0 Traceback (most recent call last): File "wandb_ssaint_train.py", line 25, in main(params) File "D:\xxx\xxx\xx\pykt-toolkit\examples\wandb_train.py", line 123, in main opt = Adam(model.parameters(), learning_rate) AttributeError: 'NoneType' object has no attribute 'parameters'

It seems like you change the saint into ssaint, have u insert the ssaint into the pykt/init_model.py for defining a new model?

elif model_name == "ssaint": model = SSAINT(data_config["num_q"], data_config["num_c"], **model_config, emb_type=emb_type, emb_path=data_config["emb_path"]).to(device) Here is my initialization code, but the problem still occurs.

How about the print results of model_name (wandb_trian.py line 103) and model (wandb_trian.py line 105) ?

model_name:ssaint and model is None

The model init was not run successfully. Is any print statement "The wrong model name was used..." when you run the wandb_ssaint_train.py? If it does, there may be some errors exist in your model definition. You can check wandb_train.py to see whether all the variables are stated when calling init_model function.

zncj2fdx commented 6 months ago

python wandb_ssaint_train.py --dataset_name=assist2009 --model_name=ssaint --seed=3407 --dropout=0.3 --use_wandb=0 --add_uuid=0 Traceback (most recent call last): File "wandb_ssaint_train.py", line 25, in main(params) File "D:\xxx\xxx\xx\pykt-toolkit\examples\wandb_train.py", line 123, in main opt = Adam(model.parameters(), learning_rate) AttributeError: 'NoneType' object has no attribute 'parameters'

It seems like you change the saint into ssaint, have u insert the ssaint into the pykt/init_model.py for defining a new model?

elif model_name == "ssaint": model = SSAINT(data_config["num_q"], data_config["num_c"], **model_config, emb_type=emb_type, emb_path=data_config["emb_path"]).to(device) Here is my initialization code, but the problem still occurs.

How about the print results of model_name (wandb_trian.py line 103) and model (wandb_trian.py line 105) ?

model_name:ssaint and model is None

The model init was not run successfully. Is any print statement "The wrong model name was used..." when you run the wandb_ssaint_train.py? If it does, there may be some errors exist in your model definition. You can check wandb_train.py to see whether all the variables are stated when calling init_model function.

I did encounter the error message "The wrong model name was used..." but I haven't been able to pinpoint the exact cause. Therefore, I attempted to directly copy my model into the saint.py file. After running the command "python wandb_saint_train.py --dataset_name=assist2009 --use_wandb=0 --add_uuid=0 --seed=3407," I noticed that the original SAINT model and the model I copied yield identical AUC and ACC values, with no changes whatsoever.