search

pykt-team / pykt-toolkit

pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models
https://pykt.org
MIT License
231 stars 64 forks source link

Missed file 'get_wandb_new' #105

Open clabra opened 1 year ago

clabra commented 1 year ago

Discussed in https://github.com/pykt-team/pykt-toolkit/discussions/104

Originally posted by **clabra** May 7, 2023 Hi, In the [quick start document](https://github.com/pykt-team/pykt-toolkit/blob/main/docs/source/quick_start.md) it is said in the Tuning Protocol section that: > We use the Bayes search method to find the best hyperparameter, it is expensive to run all the hyperparameter combinations. Hence, you can run the get_wandb_new file to check whether to stop the searching. We default to stop the searching if the number of the tuned hyperparameter combinations in each data fold is larger than 300 and there is no AUC improvement on the testing data in the last 100 rounds (output "end!"). But there is not _get_wandb_new_ file in the git repo. I think _quick_start.md_ might need an update. Thanks
sonyawong commented 1 year ago

Discussed in #104

Originally posted by clabra May 7, 2023 Hi, In the quick start document it is said in the Tuning Protocol section that:

We use the Bayes search method to find the best hyperparameter, it is expensive to run all the hyperparameter combinations. Hence, you can run the get_wandb_new file to check whether to stop the searching. We default to stop the searching if the number of the tuned hyperparameter combinations in each data fold is larger than 300 and there is no AUC improvement on the testing data in the last 100 rounds (output "end!").

But there is any _get_wandbnew file in the git repo.

I think _quickstart.md might need an update.

Thanks

Hi, thank you for your suggestions. Actually, we have simplified the parameter tunning steps, and the relevant instructions will be updated as soon as possible. Thanks!

clabra commented 1 year ago

I look forward to your update. Thanks and congrats for the project.

sonyawong commented 1 year ago

I look forward to your update. Thanks and congrats for the project.

hi, we have revised the parameter tunning steps as seen in the document https://pykt-toolkit.readthedocs.io/en/latest/quick_start.html. If you have any other questions, please feel free to contact us. Thank you again for your interest in our work.