How to: analyse training output

[hengjwj]

Christopher,

I'm unfamiliar with Python and see that training generates these files for me:

Do you have a script that plots the ROC and PR curves from the respective joblib files?

Also, may I know how the different saved models in the model_states folder can be used? I ran the training with the default values for each parameter. I'm not sure if I understand correctly but I think, the programme saves the best model every 10 epochs. However, I'm not sure how to use these saved models. Could you provide some guidance?

Hope to hear from you soon!

Joel

[chrishendra93]

hi @hengjwj ,

• The script to plot the ROC and PR curves is in one of the notebooks (I think notebook 1) from the codeocean capsule linked in the paper, but anyway I only used sklearn ROC curve and PR curve plotting.
• So normally you would want to use a model that has the best validation loss / accuracy/ insert any metrics you are interested to track here. Those metrics are recorded in val_results.joblib and after you choose your metrics, you can choose the "best" model from epoch that achieves the highest or lowest value in the metrics that you want to use. The checkpointing happens every 10 epochs in the default setting but you can always set this to 1 if you want.