allenai / allennlp

An open-source NLP research library, built on PyTorch.
http://www.allennlp.org
Apache License 2.0
11.74k stars 2.25k forks source link

Incomplete model_state_epoch files #5735

Closed entslscheia closed 1 year ago

entslscheia commented 1 year ago

Hi,

I am using AllenNLP-2.4.0. I trained my model for 11 epochs. When I tried to compare different checkpoints of my model, I found that some checkpoints are missing:

image

You can see only model_state_epoch 0, 1, 9 and 10 are there, while all other epochs are missing. I also observed something similar in my other jobs. I didn't delete any files manually. I used AllenNLP-0.9.0 before and have never run into similar situations. Is this a new feature or a bug? Any response will be greatly appreciated.

github-actions[bot] commented 1 year ago

This issue is being closed due to lack of activity. If you think it still needs to be addressed, please comment on this thread 👇