facebookresearch / ParlAI

A framework for training and evaluating AI models on a variety of openly available dialogue datasets.
https://parl.ai
MIT License
10.48k stars 2.09k forks source link

trining on SQuAD produces no model #770

Closed attardi closed 6 years ago

attardi commented 6 years ago

I have been running the example for over one day now, using the suggested command:

python3 -u examples/train_model.py -m drqa -t squad -bs 32 -mf /tmp/model_drqa

and it is producing lots of messages, like:

[ time:75503s parleys:344056 ] {'total': 352, 'train_loss': 115.6}
[ time:75505s parleys:344066 ] {'total': 320, 'train_loss': 115.6}
[ time:75507s parleys:344078 ] {'total': 384, 'train_loss': 115.6}
[ time:75509s parleys:344090 ] {'total': 384, 'train_loss': 115.6}

but only the dictionary is produced in directory /tmp:

> ls /tmp/model_drqa*
/tmp/model_drqa.dict
emilydinan commented 6 years ago

By default, the train loop saves the model only after validation as --save-every-n-secs defaults to -1. You can set this flag if you want to save a checkpoint more often, or you can increase the frequency with which you validate by setting --validation-every-n-secs to something other than -1. (Additionally, if you break out of the train loop at any point, it will save the model.)

alexholdenmiller commented 6 years ago

closing for now, let us know if we can help any more!