Closed danyaljj closed 4 years ago
You haven't specified a config file, so it used the default: models/MultiQA_BERTBase.jsonnet (you may want to change this). There was a bug in that config that it sampled 10 example in the eval set, instead of not sampling at all... i've fixed this, and pushed it.
I have trained my model on
SQuAD1-1
and now I am trying to evaluate it with the following command:Here is the output that I can see:
Everything looks good, except that it seems that the evaluation is done only on a small subset. In particular the comment that says:
and
Any thoughts on this?