Open nikhildurgam95 opened 2 years ago
@sgpyc any idea what this is about?
The MLPerf reference BERT model is in fact slightly modified from the Google Research model. As far as I remembered, the math should be the same; input dataset is different.
I need to see whether can skip the step counter in checkpoint loading.
Hello all,
We are curious on what is the MLM accuracy of our eval-set run on the pre-trained model that google-research provided. Specifically, the bert-large-uncased model. However, when trying to execute the
run_pretraining.py
script to evaluate the model, we encounter the following error:It seems that the downloaded google-research model does not have a "global_step" Key and so we're unable to load the model to predict the MLM-accuracy of it.
Script used to evaluate the model :
BERT-Large-Uncased model provided by google research : BERT-Large, Uncased (Whole Word Masking) in their github-repo
Did anyone encounter a similar issue? If there is a solution for this, kindly share.
Thank you.