nyu-dl / dl4marco-bert

BSD 3-Clause "New" or "Revised" License
476 stars 87 forks source link

How to do eval only? #46

Closed tangzhy closed 3 years ago

tangzhy commented 3 years ago

Hi, I downloaded your bert base model for MS MARCO, and try to do evaluation only. I changed the output_dir to be decompressed directory of bert base model. But it seems not initialize from the fine-tuned model properly. Any suggestion?

INFO:tensorflow:***** Running evaluation *****
I1214 12:47:57.898369 140091554572096 run_msmarco.py:404] ***** Running evaluation *****
INFO:tensorflow:  Batch size = 8
I1214 12:47:57.898453 140091554572096 run_msmarco.py:405]   Batch size = 8
INFO:tensorflow:Computing metrics...
I1214 12:47:57.898611 140091554572096 run_msmarco.py:418] Computing metrics...
INFO:tensorflow:Could not find trained model in model_dir: /search/odin/Data/marco-passage-ranking/models/BERT_Base_trained_on_MSMARCO/, running initialization to predict.
I1214 12:47:57.898884 140091554572096 estimator.py:615] Could not find trained model in model_dir: /search/odin/Data/marco-passage-ranking/models/BERT_Base_trained_on_MSMARCO/, running initialization to predict.