strongio / keras-bert

A simple technique to integrate BERT from tf hub to keras
258 stars 108 forks source link

INVALID: Not observing the same accuracy reported when reproducing #39

Open Pooventhiran opened 4 years ago

Pooventhiran commented 4 years ago

Hi. I was trying to repro your results before implementing for my research to make sure that I didn't miss anything. But, when the model was built, the number of trainable and non-trainable params were not the same but the total params was tallied. Also, the accuracy over one epoch was reported as around 85.56% in your notebook but I observed only around 50.10%. The predictions with training and with loading the saved model printed True for same predictions anyways.

Could you please help why I am not able to repro your results?

Please find the setup details below: Ubuntu 18.04 running docker image of tensorflow-1.14.0 with NVIDIA GPU.