search

google-research / tapas

End-to-end neural table-text understanding models.
Apache License 2.0
1.15k stars 217 forks source link

Possible Error in TABFACT Colab Notebook Code #79

Closed ankugupt closed 4 years ago

ankugupt commented 4 years ago

Hi, I was going through your colab( this ) notebook on recently released TABFACT classification but for all the examples and tables I could try the results were "REFUTE with 100% accuracy" even when my sentence was correct. I tried looking for any error but couldn't find one. Could you please check for any errors?

Thanks

Edit: Just now changed the model to _tapas_tabfact_inter_masklm_largereset from _tapas_tabfact_inter_masklmmedium in the notebook and now it's now working fine. Please check the tapas_tabfact_inter_masklm_medium checkpoint.

ghost commented 4 years ago

Thanks!

There was a bug in the colab where we didn't use the model with the correct options. That should be fixed now. We also updated the example.

ghost commented 4 years ago

Closing for now, feel free to reopen.