Open apurvak opened 2 years ago
Hi, I am trying to reproduce the results directly listed in the paper: https://arxiv.org/pdf/2103.12011.pdf
Specifically, I am trying to reproduce the answer to the question "which element is named for the Greek word for green?"
I used all the provided pre-trained models and all provided pre-trained tables just so I can get rid of any variables. I am using the provided collab notebook as it is: https://colab.research.google.com/github/google-research/tapas/blob/master/notebooks/retrieval_predictions.ipynb#scrollTo=3jiJ5VvrJkih
However, the answer to that question is incorrect from provided model - it is some random values. Generally, I found that quality of results is not good even for basic questions that prior model worked well.
Is there any piece missing here?
Hi, I am trying to reproduce the results directly listed in the paper: https://arxiv.org/pdf/2103.12011.pdf
Specifically, I am trying to reproduce the answer to the question "which element is named for the Greek word for green?"
I used all the provided pre-trained models and all provided pre-trained tables just so I can get rid of any variables. I am using the provided collab notebook as it is: https://colab.research.google.com/github/google-research/tapas/blob/master/notebooks/retrieval_predictions.ipynb#scrollTo=3jiJ5VvrJkih
However, the answer to that question is incorrect from provided model - it is some random values. Generally, I found that quality of results is not good even for basic questions that prior model worked well.
Is there any piece missing here?