search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
133k stars 26.54k forks source link

Add TAPAS trained on NQ #14494

Open xhluca opened 2 years ago

xhluca commented 2 years ago

The authors of the "Open Domain Question Answering over Tables via Dense Retrieval" papers released the weights for TAPAS retriever trained on NQ here: https://github.com/google-research/tapas/blob/master/DENSE_TABLE_RETRIEVER.md

It would be nice if those weights could be converted to Huggingface model (I would presume it's fairly similar the other finetuned model since they share the same architecture, and I'd be happy to do it myself if there's some scripts I can run)

NielsRogge commented 2 years ago

Hi,

Yes you can use the existing conversion script: https://github.com/huggingface/transformers/blob/master/src/transformers/models/tapas/convert_tapas_original_tf_checkpoint_to_pytorch.py

Note that there was already an attempt for this in #11907.

xhluca commented 2 years ago

Thanks @NielsRogge! I tagged you in #11907 about some differences between the current script and the new retrieval weights.

In the mean time I ran the scripts and uploaded the weights to huggingface. If it turns out bert_1 was something important to add, I'm happy to run an updated script and push the changes to the repo.

Note I also uploaded a Wq.pt and Wt.pt as well so they can be loaded separately (otherwise, I think I would need to modify the underlying code of the TAPAS model; lmk if you know a simpler way)

NielsRogge commented 2 years ago

Hi,

bert_1 is important, yes. The retriever consists of 2 encoders, one to encode the question, one to encode the table. The authors released the retriever checkpoints as "dual encoders", meaning they contain the weights of both encoders.

The encoders each correspond to TapasModel in Huggingface, with an optional additional projection layer on top (see the "down project" column which is yes/no).

The reader on the other hand corresponds to TapasForQuestionAnswering, however it should produce span indexes and logits as output, which is not yet implemented.

xhluca commented 2 years ago

Thank you for expanding on this! It's not clear whether bert_1 corresponds to the table or the question encoder unfortunately. The model I uploaded only uses bert, I'm happy to update the name of the repo to the correct version (question or table encoder) once this is disambiguated.

Moreover, I'm not sure about the best way to add an extra layer to a huggingface model, hence I saved it as a supplementary Wq.pt file inside the repo (that can be easily downloaded by requests get)