Fine-tune REALM on a different set of documents for retrieval

google-research / language

Shared repository for open-sourced projects from the Google AI Language team.

https://ai.google/research/teams/language/

Apache License 2.0

1.63k stars 348 forks source link

Fine-tune REALM on a different set of documents for retrieval #82

Open mchari opened 4 years ago

mchari commented 4 years ago

Hi, I was wondering if there is a way to take the pre-trained REALM model checkpoints and fine tune them on another much smaller set of documents ? As a first step, it would suffice to just fine tune the ICT retriever.

Thanks!

detaos commented 3 years ago

I'm wondering the same thing, but for a slightly different use-case. I'm wondering how to add/update/remove documents over time without pretraining from scratch each time.