Extend t2v-transformers module with dense passage retrieval Dual Encoding

weaviate / t2v-transformers-models

This is the repo for the container that holds the models for the text2vec-transformers module

BSD 3-Clause "New" or "Revised" License

40 stars 27 forks source link

Extend t2v-transformers module with dense passage retrieval Dual Encoding #25

Closed vblagoje closed 2 years ago

vblagoje commented 2 years ago

Introduce DPR encoding support for t2v-transformer-models. For more details about DPR see HF docs

Resolves https://github.com/semi-technologies/weaviate/issues/1828

Status (please check what you already did):

[x] First draft (up for discussions & feedback)
[x] Final code
[x] Added tests

weaviate-git-bot commented 2 years ago

To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge.

beep boop - the SeMI bot 👋🤖

vblagoje commented 2 years ago

I agree with Weviate CLA. @antas-marcin any chance we can try running Travis as is or should I open a PR on the upstream fork?

vblagoje commented 2 years ago

Ok @antas-marcin @etiennedi @laura-ham this should be it - at least from the Python side. I am more confident now that additional unit tests have been added, hooked into CI/CD and passing. What remains now is the Golang side of hooking into these vectorizers in their own containers.