Adding retrieving executors

[ierezell]

Describe your proposal/problem

Adding models for retrieving / Question answering like :

• Dense passage retrieval (DPR)
• Retrieval Augmented generation (Rag)
• Fusion in the decoder (Fid)

They are summarized in this article but appear in many others (https://arxiv.org/pdf/2107.07566.pdf).

I Finetuned a DPR model (with haystack), which performs quite well. I would like to use it in Jina. Haystack also has Rag models which are really nice to use.

As there are Transformer based retrievers (cosine of embeddings) in Jina could we also think about adding those models? Else I know there is a convergence of haystack and Jina (https://github.com/jina-ai/jina/issues/2128), could we link them in some way?

Thanks in advance, Have a great day.

[JoanFM]

Hey @Ierezell,

Thank you for your proposal. It seems like a set of interesting models to have.

For the DPR models I see there is a family of models on transformers (https://huggingface.co/transformers/model_doc/dpr.html) that we can base upon. Not sure if ur retrained model can also be easily wrapped in here. As far as I understand this model is similar to any other transofrmer based encoder except that it adds some more fields to the context right?

As for the other models u mention, it seems they are models which are used to generate new responses based upon the retrieved ones. This can easily be added as an executor, but could you point us to some implementations of those that can be wrapped as executors?

Thank you very much

[ierezell]

Hi @JoanFM,

Indeed Huggingface has them and haystack made a wrapper, so it's easier to train/finetune (https://haystack.deepset.ai/docs/latest/tutorial9md). I don't want to steal the credit from them, this is why the idea of a collaboration seems fabulous.

Once the model is trained, it's usable in the haystack stack and guess also on the bare hugging face stack (as they rely on it). It would be sick to do the same in Jina (as a trained DPR model has among the best performance right now).

You're right, but you have two different models (question and documents) and it's among the standard / sota model to use.

For the other models, hugging face also have a RAG model: https://huggingface.co/transformers/model_doc/rag.html And again haystack made a nice wrapper for it: https://haystack.deepset.ai/docs/latest/tutorial7md

I guess jina-ai/jina#2128 is aimed to bring those models into Jina (or Jina to haystack) either way it would be perfect for my use case.

I'm sure it's on your plan to add more executors but models like tapas: https://huggingface.co/transformers/model_doc/tapas.html can deal with tabular data, or even speech to text executor from hugging face, etc... I'm thrilled to see this framework grow and have more and more performant executors.

Have a great day.

[JoanFM]

Hey @Ierezell ,

Thank you for the insights, we will try to add these models to Jina. Even you as community member can contribute executors on your own.

As per DPR models, I understand that is a model that can be called with a context flag determining if it is supposed to encode a question or a document (a source of answer) right?

[ierezell]

Hi @JoanFM,

I will try to add an executor myself and if it leads to good results I will make a PR :)

DPR can be trained on a corpus but you need pairs of documents/answers/negative documents. You can use it on question or affirmation but sometimes it is used as a fallback (if it's a question or the first intent was not detected).

I will post updates here as I try to improve my POC.

Have a great day

[tadejsv]

Hey @Ierezell , we have just added DPRTextEncoder and DPRReaderRanker executors based on their huggingface implementation!

Take a look and tell us what you think :)