Enable DPRReaderRanker to copy meta information to the matches. - Githubissues

jina-ai / executors

internal-only

Apache License 2.0

31 stars 12 forks source link

Enable DPRReaderRanker to copy meta information to the matches. #288

Closed nan-wang closed 2 years ago

nan-wang commented 2 years ago

As a user of DRPReaderRanker, I would like to copy meta information to the matches besides the text.

We have some prototype in the video search QA example, https://github.com/jina-ai/example-video-qa/blob/main/executors.py#L106-L285.

After this task is done, we need to use jinahub+docker://DPRReaderRanker in the example-video-qa.

TODO:

[x] migrate DPRReaderRanker to a single repo
[x] add support to copying arbitrary meta information

nomagick commented 2 years ago

Migrated to https://github.com/jina-ai/executor-text-transformers-dprreader-ranker

nomagick commented 2 years ago

Meta information preserved by copy over the tags: https://github.com/jina-ai/executor-text-transformers-dprreader-ranker/blob/c78a1062936a74120578e91a74423e4155d8d4c7/dpr_reader.py#L181 Is this enough? All tests except GPU ones are passing.