search

phidatahq / ai-cookbook

Mozilla Public License 2.0
55 stars 11 forks source link

Example of RAG with link to reference or source/context also returned? #3

Open asmith26 opened 7 months ago

asmith26 commented 7 months ago

Hi, just wondering do you have any examples of using PhiData with RAG where a link/context is also returned, possibly something like:

image

Thanks for any help!

ashpreetbedi commented 7 months ago

@asmith26 yes 100% -- we store all the metadata in the knowledge base so its easy to build. Give me a week to get that setup?

asmith26 commented 6 months ago

Hi @ashpreetbedi, just thought I'd ask if you have been able to make any progress on this - of course no worries if not, and thanks again for your help :)

ashpreetbedi commented 6 months ago

Hey @asmith26 sorry been a bit busy so haven't had a chance to work on this, that said I should be able to get to this by end of this week/early next week.

Still deciding between showing all the resources used to generate the answer, or asking the LLM to provide the references it used (from the resources we send).

Any opinions?

For example: If we send 5 chunks for RAG, and its from 3 diff docs, but the LLM only uses 2 chunks from 1 doc to generate the answer. Do we want to show all 3 docs (i.e. whatever was retrieved from the knowledge base) or show the 1 doc that was used to generate the answer.

asmith26 commented 6 months ago

No problem at all, I really appreciate the help (and the lib!)

Good question. Perhaps showing just the 1 doc that was used to generate the answer might be sufficient as well as possibly keep things/the output simpler.

Possibly another option: could always show something like the top N matching docs/chunks, and these could be ranked based on the similarity score.

Just some thoughts/hope this helps, and thanks again for your help!

asmith26 commented 6 months ago

Hi @ashpreetbedi, hope you're well - I had one more thought, if it's possible to create an Extractive Question & Answering pipeline, where answer is highlighted directly in the source/context (like in the image), that would be very helpful:

image (image from: https://github.com/deepset-ai/haystack-demos/tree/main/healthcare)

ashpreetbedi commented 6 months ago

@asmith26 this is an awesome example, looking into it. hope to have examples for you soon :)

ashpreetbedi commented 6 months ago

Going to start working on this next monday -- this is very cool

ashpreetbedi commented 6 months ago

@asmith26 i have a working solution for this used by a client. just havent pushed it to this repo yet