search

rmusser01 / tldw

tl/dw (Too Long, Didn't Watch): Your Personal Research Multi-Tool - a naive attempt at 'A Young Lady's Illustrated Primer'
Apache License 2.0
268 stars 9 forks source link

issue with rag search using ollama #375

Closed jcl2023 closed 1 day ago

jcl2023 commented 1 day ago

I got the following error with Select API for RAG ollama:

DEBUG:matplotlib.pyplot:Loaded backend tkagg version 8.6. DEBUG:matplotlib.pyplot:Loaded backend agg version v2.2. ERROR:root:Using embedding provider: openai DEBUG:root:Using embedding provider: openai ERROR:root:

enhanced_rag_pipeline - Keywords: [] DEBUG:root:

enhanced_rag_pipeline - Keywords: [] ERROR:root:

enhanced_rag_pipeline - relevant media IDs: None DEBUG:root:

enhanced_rag_pipeline - relevant media IDs: None ERROR:root:

enhanced_rag_pipeline - Vector search results: [] DEBUG:root:

enhanced_rag_pipeline - Vector search results: [] ERROR:root:

enhanced_rag_pipeline - Full-text search results: DEBUG:root:

enhanced_rag_pipeline - Full-text search results: DEBUG:root:

enhanced_rag_pipeline - Full-text search results:

ERROR:root: enhanced_rag_pipeline - Applying Re-Ranking DEBUG:root: enhanced_rag_pipeline - Applying Re-Ranking INFO:flashrank.Ranker:Downloading ms-marco-TinyBERT-L-2-v2... DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): huggingface.co:443 DEBUG:urllib3.connectionpool:https://huggingface.co:443 "GET /prithivida/flashrank/resolve/main/ms-marco-TinyBERT-L-2-v2.zip HTTP/11" 302 1206 DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): cdn-lfs-us-1.hf.co:443 DEBUG:urllib3.connectionpool:https://cdn-lfs-us-1.hf.co:443 "GET /repos/e9/59/e9591f644d813f4e52700ee468f821dd9fa672776003bf93d1275ff0133539bf/752eddf1c5ece3c5e6115e9ab52eb3b16436bbf9b3a091cbb62b5b8eadc47105?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27ms-marco-TinyBERT-L-2-v2.zip%3B+filename%3D%22ms-marco-TinyBERT-L-2-v2.zip%22%3B&response-content-type=application%2Fzip&Expires=1729627197&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcyOTYyNzE5N319LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmhmLmNvL3JlcG9zL2U5LzU5L2U5NTkxZjY0NGQ4MTNmNGU1MjcwMGVlNDY4ZjgyMWRkOWZhNjcyNzc2MDAzYmY5M2QxMjc1ZmYwMTMzNTM5YmYvNzUyZWRkZjFjNWVjZTNjNWU2MTE1ZTlhYjUyZWIzYjE2NDM2YmJmOWIzYTA5MWNiYjYyYjViOGVhZGM0NzEwNT9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSomcmVzcG9uc2UtY29udGVudC10eXBlPSoifV19&Signature=jRLLBvckuDd5IP1svyupnbwSHfSm1GZcA8oPRok6HYgAbZ9iScp7L-S8HRQyw3ODF2UepgRKwsgR-3JnjQMwstdY3TpxBoz9956GUZ4kbK9QY7XZHEFSJ~rvOyFztfCEXqVU3OUZyk~UoG6Qdczx9AoC9saVBBfLzF2mpW5MokifY8cPqbpgjoxZMGODtESUcT6fsiNKedNDxSZ45ndB-Zopzu7~D0TG5JjK19ags20n~6GlvmM0u5SyjU2vfbTNpzck5lD5qmJKV48zWX4CnzxgqrbHL0ZWZvIIWc9JLCjKSx2-hGPwg9xWqCt7GFN-ew1dkIjJqwEZFBPvasvlLQ__&Key-Pair-Id=K24J24Z295AEI9 HTTP/11" 200 3421983 ms-marco-TinyBERT-L-2-v2.zip: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3.26M/3.26M [00:00<00:00, 12.4MiB/s] ERROR:root: enhanced_rag_pipeline - Ranker ERROR:root: enhanced_rag_pipeline - Re-Ranking Request <flashrank.Ranker.RerankRequest object at 0x7cbf3a1fb290> DEBUG:flashrank.Ranker:Running pairwise ranking.. ERROR:root:Error in enhanced_rag_pipeline: Required inputs (['token_type_ids']) are missing from input feed (['input_ids', 'attention_mask']). DEBUG:matplotlib.pyplot:Loaded backend tkagg version 8.6.

rmusser01 commented 1 day ago

Thank you for filing the issue, can you please tell me what OS / platform you're using?

Can you also please reproduce the issue while runng summarize.py with -gui -log DEBUG?

I am unable to replicate this issue on my local install on windows.

Do you happen to have any media in your media DB?

Just did some more experimentation and the issue is caused by the re-ranker having no items to re-rank, as the Embeddings DB is empty. This is expected behavior(But will add a catch/notification to the user), as the RAG search is aimed at specifically performing search against existing items, while the RAG chat is meant for everything else.

This will most likely be rolled into one UI page eventually for 1.0 .

So, in the meantime, you can use the RAG QA, or wait a bit and I'll add a sanity check for it so that it doesn't perform re-ranking if there aren't any items present.

rmusser01 commented 1 day ago

Ok, your issue should now be resolved with the latest pull. I've gone ahead and added a sanity check for re-ranking to take place during the RAG query as well as making it optional. I've gone ahead and validated that it works with ollama as well, so please update to the latest version to resolve the issue.