Exception Thrown When Using RAG Command with Ollama

[jais001]

Hi, I have set up my environment to use Ollama with R2R. While the CLI commands provided in the documentation work fine for search, I encounter an exception when using the RAG command:

r2r rag --query="who was aristotle?" --do-hybrid-search

The Exception message is:

Traceback (most recent call last):
  File "/R2R/venv/bin/r2r", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/R2R/venv/lib/python3.11/site-packages/r2r/cli/cli.py", line 609, in main
    cli()
  File "/R2R/venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/R2R/venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/R2R/venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/R2R/venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/R2R/venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/R2R/venv/lib/python3.11/site-packages/click/decorators.py", line 45, in new_func
    return f(get_current_context().obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/R2R/venv/lib/python3.11/site-packages/r2r/cli/cli.py", line 394, in rag
    response = obj.rag(
               ^^^^^^^^
  File "/R2R/venv/lib/python3.11/site-packages/r2r/main/execution.py", line 247, in rag
    response = self.client.rag(
               ^^^^^^^^^^^^^^^^
  File "/R2R/venv/lib/python3.11/site-packages/r2r/main/api/client.py", line 342, in rag
    return self._make_request(
           ^^^^^^^^^^^^^^^^^^^
  File "/R2R/venv/lib/python3.11/site-packages/r2r/main/api/client.py", line 130, in _make_request
    handle_request_error(response)
  File "/R2R/venv/lib/python3.11/site-packages/r2r/main/api/client.py", line 56, in handle_request_error
    raise R2RException(
r2r.base.abstractions.exception.R2RException: Internal Server Error

Any assistance with this issue would be greatly appreciated.

[emrgnt-cmplxty]

Looking at logs from the server would be helpful, could you share those?

[jais001]

Hi @emrgnt-cmplxty ,

Please find the logs printed in my console:

2024-07-23 17:38:38,396 - WARNING - r2r.providers.llm.litellm - Request failed (attempt 1): 
2024-07-23 17:39:10,241 - WARNING - r2r.providers.llm.litellm - Request failed (attempt 2): 
2024-07-23 17:39:10,241 - ERROR - r2r.providers.llm.litellm - Completion generation failed: Max retries reached. Last error: 
2024-07-23 17:39:10,242 - ERROR - r2r.base.pipeline.base_pipeline - Pipeline failed with error: Max retries reached. Last error: 
2024-07-23 17:39:10,242 - ERROR - r2r.main.services.retrieval_service - Pipeline error: Max retries reached. Last error: 
INFO:     127.0.0.1:36208 - "POST /v1/rag HTTP/1.1" 500 Internal Server Error

[mmshahid]

I am noticing the same with r2r==0.2.69 client, with ollama. do_hybrid_search= True appears to trigger the pipeline error. when i set do_hybrid_search= False, the error does not come up