[Bug]: RaptorRetriever query error : RuntimeError: this event loop is already running.

[ljhssga]

Bug Description

Use sanic as a web server to provide an interface. Use the RetrieverQueryEngine to query query_engine.query(""), and report RuntimeError: this event loop is already running.

Version

0.10.52

Steps to Reproduce

retriever = RaptorRetriever( [], embed_model=OpenAIEmbedding( model="text-embedding-3-small", api_key=api_key, api_base=base_url ), # used for embedding clusters llm=OpenAI(model="gpt-3.5-turbo", temperature=0.1, api_key=api_key, api_base=base_url),

used for generating summaries

vector_store=vector_store,  # used for storage
similarity_top_k=2,  # top k for each layer, or overall top-k for collapsed
mode="tree_traversal",  # sets default mode

)

query_engine = RetrieverQueryEngine.from_args( retriever, llm=OpenAI(model="gpt-3.5-turbo", temperature=0.1, api_key=api_key, api_base=base_url), text_qa_template=CHAT_TEXT_QA_PROMPT, streaming=True, response_synthesizer=synth, use_async=True )

@app.route("/test", methods=['POST']) async def test(req: request): res = query_engine.query("who are you").response_gen return res

if name == "main": app.run(host='0.0.0.0', port=8777, workers=1, access_log=False)

Relevant Logs/Tracbacks

File "/raptor/venv/lib/python3.11/site-packages/nest_asyncio.py", line 35, in run
    loop.run_until_complete(task)
  File "uvloop/loop.pyx", line 1511, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1504, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1377, in uvloop.loop.Loop.run_forever
  File "uvloop/loop.pyx", line 518, in uvloop.loop.Loop._run
RuntimeError: this event loop is already running.

[dosubot[bot]]

Hello, @ljhssga! 👋 I'm here to help you with any bugs, questions, or contributions you have. I'm an expert bot designed to assist you while you wait for a human maintainer.

To resolve the RuntimeError: this event loop is already running when using Sanic as a web server and querying with RetrieverQueryEngine, you can use the nest_asyncio library. This library allows you to nest asyncio event loops, which can help you run an event loop within an already running event loop.

Here's how you can modify your code:

1. Install nest_asyncio:

   pip install nest_asyncio

2. Apply nest_asyncio in your code:

   import nest_asyncio
   nest_asyncio.apply()

3. Update your Sanic app:

   import nest_asyncio
   from sanic import Sanic
   from sanic.response import json
   from your_module import RaptorRetriever, RetrieverQueryEngine, OpenAIEmbedding, OpenAI, vector_store, CHAT_TEXT_QA_PROMPT, synth
   
   # Apply nest_asyncio
   nest_asyncio.apply()
   
   app = Sanic("MyApp")
   
   retriever = RaptorRetriever(
      [],
      embed_model=OpenAIEmbedding(
          model="text-embedding-3-small", api_key=api_key, api_base=base_url
      ),
      llm=OpenAI(model="gpt-3.5-turbo", temperature=0.1, api_key=api_key, api_base=base_url),
      vector_store=vector_store,
      similarity_top_k=2,
      mode="tree_traversal",
   )
   
   query_engine = RetrieverQueryEngine.from_args(
      retriever, llm=OpenAI(model="gpt-3.5-turbo", temperature=0.1, api_key=api_key, api_base=base_url),
      text_qa_template=CHAT_TEXT_QA_PROMPT,
      streaming=True,
      response_synthesizer=synth,
      use_async=True
   )
   
   @app.route("/test", methods=['POST'])
   async def test(req):
      res = await query_engine.aretrieve("who are you")
      return json(res)
   
   if __name__ == "__main__":
      app.run(host='0.0.0.0', port=8777, workers=1, access_log=False)

By applying nest_asyncio, you can avoid the RuntimeError and run your Sanic web server alongside other asynchronous tasks ^[1][2].

To continue talking to Dosu, mention @dosu.

[logan-markewich]

I see uvloop, so I'm guessing you are using fastapi

Try setting the loop type unicorn.run(...., loop="asyncio")

[ljhssga]

I see uvloop, so I'm guessing you are using fastapi

Try setting the loop type unicorn.run(...., loop="asyncio")

i used Sanic

[ljhssga]

Hello, @ljhssga! 👋 I'm here to help you with any bugs, questions, or contributions you have. I'm an expert bot designed to assist you while you wait for a human maintainer.

To resolve the RuntimeError: this event loop is already running when using Sanic as a web server and querying with RetrieverQueryEngine, you can use the nest_asyncio library. This library allows you to nest asyncio event loops, which can help you run an event loop within an already running event loop.

Here's how you can modify your code:

1. Install nest_asyncio:

   pip install nest_asyncio

2. Apply nest_asyncio in your code:

   import nest_asyncio
   nest_asyncio.apply()

3. Update your Sanic app:

   import nest_asyncio
   from sanic import Sanic
   from sanic.response import json
   from your_module import RaptorRetriever, RetrieverQueryEngine, OpenAIEmbedding, OpenAI, vector_store, CHAT_TEXT_QA_PROMPT, synth
   
   # Apply nest_asyncio
   nest_asyncio.apply()
   
   app = Sanic("MyApp")
   
   retriever = RaptorRetriever(
      [],
      embed_model=OpenAIEmbedding(
          model="text-embedding-3-small", api_key=api_key, api_base=base_url
      ),
      llm=OpenAI(model="gpt-3.5-turbo", temperature=0.1, api_key=api_key, api_base=base_url),
      vector_store=vector_store,
      similarity_top_k=2,
      mode="tree_traversal",
   )
   
   query_engine = RetrieverQueryEngine.from_args(
      retriever, llm=OpenAI(model="gpt-3.5-turbo", temperature=0.1, api_key=api_key, api_base=base_url),
      text_qa_template=CHAT_TEXT_QA_PROMPT,
      streaming=True,
      response_synthesizer=synth,
      use_async=True
   )
   
   @app.route("/test", methods=['POST'])
   async def test(req):
      res = await query_engine.aretrieve("who are you")
      return json(res)
   
   if __name__ == "__main__":
      app.run(host='0.0.0.0', port=8777, workers=1, access_log=False)

By applying nest_asyncio, you can avoid the RuntimeError and run your Sanic web server alongside other asynchronous tasks [1][2].

To continue talking to Dosu, mention @dosu.

It doesn't work

[dosubot[bot]]

To resolve the RuntimeError: this event loop is already running when using Sanic and RetrieverQueryEngine, you can try the following steps:

1. Ensure nest_asyncio is applied correctly:

   import nest_asyncio
   nest_asyncio.apply()

2. Modify your Sanic route to use asynchronous methods:

   from sanic import Sanic, response
   from sanic.exceptions import SanicException
   from your_module import RaptorRetriever, RetrieverQueryEngine, OpenAIEmbedding, OpenAI, vector_store, CHAT_TEXT_QA_PROMPT, synth
   
   app = Sanic("MyApp")
   
   retriever = RaptorRetriever(
      [],
      embed_model=OpenAIEmbedding(
          model="text-embedding-3-small", api_key=api_key, api_base=base_url
      ),
      llm=OpenAI(model="gpt-3.5-turbo", temperature=0.1, api_key=api_key, api_base=base_url),
      vector_store=vector_store,
      similarity_top_k=2,
      mode="tree_traversal",
   )
   
   query_engine = RetrieverQueryEngine.from_args(
      retriever, llm=OpenAI(model="gpt-3.5-turbo", temperature=0.1, api_key=api_key, api_base=base_url),
      text_qa_template=CHAT_TEXT_QA_PROMPT,
      streaming=True,
      response_synthesizer=synth,
      use_async=True
   )
   
   @app.route("/test", methods=['POST'])
   async def test(req):
      try:
          res = await query_engine.aretrieve("who are you")
          return response.json(res)
      except Exception as e:
          raise SanicException(f"Error processing query: {str(e)}", status_code=500)
   
   if __name__ == "__main__":
      app.run(host='0.0.0.0', port=8777, workers=1, access_log=False)

3. Ensure you are not running conflicting event loops:

   • Make sure no other part of your code is trying to run an event loop concurrently.

By applying nest_asyncio and ensuring the use of asynchronous methods, you should be able to avoid the RuntimeError and run your Sanic web server alongside other asynchronous tasks ^[1][2].

To continue talking to Dosu, mention @dosu.

[ljhssga]

I tried to add import nest_asyncio nest_asyncio.apply() But he still didn't work