search

dorianbrown / rank_bm25

A Collection of BM25 Algorithms in Python
Apache License 2.0
1.05k stars 89 forks source link

[Feature Request] Async Support #34

Open logan-markewich opened 1 year ago

logan-markewich commented 1 year ago

Hey! We are adding BM25 search to LlamaIndex here https://github.com/jerryjliu/llama_index/pull/7342

It would be super cool if the library supported async retrieval at some point 🙏🏻 If that's not possible though, that's ok too, but it would improve the usability of the library quite a bit

andersfylling commented 1 year ago

How does coroutine support matter here?

logan-markewich commented 1 year ago

Being able to call the retriever asynchronously helps a lot if you are running on a server for example

For example, in LlamaIndex, we can call embeddings or LLMs asynchronously so that they don't block the entire server thread

andersfylling commented 1 year ago

coroutine scheduling happens within a single thread. Given this is CPU bound, you won't be able to execute any other coroutines at the time. Meaning, it will actually harm your performance instead of improving it.

For CPU bound tasks I suggest using anyio and send off the work to a background process:

result = await anyio.to_thread.run_sync(some_task, "arg")