Closed daiwaid closed 4 months ago
For naming: let's name all async methods explicitly with an async_
prefix, e.g., async_match
. Its sync wrapper should name as-is.
We should get an idea of performance (tput) improvement with async
first, also the overhead of asyncio loop init in the sync call.
Note that vs. #34 (query batching, async support is needed because query batching does not work well when there is tool use (e.g., browsing) or more complex match outputs.
With #34 in, we should decide the necessity for async match (async model call is still well motivated, though) -- it should further improve the speed exploiting request-level parallelism.
Continued in #37.
Thanks @daiwaid. The async implementation looks good to me overall. Two proposals: