Async MLM

[sundar-sarvam]

Async llm

[liqul]

Not on our backlog due to limited bandwidth. A workaround is to wrap TaskWeaver as an async module which would cause some performance loss. However, the current situation is that the bottleneck are usually the LLM callings, not because of the way of implementation (sync or async). That's why we lower the priority of adding another path of async calls to LLM in our project.