Investigate potential issue when using Discord Bot with LLM application

bifrostlab / llm-assistant

Multifunctional LLM Assistant for Discord

8 stars 6 forks source link

Investigate potential issue when using Discord Bot with LLM application #13

Closed rodonguyen closed 8 months ago

rodonguyen commented 8 months ago

Some concerns are:

LLM may need longer response time
Async/await may not work with LLM app's response

rodonguyen commented 8 months ago

I think the concern regarding long response time from LLM can be resolved by adding await ctx.response.defer() to tell Discord that "Hey, this function will take longer than 3s to respond, don't timeout it"

Interactions docs:

Example: https://github.com/hc20k/LLMChat/blob/master/llmchat/client.py#L306-L307