Bot real slow to post / edit.

jakobdylanc / llmcord

A Discord LLM chat bot that supports any OpenAI compatible API (OpenAI, xAI, Mistral, Groq, OpenRouter, Ollama, LM Studio and more)

MIT License

362 stars 68 forks source link

Bot real slow to post / edit. #21

Closed JustinCase144 closed 7 months ago

JustinCase144 commented 8 months ago

Hi,

I had great success with this project when using ollama.

I swapped to a model that runs on koboldai (openai compatible) and now the bot responses take ages.

It processes the input quick and has the answer ready in ~30 seconds max, but when the bot is posting the answer to discord, python3 starts running 100% on a single core and the bot takes ages to finish posting.

Any idea what I'm doing wrong?

jakobdylanc commented 8 months ago

I haven't tested with koboldai so I can't guarantee support for it. What specific model are you using? Have you tried others?