search

jakobdylanc / llmcord.py

A Discord LLM chat bot that supports any OpenAI compatible API (OpenAI, Mistral, Groq, OpenRouter, ollama, oobabooga, Jan, LM Studio and more)
MIT License
310 stars 57 forks source link

Sorry for it, but i want to ask smthing (The bot is a bit stupid) #57

Closed Inklare closed 2 weeks ago

Inklare commented 2 weeks ago

Why is this bot dumber than if I turn to a regular model (I mean to gemma 2 via terminal for example). It can't solve 5+5*30... Can i fix it?

jakobdylanc commented 2 weeks ago

You could try writing a better system prompt that's more suited to your task. You could also try adjusting API parameters that affect the LLMs behavior like "temperature" and "top_p" (google around to learn more).

If you're seeing better performance with the same LLM in a different frontend, it's just a matter of copying whatever system prompt / API parameters that frontend is using. There's nothing else to it really.

(Besides stuff like RAG but I don't think that applies here)

Inklare commented 2 weeks ago

You could try writing a better system prompt that's more suited to your task. You could also try adjusting API parameters that affect the LLMs behavior like "temperature" and "top_p" (google around to learn more).

If you're seeing better performance with the same LLM in a different frontend, it's just a matter of copying whatever system prompt / API parameters that frontend is using. There's nothing else to it really.

(Besides stuff like RAG but I don't think that applies here)

I set temperature to 0.3 and this fixed the problem, thank you