btc-raspberrypiclub / piclub-bot

The official bot for the BTC Rasperry Pi Club Discord server
MIT License
0 stars 1 forks source link

Use the Ollama chat API instead of the generate API #15

Closed Agent-E11 closed 3 months ago

Agent-E11 commented 4 months ago

It currently uses the /api/generate endpoint to generate one-off responses. The /api/chat endpoint fits our use case better, and is better for integrating history/context

joshbuker commented 4 months ago

I'd recommend making a class to contain the conversation history, and instantiating it for each context environment you want (I did per channel, but IIRC you wanted to do per Server).

https://github.com/joshbuker/discord-bot/blob/main/lib/discord_bot/llm/conversation.rb https://github.com/joshbuker/discord-bot/blob/main/lib/discord_bot/bot.rb#L197 https://github.com/joshbuker/discord-bot/blob/main/lib/discord_bot/bot.rb#L253

I'd also recommend abstracting the API calls into a class that can handle the implementation details: https://github.com/joshbuker/discord-bot/blob/main/lib/discord_bot/llm/api_request.rb

Usage example: https://github.com/joshbuker/discord-bot/blob/main/lib/discord_bot/llm/response.rb