DiscordRP
This Discord bot is designed to offer dynamic interactions with locally hosted LLMs inside of a Discord server, featuring character roleplay, retrieval augmented generation (RAG), and custom API interactions for generating responses. It's built with Python and utilizes discord.py, the original intent of this bot was to rewrite my original bot, GPT-4-Discord-Bot-Long-Term-Memory, to work with locally hosted LLMs. In the process of doing so, I found it easier to start from scratch and added a significant number of features along the way.
Local LLM Support:
DiscordRP is designed around using locally hosted LLMs, as such I recommend using koboldcpp for easy setup, however, the bot should work with any LLM API, local or not, that utilizes the same framework.
OpenAI Support:
Currently the bot does work with OpenAI LLMs and 3.5 Turbo is hardcoded, you can switch to OpenAI utilizing "/select_api openai" inside your chat channel. Changing from 3.5 is easy, but will require editing the code. Functionally, there is a loss using OpenAI LLMs and I don't recommend it.
Retrieval Augmented Generation (RAG):
RAG is how the bot extends its memory beyond the context allowed by the LLM (or your hardware). Each message is logged with a vector embedding and the bot retrieves relevant memories to add to the prompt for each message sent. It's certainly not perfect but it does help the bot remember details.
Embeddings:
RAG is done via vector embeddings, currently the bot uses OpenAI embeddings model ada-002 which does require an API key from OpenAI and its extremely cheap (pennies), optionally, "/select_embeddings local" can switch you to hardcoded local embeddings utilizing Swiss Army Llama on Docker. I have done very little work to make this better as my hardware doesn't really support running both locally.
Characters:
Characters are managed via "/select_character", setting a new name will generate a new character card, which simply contains prompt information for that character. The bot should default to "Narrator", which I personally find the best sort of base character for RP. To change the prompt for the character utilize "/set_character_prompt".
Roleplay Setting:
Roleplay setting is managed via "/set_roleplay_setting", this adds a line into the prompt that provides setting details, useful if you are using multiple characters.
Setup:
Bot Commands:
/set_character_prompt - Sets the primary portion of the prompt, this tells the LLM what you want it to do, this is where you input the character information and how the character should behave, default character is "Narrator"
/select_character - Sets the name the bot will use for the next message, if its a new name you should /set_character_prompt to provide details for the new character, otherwise it will use a default prompt
/set_roleplay_setting - This is optional, provides a fixed (or changing) setting information for the RP. Very important if you're switching characters often to maintain the setting details (when and where)
/delete_logs_and_messages - This deletes the input number of logs and messages from the server, important if you want to rewind the conversation or get a bad generation
/set_max_length - Sets the length of the response for the LLM, defaults to 200
/set_message_history - Sets the number of past messages to include in the prompt, defaults to 20. The higher the better,but consider your hardware and the context length of the LLM and how long you want to wait for a response
/set_temperature - Sets the temperature for the LLM, defaults to .9
/add_stopping_string - Adds stopping strings that will stop the LLM generation. Use this if the LLM is responding in ways you don't want, usually things like [user] will crop up from time to time, I probably can fix most of this later
/select_api - "openai" OR "local", discussed above
/set_response_url - Use this if you need to change where the API is listening, for instance if you're using cloudflare or LAN or change the KoboldCPP port
Tips:
The default prompts are there to make the bot function out of the box, I suggest rewriting them for your use case. The Narrator character works really well for me, I recommend using it but changing the prompt to include more details about what you want the Narrator to do.
For lower system harware I recommend, openchat-3.5-1210.Q8_0
For high end hardware, Miqu 70b varients.