search

etkecc / baibot

🤖 A Matrix bot for using diffent capabilities (text-generation, text-to-speech, speech-to-text, image-generation, etc.) of AI / Large Language Models (OpenAI, Anthropic, etc.)
GNU Affero General Public License v3.0
48 stars 4 forks source link

An agent's overridden prompt is not reflected when subsequently inspecting agent details #16

Closed hardye closed 2 months ago

hardye commented 2 months ago

After overiding an agent's prompt using !bai config room text-generation set-prompt-override the new prompt is not reflected when trying to view the current active prompt using !bai agent details MY_AGENT.

To reproduce:

  1. Create a new room-local agent (I used the mistral provider but I assume the others work the same) and configure it with a generic prompt: "You are a brief but helpful bot"
  2. Set the agent as a handler for text-generation or catch-all.
  3. Ask the bot what the current time is. Observe that it doesn't know.
  4. Override the generic prompt with a better one using the command !bai config room text-generation set-prompt-override You are a brief, but helpful bot called {{ baibot_name }} powered by the {{ baibot_model_id }} model. The date/time now is: {{ baibot_now_utc }}.
  5. Ask the bot, what the current time is. Observe that it will give a correct answer. Thus, the new prompt is now in effect.
  6. Inspect the agent's current configuration using !bai agent details MY_AGENT

At this point I would expect the new prompt to be printed in the YAML output. However, the prompt in the YAML configuration lists the original prompt, before it was overriden using set-prompt-override.

spantaleev commented 2 months ago

This is by design. Agents can be defined statically (in the YAML configuration) or dynamically (per room or globally).

To change the prompt, you either need to edit the agent (wherever its configuration is defined) or you can apply an override (per room or globally).

The override is just that - an override. The original agent configuration remains in place - other agent users are potentially unaffected.

So it sounds like you'd like to edit the agent, not use an override.

The voice override and speed override settings behave the same way - overriding agent configuration (for a given room or globally).

You can use the !bai config status command at any time to see what the current effective configuration is.

hardye commented 2 months ago

Thank you, @spantaleev . Given that the agent's prompt can be dynamically reconfigured I was essentially looking for a way to view the current prompt configuration. And indeed !bai config status includes the prompt; I had missed that earlier.

Problem solved. Appreciate the quick response.

And let me add that this project is a real pleasure to work with. Being able to test, compare and evaluate various LLM providers in such a simple way is phenomenal. I'm having a ton of fun. Thank you for coming up with this.