Advanced Mode Prompt Editing

[lenaxia]

Description: Allow advanced users to modify the core prompts to fit their needs more appropriately.

Ask:

• A page which allows the user to modify the core prompts with templating support
• A button to reset the prompt to default should the user need to.
• Bonus: Prompts are configurable via file or env var to allow for better gitops deployment (e.g. k8s)

[david1542]

Thanks for creating the issue, @lenaxia.

Can you elaborate a bit more about what would you hope to achieve by manipulating the prompts? Do you wish to "steer" the LLM during investigation/chat mode?

[lenaxia]

Yes, I have found projects that were promising but the built in prompts were less than great, or did not work well with the LLM backend I was using.

MemGPT is an example where they have a great idea, but their prompts are insanely long and cause some LLMs to lose context or eat up all the available context (e.g. on Llama3).

Also as prompting techniques evolve, its nice to be able to update without having to wait for the maintainer to update.

[david1542]

@lenaxia thanks for the clarification.

Do you have a specific flow in mind? I'm trying to understand how you view the "steering" ability, and make sure we'll build the right thing.

Would you want to specifically guide the LLM to specific places via prompt editing by editing the system prompt? Or would you like to build custom workflows for each incident/alerts?

We're thinking about creating a workflows feature where you can guide the agent explicitly (for each incident) by creating a workflow, and then you can edit the prompt in each step phase, kinda like n8n. We think that can make the tool extremely flexible.

I'm trying to understand how much control would you want over the investigation process :)