search

road-core / service

Apache License 2.0
10 stars 20 forks source link

[RFE] Allow to override system prompt in "debug" mode #158

Open TamiTakamiya opened 3 days ago

TamiTakamiya commented 3 days ago

Is your feature request related to a problem? Please describe. When a chatbot is being developed, developers occasionally want to try out different variations of system prompt. If the service accepts them as an input parameter to /query, it would be useful in such cases.

Describe the solution you'd like Add the system prompt as an optional parameter to the /query api and accept it only in the debug mode, which is set either by the dev_config.enable_dev_ui flag or a new one.

Describe alternatives you've considered Define a new endpoint for that purpose.

Additional context PoC video (code (ansible-chatbot-service))

https://github.com/user-attachments/assets/0147d8ef-6c6d-4d73-b792-6bb1d961e918

tisnik commented 2 days ago

Thanks @TamiTakamiya. Yes it sounds like a very good technique. We can enable it by dev_config.enable_dev_ui as you mentioned, or probably have a new option enable_custom_system_prompt perhaps? (I mean, the enable_dev_ui will enable/disable many "magic" operations, which might not be expected).