katanemo / arch

Arch is an intelligent prompt gateway. Engineered with (fast) LLMs for the secure handling, robust observability, and seamless integration of prompts with APIs - all outside business logic. Built by the core contributors of Envoy proxy, on Envoy.
https://archgw.com
Apache License 2.0
367 stars 19 forks source link

System prompt not sent to default LLM, when default promp_target not found #180

Open salmanap opened 1 week ago

salmanap commented 1 week ago

promp_target not found. Arch accepts an arch_config.yaml without a default target. But if the user sends a general QA prompt that doesn't match a specific prompt target, the system prompt is not being sent to the default LLM. This results in a behavior where the default LLM simply returns back its capabilities as compared to the capabilities being designed by the developer.