search

foundation-model-stack / fms-guardrails-orchestrator

🚀 Guardrails orchestration server for application of various detections on text generation input and output.
https://foundation-model-stack.github.io/fms-guardrails-orchestrator/
Apache License 2.0
1 stars 13 forks source link

Make generation and chunker sections of configuration optional #166

Open gkumbhat opened 3 weeks ago

gkumbhat commented 3 weeks ago

Description

As a user of detection only / standalone detection endpoints, we want the ability to configure orchestrator with only detectors and potentially not even need chunkers (if users are using the default whole_doc_chunker provided with the orchestrator).

Detection only endpoints (i.e. those under Detection tasks here only work with detectors and do not require generation / calling out to a generation provider such as in config here), making the generation section of the configuration not entirely necessary.

evaline-ju commented 3 weeks ago

Here, we may want to consider making conditionally enabled routes, based on the optional presence of providers, ref. from other work by @declark1 https://github.com/IBM/text-generation-router/blob/3bd9789b2a7039c9f84407903f7e177fcd8bf89f/fmaas-router/src/server.rs#L64-L75