Guards do not support LiteLLM Router API

[jerome-wego]

Issue

Guardrails is not detecting LiteLLM Router API as a LiteLLM API-compatible LLM API.

Expected usage

from litellm import Router
router = Router(..)
guard = AsyncGuard(..)
messages = [..]

validated_response = await guard(
                router.acompletion,
                model="gpt-4o"
                msg_history=messages,
            )

Where router.acompletion is treated the same as litellm.acompletion.

Error

Currently using Guards as described above results in an error

The callable fn passed to Guard(fn, ...) failed with the following error: Router.acompletion() missing 1 required positional argument: 'messages'. Make sure that fn can be called as a function that takes in a single prompt string and returns a string.

Suspect changes needed here

https://github.com/guardrails-ai/guardrails/blob/5363c0793a6b155d7f72eb85c7504079bac61c2a/guardrails/llm_providers.py#L923

[CalebCourier]

@jerome-wego Thanks for raising this issue! Currently guardrails does not explicitly support LiteLLM's Router API internally. In order to use litellm routers with the current version you will need to wrap the method as a custom llm. See here for more details: https://www.guardrailsai.com/docs/how_to_guides/llm_api_wrappers#build-a-custom-llm-wrapper

That said, we will take adding support for this into consideration.

[Jqnxyz]

Thanks for pointing me in the right direction + the prompt response @CalebCourier, I greatly appreciate it!

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 14 days with no activity.