Chainlit / chainlit

Build Conversational AI in minutes ⚡️
https://docs.chainlit.io
Apache License 2.0
6.86k stars 903 forks source link

Upperbound does not work for "duration" #1343

Closed martinskeem closed 1 week ago

martinskeem commented 1 week ago

Describe the bug The documentation mentions that duration can be specified as an upperbound:

https://github.com/BerriAI/litellm/blob/5fb270a559ab9ac26a1c0328ff822c0cc04418f9/docs/my-website/docs/proxy/ui.md?plain=1#L76

Yet, when specified, it does not seem to take effect.

The budget_duration upper bound is respected in the code, but it does not appear that duration is:

https://github.com/BerriAI/litellm/blob/5fb270a559ab9ac26a1c0328ff822c0cc04418f9/litellm/proxy/management_endpoints/key_management_endpoints.py#L158

To Reproduce Steps to reproduce the behavior:

  1. Set litellm_settings.upperbound_key_generate_params.duration to 14d in the configuration.
  2. Reload configuration
  3. Create a key with no duration or a duration which exceeds the specified upper bound duration.
  4. Observe that the upperbound is not implemented.

Expected behavior That the duration is always less than or equal to litellm_settings.upperbound_key_generate_params.duration. Either an error is returned by the API or the value is capped.

martinskeem commented 1 week ago

incorrect posting - apologies