[Feature]: [29/01/2024 - 05/02/2024] New Models/Endpoints/Providers/Improvements

[ishaan-jaff]

UI v2.0 Improvements

• [x] Proxy SSO Login - Add OpenID, Azure AD https://github.com/BerriAI/litellm/issues/1658
• [x] [Feature-UI]: UI Static Web App on Proxy
• [ ] [Feature-UI]: Show users the available models for a Key
• [x] Quick Spin Up UI https://github.com/BerriAI/litellm/issues/1742
• [x] Make SSO Optional on UI https://github.com/BerriAI/litellm/issues/1687
• [ ] [Feature-UI]: Show the User ID, User Spend attributes on UI v2 https://github.com/BerriAI/litellm/issues/1695
• [x] Make it easy to use UI with Proxy - Static Web App https://github.com/BerriAI/litellm/issues/1631
• [x] UI - when user sets up SSO for UI - ensure allow_user_auth is True https://github.com/BerriAI/litellm/issues/1710
• [x] UI - stop pasting proxy api keys in the url https://github.com/BerriAI/litellm/issues/1710
• [ ] Have admin role defined by sub in oath token - https://github.com/BerriAI/litellm/issues/1808

Sharing Proxy API Keys

• [x] include version of LiteLLM package in use in the response json for /health/readiness https://github.com/BerriAI/litellm/issues/1664
• [x] Allow Proxy Admin to specify success_callback + success_callback params per user https://github.com/BerriAI/litellm/issues/1725
• [x] Dockerfile for litellm dashboard v2 - https://github.com/BerriAI/litellm/issues/1671
• [x] Cache Controls Per Key - https://github.com/BerriAI/litellm/issues/1599
• [x] Support langfuse cost tracking https://github.com/BerriAI/litellm/issues/1699
• [ ] Expose /user/delete endpoint https://github.com/BerriAI/litellm/issues/1711

Misc

• [x] Allow users to set organization for OpenAI on Proxy Config.yaml https://github.com/BerriAI/litellm/issues/1666
• [x] Graceful rejection of token input for AWS Embeddings API https://github.com/BerriAI/litellm/issues/1681
  • [ ] See (By User) to see total requests, Total tokens used (completion, embedding), spend,
  • [ ] End User Emails when a key is near its budget
  • [ ] add Region-specific amazon text embedding pricing - https://github.com/BerriAI/litellm/issues/1566
  • [x] Allow local model cost maps for proxy https://github.com/BerriAI/litellm/issues/1434
  • [ ] Gemini Pro - VertexAI function calling when streaming support - https://github.com/BerriAI/litellm/issues/1263
  • [x] Caching improvement for embedding calls - https://github.com/BerriAI/litellm/issues/1350
  • [x] Reduce proxy cold start times - https://github.com/BerriAI/litellm/issues/1636
  • [x] Dynamically set vertex ai project - https://github.com/BerriAI/litellm/issues/1613
  • [ ] Add whisper support - https://github.com/BerriAI/litellm/issues/1672
  • [x] Router - Timeouts + Fallbacks raise exception on router.completion https://github.com/BerriAI/litellm/issues/1583
  • [ ] Logging for cached objects is none https://github.com/BerriAI/litellm/issues/1698
  • [x] Don't log generated token in logging object - https://github.com/BerriAI/litellm/issues/1706
  • [ ] Add vertex ai imagen support - https://github.com/BerriAI/litellm/issues/1707
  • [ ] log image count in spend logs https://github.com/BerriAI/litellm/issues/1731
  • [x] gecko embedding support https://github.com/BerriAI/litellm/issues/1791
  • [x] Bedrock stable diffusion support - https://github.com/BerriAI/litellm/issues/1770
  • [x] VertexAIException - list index out of range Error (high-traffic) https://github.com/BerriAI/litellm/issues/1774
  • [ ] Improve proxy cold start time - https://github.com/BerriAI/litellm/issues/1769

[krrishdholakia]

Known Backlog:

1. OpenMeter spend tracking support - https://github.com/BerriAI/litellm/issues/1268
2. tracking - https://github.com/BerriAI/litellm/issues/997
3. tracking - https://github.com/BerriAI/litellm/issues/1188 cc: @r1cc4rd0m4zz4
4. tracking - https://github.com/BerriAI/litellm/issues/1192
5. yandexgpt support requested by 2 users - https://github.com/BerriAI/litellm/issues/1254
6. Aphrodite engine endpoint - https://github.com/PygmalionAI/aphrodite-engine, https://github.com/BerriAI/litellm/pull/1153
7. TheBloke/NexusRaven-V2-13B-GGUF function calling support https://github.com/BerriAI/litellm/issues/1060 cc: @homanp
8. Predibase lorax support - [Feature]: Add support for Predibase Lorax apis #1253
9. Deepsparse CPU inference support - https://github.com/neuralmagic/deepsparse/issues/1452
10. Consistent format for model prices and context window - https://github.com/BerriAI/litellm/issues/1375
11. Faster bedrock embedding calls - https://github.com/BerriAI/litellm/issues/1798