Add support for multiple azure deployments

chenfei-wu / TaskMatrix

Other

34.55k stars 3.32k forks source link

Add support for multiple azure deployments #451

Open krrishdholakia opened 1 year ago

krrishdholakia commented 1 year ago

Hey @chenfei-wu @jordddan,

This PR adds support for letting users call across multiple Azure deployments. It uses LiteLLM's Router to manage across rate limits, to prevent requests from failing in prod.