search

Azure / aoai-smart-loadbalancing

Smart load balancing for Azure OpenAI endpoints
MIT License
75 stars 26 forks source link

feat: add httpclient proxy support #11

Open plog3050 opened 4 months ago

plog3050 commented 4 months ago

Description

This change is about supporting use cases where this reverse-proxy system needs to have its client counterpart going through an HTTPS_PROXY.

Having this configuration at the HttpClient level directly instead of per backends makes sense in deployment models where all or no backends need to go through the proxy, and never a subset of backends.

plog3050 commented 4 months ago

@microsoft-github-policy-service agree company="GoTo"