timoklimmer / powerproxy-aoai

Monitors and processes traffic to and from Azure OpenAI endpoints.
MIT License
85 stars 24 forks source link

Perf: Reduce scaling delay for about 10 seconds #5

Closed clemlesne closed 7 months ago

clemlesne commented 9 months ago

Note: Requires to be merged after https://github.com/timoklimmer/powerproxy-aoai/pull/3.

timoklimmer commented 7 months ago

This PR is not required anymore with the smart load balancing capabilities added recently. Therefore, closing the PR.