search

simonkurtz-MSFT / python-openai-loadbalancer

Smart Python OpenAI Load Balancer using priority endpoints and request retries. | Python package at link below:
https://pypi.org/project/openai-priority-loadbalancer
MIT License
11 stars 0 forks source link

LoadBalancer does not yet support async #1

Closed simonkurtz-MSFT closed 6 months ago

simonkurtz-MSFT commented 6 months ago

The initial implementation focused on the synchronous flow. It wouldn't take much to expand load_balancer.py to add support for async, I suspect.