search

simonkurtz-MSFT / python-openai-loadbalancer

Smart Python OpenAI Load Balancer using priority endpoints and request retries. | Python package at link below:
https://pypi.org/project/openai-priority-loadbalancer
MIT License
11 stars 0 forks source link

Investigate Streaming #4

Closed simonkurtz-MSFT closed 6 months ago

simonkurtz-MSFT commented 6 months ago

I have not yet focused on streaming. As I only change URL and Host header, I don't immediately expect issues with streaming, but it's to be tested.