This release contains 1 dependency update and 2 bug fixes.
⚙ Dependency update
Update GRPC version requirements (#6110)
The grpc version requirements have been updated to allow grpcio<=1.57.0.
🐞 Bug Fixes
Better handling of Exceptions in dynamic batching (#6128)
An issue was identified when a dynamic batch raises an Exception that could affect some unrelated requests.
Load-balancing streaming based on response type (#6122)
When using Deployment locally with multiple replicas, a load-balancing process is added in front of the replicas.
The load balancer assumed all GET requests to be streaming, but this may not be true for user-added FastAPI endpoints. We have fixed this assumption and now use the response type to determine if a request is streaming.
🤟 Contributors
We would like to thank all contributors to this release:
Release Note
This release contains 1 dependency update and 2 bug fixes.
⚙ Dependency update
Update GRPC version requirements (#6110)
The
grpcversion requirements have been updated to allowgrpcio<=1.57.0.🐞 Bug Fixes
Better handling of Exceptions in dynamic batching (#6128)
An issue was identified when a dynamic batch raises an Exception that could affect some unrelated requests.
Load-balancing streaming based on response type (#6122)
When using
Deploymentlocally with multiple replicas, a load-balancing process is added in front of the replicas. The load balancer assumed allGETrequests to be streaming, but this may not be true for user-added FastAPI endpoints. We have fixed this assumption and now use the response type to determine if a request is streaming.🤟 Contributors
We would like to thank all contributors to this release: