Batch operations improve the performance of middlewares across multiple use-cases (GPU batch operations, API calls with multiple elements, etc.). Middlewares today do not entirely leverage batch operations, and a refactor of several middlewares would be required to improve throughput and performance.
Solution/User Experience
Middlewares can leverage batch operations when consuming messages from their input SQS queue. We want to refactor some middlewares to drop the use of partial responses and leverage the batch of input messages.
Use case
Batch operations improve the performance of middlewares across multiple use-cases (GPU batch operations, API calls with multiple elements, etc.). Middlewares today do not entirely leverage batch operations, and a refactor of several middlewares would be required to improve throughput and performance.
Solution/User Experience
Middlewares can leverage batch operations when consuming messages from their input SQS queue. We want to refactor some middlewares to drop the use of partial responses and leverage the batch of input messages.
Alternative solutions
No response