Closed tkilias closed 5 years ago
Add batch-wise streaming for transformer and model inference. Enforce parallel inference with group by.
Add batch-wise streaming for transformer and model inference. Enforce parallel inference with group by.