Added server query coalescing

krai / axs2mlperf

Automated KRAI X workflows for reproducing MLPerf Inference submissions

https://krai.ai

MIT License

1 stars 1 forks source link

Closed Akshat-Tripathi closed 7 months ago

Akshat-Tripathi commented 7 months ago

Added an option to enable query coalescing for the server mode. I've turned it on by default, since I don't see why this could ever hurt