search

triton-inference-server / fil_backend

FIL backend for the Triton Inference Server
Apache License 2.0
71 stars 36 forks source link

Remove batch size preferences #116

Closed wphicks closed 3 years ago

wphicks commented 3 years ago

Given that FIL does not perform substantially better with specific batch sizes, remove batch size recommendations and default to a shorter queuing window for batch accumulation.

Close #111