search

triton-inference-server / fil_backend

FIL backend for the Triton Inference Server
Apache License 2.0
68 stars 35 forks source link

[FEA] Support max_batch_size 0 #337

Open wphicks opened 1 year ago

wphicks commented 1 year ago

Allow user to turn off batch dimension entirely.