Pad input tensor data to handle requests with bs != max_batch_size

triton-inference-server / openvino_backend

OpenVINO backend for Triton.

BSD 3-Clause "New" or "Revised" License

29 stars 16 forks source link

Closed tanmayv25 closed 3 years ago

tanmayv25 commented 3 years ago

Its not as a padding in strict sense. we are just not filling the complete allocated memory for the input blobs.