search

triton-inference-server / dali_backend

The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.
https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html
MIT License
120 stars 28 forks source link

Support video input with data_id mechanism #173

Closed banasraf closed 1 year ago

banasraf commented 1 year ago

This PR adds support for data_id mechanism enabling decoupled models for video input.

Decoupled model in Triton is a model that can send 0...n responses for each request (as opposed to standard models which have 1 to 1 response-request relationship). In case of DALI it lets us utilize video input by sending multiple responses for each sent video file. This is achieved by using the data_id mechanism - it tells us when the video file has been consumed, so we know when to stop processing the current request and accept the next one.

The most important changes are those in dali_executor/** files. The clue of this feature is to set the data_id for each request and then correctly determine if the input was consumed.

dali-automaton commented 1 year ago

CI MESSAGE: [7323475]: BUILD STARTED

dali-automaton commented 1 year ago

CI MESSAGE: [7323475]: BUILD FAILED

dali-automaton commented 1 year ago

CI MESSAGE: [7331839]: BUILD STARTED

dali-automaton commented 1 year ago

CI MESSAGE: [7332259]: BUILD STARTED

dali-automaton commented 1 year ago

CI MESSAGE: [7333806]: BUILD STARTED

dali-automaton commented 1 year ago

CI MESSAGE: [7332259]: BUILD PASSED

dali-automaton commented 1 year ago

CI MESSAGE: [7333806]: BUILD PASSED

dali-automaton commented 1 year ago

CI MESSAGE: [7368332]: BUILD STARTED

dali-automaton commented 1 year ago

CI MESSAGE: [7368332]: BUILD PASSED

dali-automaton commented 1 year ago

CI MESSAGE: [7438182]: BUILD STARTED

dali-automaton commented 1 year ago

CI MESSAGE: [7438182]: BUILD PASSED