search

triton-inference-server / dali_backend

The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.
https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html
MIT License
118 stars 28 forks source link

Add `release_after_unload` backend option #225

Closed szalpal closed 5 months ago

szalpal commented 5 months ago

Per requests included in #165 , this PR adds a possibility to release memory used by DALI Backend after unloading the DALI model.

To do so, user shall pass proper option to tritonserver binary invocation:

tritonserver --backend-config=dali,release_after_unload=true
dali-automaton commented 5 months ago

CI MESSAGE: [12465440]: BUILD STARTED

dali-automaton commented 5 months ago

CI MESSAGE: [12465440]: BUILD PASSED

szalpal commented 5 months ago

Helpful information for the review:

  1. Most of the code is for the test.
  2. The test uses command-line nvidia-smi way to determine used memory, as from our experience it's the most reliable way.
  3. The test is "duplicated": release_after_unload and dont_release_after_unload. The duplication stems from the CI design - it's the easiest this way.
dali-automaton commented 5 months ago

CI MESSAGE: [12469440]: BUILD STARTED

dali-automaton commented 5 months ago

CI MESSAGE: [12469440]: BUILD FAILED

dali-automaton commented 5 months ago

CI MESSAGE: [12475485]: BUILD STARTED

dali-automaton commented 5 months ago

CI MESSAGE: [12475706]: BUILD STARTED

dali-automaton commented 5 months ago

CI MESSAGE: [12476098]: BUILD STARTED

dali-automaton commented 5 months ago

CI MESSAGE: [12475706]: BUILD FAILED

dali-automaton commented 5 months ago

CI MESSAGE: [12476098]: BUILD FAILED

dali-automaton commented 5 months ago

CI MESSAGE: [12482821]: BUILD STARTED

dali-automaton commented 5 months ago

CI MESSAGE: [12482821]: BUILD FAILED

dali-automaton commented 5 months ago

CI MESSAGE: [12482821]: BUILD PASSED