Can dali backend support default values or optional input?

[SunXuan90]

I want to use the same dali.py for preprocessing for different models, most of them need to resize input image to 640x640, only one is 320x320. It would be very convenient if there's a way to set 640x640 as default value, or use an optional input. Right now if I set optional to true, the model won't serialize and will throw an error:

I0316 05:27:27.697150 1474 dali_backend.cc:142] backend state is 'backend state'
terminate called after throwing an instance of 'triton::backend::BackendModelException'

The Triton image is use is nvcr.io/nvidia/tritonserver:23.01-py3, the config.pbtxt I use is as follows:

name: "constant"
backend: "dali"
dynamic_batching {}
max_batch_size: 256
input [
  {
    name: "DALI_INPUT"
    data_type: TYPE_FP32
    dims: [ 3, -1, -1 ]
  },
  {
    name: "OPTIONAL"
    data_type: TYPE_FP32
    dims: 1
    optional:true
  }
]

output [
  {
    name: "DALI_OUTPUT"
    data_type: TYPE_FP32
    dims: [ 3, -1, -1 ]
  },
  {
    name: "CONSTANT"
    data_type: TYPE_FP32
    dims: 1
  }
]

Thanks!

[szalpal]

Hi @SunXuan90 ,

unfortunately, using optional input won't help in this case. To work, DALI Backend has to serialize the model - in other words "freeze" it for some particular parameters. And these parameters include size argument for fn.resize operator.

What you could do is to use DALI conditional execution to pick proper code path during runtime. Your DALI pipeline might look something like this:

from nvidia.dali.pipeline import experimental

@autoserialize
@experimental.pipeline_def(enable_conditionals=True, batch_size=256, num_threads=1, device_id=0
def pipe():
    img = fn.external_source(device="cpu", name="DALI_INPUT")
    default_resize = fn.external_source(device="cpu", name="DEFAULT_RESIZE")
    if default_resize:
        img = fn.resize(data, size=(640, 640))
    else:
        img = fn.resize(data, size=(320, 320))
    return img

Please note, that the conditional execution in DALI is available from version 1.23, which corresponds to Triton version 23.03. This should be released by the end of March 2023. Should you like to test it before, you'd need to build "nightly" version of tritonserver and DALI, using this Dockerfile. Or just wait a week or two :)

I'm currently putting together and example of using DALI conditional execution via Triton. It will be merged around the time tritonserver:23.03 is released.

[SunXuan90]

Thank you, this is very helpful.