Unable to initialize bit stream filter with fn.experimental.inputs.video

[runitao]

Version

1.42.0

Describe the bug.

I use fn.experimental.inputs.video to decode a video, it can't initialize bit stream filter. With fn.readers.video, while it can works.

the video sample file stream_filter_error.mp4

PS. I test master branch yet, the bug is still here.

Minimum reproducible example

#!/usr/bin/env python
#encoding: utf8

import sys

import numpy as np

from nvidia.dali import pipeline_def
import nvidia.dali.fn as fn
import nvidia.dali.types as types

@pipeline_def
def reader_pipeline(filenames,
                    pad_sequences=True,
                    sequence_length=30,
                    device='gpu',
                    name='video_reader'):
  video = fn.readers.video(device=device,
                           filenames=filenames,
                           sequence_length=sequence_length,
                           random_shuffle=False,
                           image_type=types.RGB,
                           dtype=types.FLOAT,
                           pad_sequences=pad_sequences,
                           file_list_include_preceding_frame=True,
                           name=name)

  return video

def reader_decode(filename):
  pipe = reader_pipeline(filenames=[filename],
                         pad_sequences=True,
                         batch_size=1,
                         num_threads=1,
                         device_id=0)
  pipe.build()
  for i in range(20):
    (video,) = pipe.run()

@pipeline_def
def input_pipeline(sequence_length=30, device='gpu', name='video_reader'):
  output = fn.experimental.inputs.video(device=device,
                                        sequence_length=sequence_length,
                                        name=name)
  return output

def input_decode(filename):
  pipe = input_pipeline(batch_size=1,
                        num_threads=1,
                        device_id=0,
                        device='mixed')
  pipe.build()

  pipe.feed_input('video_reader',
                  np.array([[np.fromfile(filename, dtype=np.uint8)]]))
  for i in range(20):
    (video,) = pipe.run()

if __name__ == '__main__':
  filename = sys.argv[1] if len(sys.argv) > 1 else 'stream_filter_error.mp4'

  print('decode with readers.video ...')
  reader_decode(filename)
  print('decode with readers.video OK')

  print('\ndecode with inptus.video ...')
  input_decode(filename)
  print('decode with inptus.video OK')

Relevant log output

[h264_mp4toannexb @ 0x7fcc88008100] Codec 'aac' (86018) is not supported by the bitstream filter 'h264_mp4toannexb'. Supported codecs are: h264 (27) 
Traceback (most recent call last):
  File "dali_stream_filter.py", line 72, in <module>
    input_decode(filename)
  File "dali_stream_filter.py", line 61, in input_decode
    (video,) = pipe.run()
  File "/opt/conda/lib/python3.8/site-packages/nvidia/dali/pipeline.py", line 1330, in run
    return self.outputs()
  File "/opt/conda/lib/python3.8/site-packages/nvidia/dali/pipeline.py", line 1168, in outputs
    return self._outputs()
  File "/opt/conda/lib/python3.8/site-packages/nvidia/dali/pipeline.py", line 1253, in _outputs
    return self._pipe.Outputs()
RuntimeError: Critical error in pipeline:
Error in MIXED operator `nvidia.dali.fn.experimental.inputs.video`,
which was used in the pipeline definition with the following traceback:

  File "dali_stream_filter.py", line 45, in input_pipeline
    output = fn.experimental.inputs.video(device=device,

encountered:

Assert on "av_bsf_init(bsfc_) >= 0" failed: Unable to initialize bit stream filter
C++ context: [/opt/dali/dali/operators/reader/loader/video/frames_decoder_gpu.cc:336] 
Current pipeline object is no longer valid.

Other/Misc.

No response

Check for duplicates

• [x] I have searched the open bugs/issues and have found no duplicates for this bug report

[JanuszL]

Hi @runitao,

Thank you for reaching out and for the complete repro. I can confirm it is a bug. I will get back to you soon once I learn more.

[JanuszL]

https://github.com/NVIDIA/DALI/pull/5682 should provide a proper fix.