search

NVIDIA / DALI

A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.
https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html
Apache License 2.0
5.13k stars 619 forks source link

Unable to read avi files #3044

Closed psandev closed 3 years ago

psandev commented 3 years ago

Hi, some of my dateset consists of avi files, which unfortunately cannot be read with the latest binaries. As I understand from #1169 dali needs to be expicitly built for avi support, but the building procedure is not very clear. Could you , please, post a clear build procedure for compiling dali with gpu support and reading other formats: mov, avi, matroska, etc.

Thank you very much.

Dali: 1.2.0 Cuda Toolkit 11.1

File "/home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/pipeline.py", line 660, in build self._pipe.Build(self._names_and_devices) RuntimeError: Critical error when building pipeline: Error when constructing operator: readersVideoResize encountered: [/opt/dali/dali/operators/reader/loader/video_loader.cc:440] Unhandled codec 7 in /media/peter/SSD500GB/Kinetics400_1/train/farPlayerServeHit/4942863_9940.avi Stacktrace (27 entries): [frame 0]: /home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/libdali_operators.so(+0x4ccfde) [0x7f1d6858dfde] [frame 1]: /home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/libdali_operators.so(+0x26a8af4) [0x7f1d6a769af4] [frame 2]: /home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/libdali_operators.so(+0x26b02a3) [0x7f1d6a7712a3] [frame 3]: /home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/libdali_operators.so(+0x26edb8d) [0x7f1d6a7aeb8d] [frame 4]: /home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/libdali_operators.so(+0x26f143c) [0x7f1d6a7b243c] [frame 5]: /home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/libdali_operators.so(std::_Function_handler<std::unique_ptr<dali::OperatorBase, std::default_delete > (dali::OpSpec const&), std::unique_ptr<dali::OperatorBase, std::default_delete > (*)(dali::OpSpec const&)>::_M_invoke(std::_Any_data const&, dali::OpSpec const&)+0xc) [0x7f1d68589fec] [frame 6]: /home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/libdali.so(+0x151561) [0x7f1d675df561] [frame 7]: /home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/libdali.so(dali::InstantiateOperator(dali::OpSpec const&)+0x264) [0x7f1d675dc3b4] [frame 8]: /home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/libdali.so(dali::OpGraph::InstantiateOperators()+0xa2) [0x7f1d6758d672] [frame 9]: /home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/libdali.so(dali::Pipeline::Build(std::vector<std::pair<std::string, std::string>, std::allocator<std::pair<std::string, std::string> > >)+0x9f0) [0x7f1d675fd070] [frame 10]: /home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/backend_impl.cpython-38-x86_64-linux-gnu.so(+0x4b49f) [0x7f1d7be3849f] [frame 11]: /home/peter/virtualenvs/dl_387/lib/python3.8/site-packages/nvidia/dali/backend_impl.cpython-38-x86_64-linux-gnu.so(+0x889ee) [0x7f1d7be759ee] [frame 12]: /home/peter/virtualenvs/dl_387/bin/python(PyCFunction_Call+0x10a) [0x560c1c72d5fa] [frame 13]: /home/peter/virtualenvs/dl_387/bin/python(_PyObject_MakeTpCall+0x91) [0x560c1c729dd1] [frame 14]: /home/peter/virtualenvs/dl_387/bin/python(+0x23bc6d) [0x560c1c8eec6d] [frame 15]: /home/peter/virtualenvs/dl_387/bin/python(_PyEval_EvalFrameDefault+0x7849) [0x560c1c719649] [frame 16]: /home/peter/virtualenvs/dl_387/bin/python(_PyEval_EvalCodeWithName+0xb60) [0x560c1c7ee380] [frame 17]: /home/peter/virtualenvs/dl_387/bin/python(_PyFunction_Vectorcall+0x92) [0x560c1c72a322] [frame 18]: /home/peter/virtualenvs/dl_387/bin/python(_PyEval_EvalFrameDefault+0x5762) [0x560c1c717562] [frame 19]: /home/peter/virtualenvs/dl_387/bin/python(_PyEval_EvalCodeWithName+0xb60) [0x560c1c7ee380] [frame 20]: /home/peter/virtualenvs/dl_387/bin/python(PyEval_EvalCode+0x23) [0x560c1c7ee423] [frame 21]: /home/peter/virtualenvs/dl_387/bin/python(+0x183841) [0x560c1c836841] [frame 22]: /home/peter/virtualenvs/dl_387/bin/python(PyRun_SimpleFileExFlags+0x167) [0x560c1c83ae37] [frame 23]: /home/peter/virtualenvs/dl_387/bin/python(Py_RunMain+0x7b0) [0x560c1c71c460] [frame 24]: /home/peter/virtualenvs/dl_387/bin/python(Py_BytesMain+0x56) [0x560c1c71c996] [frame 25]: /lib/x86_64-linux-gnu/libc.so.6(libc_start_main+0xe7) [0x7f1e91dc5bf7] [frame 26]: /home/peter/virtualenvs/dl_387/bin/python(_start+0x2a) [0x560c1c71b60a]

Current pipeline object is no longer valid.

psandev commented 3 years ago

4942863_9940.zip

JanuszL commented 3 years ago

Hi @psandev,

AVI is the container, the video stream itself is MJPEG which is not supported. It is not supported in DALI yet, https://github.com/NVIDIA/DALI/pull/3045 should enable it. Please check the nightly/weekly build that follows the merge of this commit or release 1.4 that is the most likely to pick this change.

psandev commented 3 years ago

Thanks a lot. Looking forward to the merge. Release 1.4 will probably come in a few months.

JanuszL commented 3 years ago

Hi @psandev,

Release 1.4 will probably come in a few months.

We do our best to keep a monthly release cadence. So 1.3 in June, 1.4 most likely in July.