search

NVIDIA / DALI

A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.
https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html
Apache License 2.0
5.06k stars 615 forks source link

Feature request: support for EXR images #3061

Open thierrypin opened 3 years ago

thierrypin commented 3 years ago

I'm interested in working with HDR images, and noticed there is no native support for this type of image on DALI. I would prefer to avoid the performance loss of using a custom reader.

JanuszL commented 3 years ago

Hi @thierrypin,

Currently, we don't have this format on our roadmap. We can get this next year at the earliest. Still, we would love to see a community contribution adding support for this format. Until then you can still try using the external source operator. Please check parallel and prefetch_queue_depth to increase the performance.

thierrypin commented 3 years ago

@JanuszL ,

Thank you for the speedy response. I know of external source, but it makes the loader considerably slower; I was thinking about making my custom reader. There is something about it in the documentation, but I still haven't read it carefully. Do you have any tips?

JanuszL commented 3 years ago

Hi @thierrypin,

Yes, it would be slower but now the external source can run in parallel and overlap with other operations. So maybe it will be fast enough in your case. Regarding the development of a custom solution, I think it would be easiest to create a dedicated decoder, not the reader itself. You can refer to the custom operator development guide, and see how the CPU image decoder is implemented:

The main difference compared to the current implementation is the output type - DALI uses uint8, while EXR supports float16, float32, and int32.

thierrypin commented 3 years ago

Thank you @JanuszL, I'll study about it.