search

NVIDIA / DALI

A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.
https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html
Apache License 2.0
5.05k stars 615 forks source link

Support for JPEG-XL as a codec #3888

Open RaananHadar opened 2 years ago

RaananHadar commented 2 years ago

JPEG-XL is next generation image compression format. It already has support in chrome, firefox and official support in python libraries such as GDAL. It has an open source cpu implementation.

The biggest benefit for adding JPEG-XL support to DALI is providing a modern alternative to JPEG-2000. JPEG-2000 is a semi legacy codec (coined originally in 2000) which has many applications and is very popular in large satellite images. Current user experience with DALI and JPEG2000 is not good. Not because of DALI but because of JPEG2000 dependencies, let me explain:

Being a relatively prorpietary codec, JPEG2000 decompression is very slow on OPENJPEG, which both DALI and GDAL depend upon. Also, as there are many JPEG2000 implementations in practice, decoding can be unstable and i've had an issue where OPENJPEG can crash leading DALI to crash while OPENJPEG or the JPEG2k encoder implementation is the culprit (as these are often legacy, they are extremely hard to change). Bottom line: Today, many serious JPEG2000 users probably need to buy a proprietary CPU codec (kakadu) or suffer. JPEG2000 with OPENJPEG and DALI is pretty on par as a proprietary library such as kakadu performance wise (kakadu is CPU), and as kakadu is very stable (and I need to stress this, stability is a big issue), this prevents more JPEG2000 users using DALI.

As JPEG-XL is open source it can really improve the user experience and provide a modern alternative. Added support for Tiff Files with JPEG-XL compression would make DALI very useful for AI applications with aerial and satellite imagery.

Please do consider 🙏

jantonguirao commented 2 years ago

Thank you @RaananHadar for your request.

I think your request makes sense. Satellite imagery is something we have in our radar for some time.

To help us prioritizing JPEG-XL support, we would like to understand how and how often it is used in real applications. Do you have any reference models/applications where such format is used? A pointer to some well-known datasets would be useful as well.

Regarding OpenJPEG and stability, could you elaborate on that? Are there any known issues, features not supported, etc? Also, DALI's 'mixed' backend decoder uses nvJPEG2000 library under the hood (instead of OpenJPEG). Have you tried that approach? Are there any issues with it? If so, we'd also like to learn about it.