improve decode JPEG 2000 data performance

[sawyerzhu]

Motivation

When parse grib2 file with 36x5011x7501 shape, the data encode with JPEG 2000, the performance is very slow. with one submessage 5011x7501 spent 10s, for parse all 36 submessage is almost need 6 min on my labtop.

Proposed Solution

Is there some optimization? Change openjpeg-sys to other jpeg 2000 library?

Additional Context

No response

[noritada]

Thank you very much for reporting.

As you say, there are performance issues with JPEG 2000 decoding. I have not examined it carefully, but I suspect that this is due to the fact that the C library is called inside, and the array obtained from the primary decoding by the C library is processed again as an iterator on the Rust side to obtain the final decoded result. In other words, the larger the array, the more it costs to temporarily hold the intermediate data.

The above is just my imagination, but it is possible that performance could be improved by replacing the JPEG 2000 decoder C implementation with a pure Rust implementation. At the time I added JPEG 2000 support, there was no pure Rust implementation, but it has been some time since then, so I would like to look into it again.

[sawyerzhu]

Thank for your responding.

I do some search, but not found libray implemented with pure Rust code, this is another wrapper for openjpeg-sys https://github.com/Neopallium/jpeg2k, It seems has some performance improvement.