search

osamu620 / OpenHTJ2K

An open source implementation of ITU-T Rec.814 | ISO 15444-15 (a.k.a. HTJ2K).
BSD 3-Clause "New" or "Revised" License
36 stars 10 forks source link

rate control #12

Open boxerab opened 3 years ago

boxerab commented 3 years ago

Hello! Very glad to see another HTJ2K project getting started. Do you plan on implementing rate control or new Q factor for HTJ2K ? Many thanks, Aaron

osamu620 commented 3 years ago

Hi @boxerab

Thanks for bringing a good point. The latter one is in my plan. It will require adding a feature to quantize images with the visual weighting technique. I think it won't take long but still needs some time to do. The rate control will be much complex for me but it is not excluded from my future plan.

boxerab commented 3 years ago

Thanks, @osamu620 . Looking forward to the visual weighting feature.

osamu620 commented 3 years ago

For the future plan for the rate control in the HT block encoding, I think this issue should remain.

boxerab commented 3 years ago

Hello @osamu620, I am interested in working on a rate control algorithm for HTJ2K. Do you know of any papers that may be relevant ? Apparently the only existing non-iterative precise rate control algorithm at the moment is inside Kakadu library.

osamu620 commented 3 years ago

@boxerab Hi, sorry for taking a long time to respond. As far as I know, there is no published paper on a rate-control algorithm for the HT block coder.

Apparently the only existing non-iterative precise rate control algorithm at the moment is inside Kakadu library.

Yes. But Kakadu's rate-control algorithm is patent-protected, I think. One way to implement a rate-control for HTJ2K is to produce multiple HT Cleanup passes with distortion estimation. It will require highly optimized (= using SIMD and parallelism) HT block encoding..

boxerab commented 3 years ago

@boxerab Hi, sorry for taking a long time to respond.

No worries, thanks for your reply.

Apparently the only existing non-iterative precise rate control algorithm at the moment is inside Kakadu library.

Yes. But Kakadu's rate-control algorithm is patent-protected, I think.

that makes sense

One way to implement a rate-control for HTJ2K is to produce multiple HT Cleanup passes with distortion estimation. It will require highly optimized (= using SIMD and parallelism) HT block encoding..

what about magref and sigprop passes - wouldn't you need those passes as well?

osamu620 commented 3 years ago

@boxerab

what about magref and sigprop passes - wouldn't you need those passes as well?

I would say the generation of HT MagRef and HT SigProp is not mandatory. The bit-position of those two passes depends on the number of bits to be accommodated in HT Cleanup pass. HT SigProp and MagRef passes can represent only a single bit-plane, therefore, the contribution to image quality might be relatively small.

I think generating multiple HT Cleanup passes is critical for the throughput of HTJ2K encoding pipeline.

boxerab commented 3 years ago

Thanks - is it difficult to run multiple HT cleanup passes with the current state of the codec ?

osamu620 commented 3 years ago

@boxerab I think multiple HT Cleanup passes will lead longer time for compression. This is the reason why HT Cleanup pass encoding should be optimized.

boxerab commented 3 years ago

I completely agree. Let me know if I can help with optimization. It wouldn't be too hard to add a thread pool to the block coding logic

osamu620 commented 3 years ago

I completely agree. Let me know if I can help with optimization. It wouldn't be too hard to add a thread pool to the block coding logic

I'm not familiar with thread pool implementations, so your help would be highly appreciated.

One additional thing: Actually, the current code takes a certain amount of time for memory allocation and move. Optimization for the allocation might come first.

boxerab commented 3 years ago

@osamu620 sorry for the delay - I will create a PR with a thread pool for the block coder. The implementation I currently use requires C++17 - is that ok for you? Older compilers wouldn't be able to compile the project, for example GCC 6.

Improving memory allocation will certainly improve performance, but I think we get the biggest increase from thread pool and SIMD.

osamu620 commented 3 years ago

@boxerab Thanks. About C++17, I think nothing prevents us to use it!

aous72 commented 3 years ago

I am here to put in my 2 cents.

Yes. But Kakadu's rate-control algorithm is patent-protected, I think.

Kakadu having a patent on the method does not exclude you from using it. I, and I can NOT talk for Kakadu, think that you should be fine if you used it in your open source JPEG2000 library. In the past, Kakadu has been generous to other parties using its patents; for example, the HT algorithm is available free of charge, even though it is patented. In any case, you can contact Prof. David Taubman directly regarding this matter.

Osamu and I can suggest ways of doing this.

Cheers, Aous.

osamu620 commented 3 years ago

@aous72 Thanks for your detailed explanation. It is valuable insight rather than two cents.

boxerab commented 3 years ago

@aous72 thanks very much for your comments. I hope there are many good solutions to the HTJ2K rate control issue.