search

Beep6581 / RawTherapee

A powerful cross-platform raw photo processing program
https://rawtherapee.com
GNU General Public License v3.0
2.81k stars 317 forks source link

Add support for new compressed CLUTs #5228

Open Beep6581 opened 5 years ago

Beep6581 commented 5 years ago

@dtschump created a new, highly efficient compression mechanism for HaldCLUTs, capable of storing all our 500 CLUTs (~500MB) in a single ~2MB file. This issue is to get support into RawTherapee.

A name or description for each CLUT could be stored in the metadata, e.g. in a JSON structure.

sguyader commented 5 years ago

That would be nice. I have a question though: if we want to add our own HaldCLUT, how will it be added to the compressed storage file? Would the added feature in RT do this?

heckflosse commented 5 years ago

Though I personally don't mind wasting 0.5 GB of disk storage for the LaldCLUTs, getting them by a 2 MB download clearly is better in case the result of applying them is on par with the uncompressed ones. As soon as @dtschump points us to the decompressing code I will have a look.

agriggio commented 5 years ago

this could be very interesting if applicable to luts inside dcp profiles IMHO. reducing their size would have a positive impact on the size of the official RT packages in the long run

Floessie commented 5 years ago

I have a question though: if we want to add our own HaldCLUT, how will it be added to the compressed storage file? Would the added feature in RT do this?

I'd say the content of the classical HaldCLUT folder takes precedence over the supplied compressed ones and overrides those with the same names and otherwise adds to the selection structure. So there would be no need to add compressed CLUTs for the user (or us), as compressing CLUTs is computationally intensive.

dtschump commented 5 years ago

I'll try to clean the c++ code today, using the .png and .txt files I've just sent on Pixls.us. Will let you know when it is ready.

Beep6581 commented 5 years ago

@Floessie perhaps you would like to elaborate what you mean by "computationally intensive"? :smile:

dtschump commented 5 years ago

@Floessie is right actually, compression the 552 CLUTs in G'MIC took something like 3.5 days of computation on 4 different computers, with at least 30 CPUs each. In any case, if you want to add new CLUTs to RT in the future, the compression algorithm can be run directly from G'MIC (command compress_clut).

Floessie commented 5 years ago

Thanks David. That's what I was alluding to...

dtschump commented 5 years ago

I've set up a GIT repository with the C++ source code for CLUT decompression, as well as the CLUT data. You can get it here: https://framagit.org/dtschump/libclut

Let me know if you need more information or if you have issues with this code.

heckflosse commented 5 years ago

@dtschump David, I will study the decompression code and try to make it even faster ;-)