[Q] Option to choose DNG compression?

[LebedevRI]

Hi from darktable.

We were asked about an issue of hdrmerge-produced dng's not being able to be loaded by darktable several times, and IIRC we always said that it is the deflate compression is the problem, uncompressed / JPEG-compressed 32-bit FP DNG's are supported, so just don't compress them for now. And as i have just found out, hdrmerge actually can only write compressed DNG... No one corrected us...

So while the issue of zlib-compression support in rawspeed still stands, what do you think about adding a configuration parameter to optionally produce uncompressed DNG's?

Looking at the code, i believe, i would be able to produce a patch for that.

[heckflosse]

@LebedevRI : :+1: if you produce a patch!!!

[heckflosse]

@LebedevRI : Sorry for short answer in last post. Now a bit more detailed. You're right that HDRMerge currently only writes compressed DNG files. A patch (or PR) writing uncompressed files would be very welcome. But I won't be able to test the output of the patch as I'm using Rt which does not read uncompressed float DNG files. For this reason I would appreciate that another DT dev (houz, hanatos, whoever) will test your pr before merging it.

Ingo

[LebedevRI]

Hm, the hdrmerge part works. But now i have to ask: why does the hdrmerge produce tiled dng's? Apparently rawspeed does not support tiled uncompressed dngs, so if hdrmerge dng's follow specification, i'd like to use the response to justify adding tiling uncompressed support to rawspeed...

[jcelaya]

No real reason. The code is inspired in the Adobe DNG SDK, which (if I remember well) always produces tiled dngs.

[LebedevRI]

@jcelaya Looking into this again... How exactly does hdrmerge create float dng's? I see that it writes whitePoint: 15438 into exif. Does that mean that the actual white point of that dng is 15438.0f? (well, and max(data) ~= 16384.0f ?) Or is the float data normalized (as it should be) so that whitePoint == 1.0f ?

[heckflosse]

@LebedevRI Where does hdrmerge write 15438 as white point into exif? Please point us to the line of source which does that!

[LebedevRI]

https://github.com/jcelaya/hdrmerge/blob/8bd6e7f199ec3e55206715ef5ca44aee5df5a983/DngFloatWriter.cpp#L266 (Of course, i did not mean it writes the same const 15438 into every dng)

[heckflosse]

@LebedevRI The float data is not normalized to 1.0. It is in the range of the input data. The WhiteLevel written to the dng file is the one hdrmerge gets from libraw as are the black levels.

WhiteLevel has integral type, means normalizing it to 1.0 would end up in 1 being the only usable WhiteLevel which doesn't make sense. The Custom White Level you can specify is only used for internal calculations and not written to the dng.

[jcelaya]

Well, data could be normalized so that blackpoint == 0.0f and whitepoint == 1.0f, but the problem is that most dcraw-based loaders (like Rawtherapee) assume some parameters based on the maker-model combination, instead of reading the DNG tags. So I keep the black and white points from the source images, and scale the merged data to that interval. In your case, 15438 was either the maximum among the source data or the white point read from the source raw images.

Javi

2016-10-24 9:01 GMT+02:00 Roman Lebedev notifications@github.com:

https://github.com/jcelaya/hdrmerge/blob/8bd6e7f199ec3e55206715ef5ca44a ee5df5a983/DngFloatWriter.cpp#L266 (Of course, i did not mean it writes const 15438 into every dng)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jcelaya/hdrmerge/issues/86#issuecomment-255662683, or mute the thread https://github.com/notifications/unsubscribe-auth/AB9t_m7TU6eyon4g_24IzRbdfkA96IzPks5q3Fe2gaJpZM4JqV7T .

[heckflosse]

@jcelaya

most dcraw-based loaders (like Rawtherapee) assume some parameters based on the maker-model combination, instead of reading the DNG tags

which is good, because white levels and black levels provided by libraw sometimes are far from being optimal...

For that reason (wrong white levels from libraw) I implemented the custom white level feature for hdrmerge.

Ingo

[LebedevRI]

JFYI the darktable git master as of right now finally supports 32/24/16 bit unnormalized fp DNG with predictor + deflate. I don't like that they are not normalized, but the dng spec is quite broken in this regard. I would still like to have some more settings over the output DNG, but that would be a pr. So this issue can be closed.

[heckflosse]

@LebedevRI :+1:

[DavidOliver]

Many thanks, @LebedevRI and @heckflosse.

[LebedevRI]

and let's not forget @anarsoul for doing the rawspeed part https://github.com/darktable-org/rawspeed/pull/17