JPEGs exported from DNGs are much darker than from the corresonding CR3s

[koslowj]

DxO's Pure Raw 2 produces rather nice DNGs that can be processed further in R&Darktable. However, when exporting to JPEG (or PNG), the results look much darker than exporting from the original CR3s (from my Canon R5), even though in R&Darktable both versions look quite similar (except for denoising and geometry corrections). Actually, I just copied the development stack from the DNG to the CR3.

This happens on a Gentoo Linux system, Kernel 5.15.59. I've compiled R&Darktable myself according to your instructions. I do not know if DNGs generated with other software exhibit the same problem.

Unfortunately, when trying to attach sample images, I get an error message: "Something went really wrong, and we can't process that file.", both on firefox and on Google Chrome.

[koslowj]

For comparison I've installed darktable 4.0.0, and the problem described above does not occur there. The resulting JPEGs are roughly twice the size of those produced by R&Darktable. For the time being I can keep darktable 4.0.0 just for exporting, but eventually I hope this issue will be fixed.

[ptilopteri]

why not use the native raw images in dt rather than an extra, superfluous step in converting to dng? and perhaps no more problem.

[koslowj]

The conversions by Pure Raw 2 are often a bit better than what darktable produces, in particular concerning noise reduction. Moreover, lens corrections are built in. This is no surprise, after all, Pure Raw 2 is a commercial product (I bought it for 80 Euros).

I got my Canon R5 in the Summer of 2021, after seriously starting with darktable in the Spring. (I was getting out of Lightroom and I'm still in the process of re-editing pictures since 2004...) At that time it was difficult to get darktable to digest the new Canon CR3 format. Initially I used Adobe's DNG-converter, but the results looked terrible. Then Rure Raw 2 showed up and got rave reviews. I tried it and was immediately convinced (it is slow though, can oversharpen pictures in certain situations, and its DNGs are about 3 times as big as the original raw files :-( ). The next darktable version then was able to process CR3s directly, with very good results, but not quite up there with Pure Raw 2.

If sufficiently many people provide lens data and noise patterns of their cameras, darktable can probably be tweaked to get even better results (I hope to do this eventually...)

I've since found out that the problem does not occur in darktable 4.0.0. Keeping that around just for exporting pictures can be a stop-gap solution, but I do like Aurelien's approach better.

Best regards,

-- Jürgen

-- If I don't see you no more on this world Juergen Koslowski I'll meet you on the next one @.**@.) and don't be late!Sent from Swiss-based encrypted ProtonMail, Jimi Hendrix (Voodoo Child, SR)

Sent with Proton Mail secure email.

------- Original Message ------- On Tuesday, October 4th, 2022 at 9:42 PM, ptilopteri @.***> wrote:

why not use the native raw images in dt rather than an extra, superfluous step in converting to dng? and perhaps no more problem.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[jorismak]

I've used DNGs exported by DxO Photolab for ages, in Darktable and in R&Darktable, and without issue.

But since the 'remove colorout' changes are done, something doesn't work nicely when exporting files with knowing which profile the output needs to be exported to.

So make a build where those commits are not included, or try forcing a specific output profile when exporting. It's somewhere in those areas where you need to look.

[koslowj]

Thanks for your suggestion, however, the cause for the problem was sitting behind the keyboard -- user error. It's rather embarrassing, but I had managed to somehow mess up the values of "Input color profile". Fixing those now allows me to export beautiful JPEGs directly from R-Darktable. No need for having Darktable 4.0.1 around anymore!

-- Jürgen

-- If I don't see you no more on this world Juergen Koslowski I'll meet you on the next one @.**@.) and don't be late!Sent from Swiss-based encrypted ProtonMail, Jimi Hendrix (Voodoo Child, SR)

Sent with Proton Mail secure email.

------- Original Message ------- On Monday, October 31st, 2022 at 11:37 AM, Joris Mak @.***> wrote:

I've used DNGs exported by DxO Photolab for ages, in Darktable and in R&Darktable, and without issue.

But since the 'remove colorout' changes are done, something doesn't work nicely when exporting files with knowing which profile the output needs to be exported to.

So make a build where those commits are not included, or try forcing a specific output profile when exporting. It's somewhere in those areas where you need to look.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[koslowj]

Small correction: my wrong settings did not occur in "input color profile" (although I played with those), but rather in the profile and intent of the export dialogue. The profile was set to "same as original", which didn't work. Instead, "Adobe RGB (compatible)" does work, as I have set my Canon R5 to Adobe RGB. Strangely enough, using "Adobe RGB (compatible)" as INPUT proflie produces very dark pictures within R-Darktable, resembling the exported output I originally complained about. I have to use "embedded matrix" instead. Could it be that what Canon claims to be Adobe RGB does not coincide with R-Darktable's definition of Adobe RGB?

-- Jürgen

@.*** https://protonmail.com

https://proton.me/

------- Original Message ------- On Saturday, November 12th, 2022 at 1:27 AM, Jürgen Koslowski @.***> wrote:

Thanks for your suggestion, however, the cause for the problem was sitting behind the keyboard -- user error. It's rather embarrassing, but I had managed to somehow mess up the values of "Input color profile". Fixing those now allows me to export beautiful JPEGs directly from R-Darktable. No need for having Darktable 4.0.1 around anymore!

-- Jürgen

-- If I don't see you no more on this world Juergen Koslowski I'll meet you on the next one @.**@.) and don't be late!Sent from Swiss-based encrypted ProtonMail, Jimi Hendrix (Voodoo Child, SR)

Sent with Proton Mail secure email.

------- Original Message ------- On Monday, October 31st, 2022 at 11:37 AM, Joris Mak @.***> wrote:

I've used DNGs exported by DxO Photolab for ages, in Darktable and in R&Darktable, and without issue.

But since the 'remove colorout' changes are done, something doesn't work nicely when exporting files with knowing which profile the output needs to be exported to.

So make a build where those commits are not included, or try forcing a specific output profile when exporting. It's somewhere in those areas where you need to look.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[aurelienpierre]

But since the 'remove colorout' changes are done, something doesn't work nicely when exporting files with knowing which profile the output needs to be exported to.

Colorout is still in the pipeline, but its settings get overridden by whatever color profile is set in the export module, so the difference now is it's only hidden from darkroom and controlled only from the export module.

[aurelienpierre]

The profile was set to "same as original", which didn't work. Instead, "Adobe RGB (compatible)" does work, as I have set my Canon R5 to Adobe RGB.

Ok, that's concerning. Please send a test DNG here or privately at contact [at] aurelienpierre.com if it's private pictures.

[aurelienpierre]

Strangely enough, using "Adobe RGB (compatible)" as INPUT proflie produces very dark pictures within R-Darktable, resembling the exported output I originally complained about.

That's not strange at all. Adobe RGB has a "gamma" built-in, that gets remove (linearized) in input color profile if you set Adobe RGB as the input color space. Most likely, your DNG is already is encoded linearly, so linearizing it actually darkens it. The real question here is why the embedded matrix that should be in your DNG doesn't get detected.

[magicgoose]

The conversions by Pure Raw 2 are often a bit better than what darktable produces, in particular concerning noise reduction. Moreover, lens corrections are built in.

1) Unfortunately darktable NR settings are too aggressive out of the box and require fine tuning, but I found at least for some cameras (only tested DT 4.2.0 with X-Trans sensor so far) it achieves really nice results in the following combination: First instance, wavelets auto, all other settings on defaults, blend mode = chromaticity. Second instance (starts with the output from the first), non-local means auto, reduce strength to taste, all other settings on defaults (default blend mode too).

2) For supported cameras it's auto-applied in darktable as well. The problem is, it may require manual configuration patching with some unlucky camera+lens combinations because the lens names don't match.

[aurelienpierre]

DNG input profile detection is broken for embedded profiles. Some CR2 vendor profiles too. ICC extraction from files needs to be rewritten.

[aurelienpierre]

Affects monochrom DNG from Leica monochrom cameras too: creates color fringes on sharp bright objects (note to self: see Ricky pic).

[aurelienpierre]

Should be fixed with c25d62fc44574e4b719a93d37f833ffd0f2914aa, please test and report.

[aurelienpierre]

Closing for project-management tidiness. DNG files from DxO Pure RAW are reported to work properly now. Re-open if they don't.

[koslowj]

Dear Aurelien,

I'm glad you are still improving Ansel, which I'm using almost daily. Is the a changelog to check what updates have been implemented since the previous version? I'm up to 85f2b8b, as I keep checking every couple of days, since there are still some problems, in particular when it comes to retouching, and the occasional segmentation fault.

Keep up the good work,

-- Jürgen

-- If I don't see you no more on this world Juergen Koslowski I'll meet you on the next one @.**@.)and don't be late! Sent from Swiss-based encrypted ProtonMail, (Jimi Hendrix, Voodoo Child, SR)

Sent with Proton Mail secure email.

On Saturday, July 13th, 2024 at 3:58 PM, Aurélien PIERRE @.***> wrote:

Closing for project-management tidiness. DNG files from DxO Pure RAW are reported to work properly now. Re-open if they don't.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>