search

darktable-org / darktable

darktable is an open source photography workflow application and raw developer
https://www.darktable.org
GNU General Public License v3.0
9.91k stars 1.15k forks source link

3D LUT: Some haldclut pngs not working with darktable 3.0 while opencl is activated #3299

Closed herbert-50 closed 5 years ago

herbert-50 commented 5 years ago

Some of the pngs of the haldclut package are not working right. In the preview window of the darkroom the result look good. But in the main window and in the exported jpg the result looks very bad (no contrast).

To Reproduce Steps to reproduce the behavior:

  1. Load the haldclut pack from http://rawtherapee.com/shared/HaldCLUT.zip and unzip it
  2. Enable opencl. Without opencl the problem does not occur.
  3. Set opencl profile to "standard" -> for the preview window opencl is not used
  4. Set the Path of the haldclut files in the settings of darktable
  5. Load an image
  6. Open the 3D LUT Module and load the file HaldCLUT/Black-and-White/Fuji/Fuji Neopan 1600 2.png

Compare at the preview window with the main window in the darkroom view.

Here is a small list of pngs i've tested: Fuji Neopan 1600 2.png - works not Fuji Neopan Acros 100.png - works Fuji FP-3000b 1 --.png - works Fuji FP-3000b HC.png - works not Fuji 160C 1 -.png - works Fuji Superia 800 2.png - works not

Platform (please complete the following information):

grafik

junkyardsparkle commented 5 years ago

That zip file is 400MB... could you possibly upload a single haldclut which exhibits this behavior to make it easier for others to try to reproduce? :-)

(Also, I think your listed darktable version is wrong. AFAIK rc1 doesn't exist yet.)

flannelhead commented 5 years ago

Reproduced this on Linux/i915 (already had the CLUTs in question downloaded) so at least doesn't seem to be AMD specific.

chhil commented 5 years ago

You are right about it being opencl related. I don't think it's related to a specific LUT. I saw this happen yesterday on a pana cine to rec lut where the main image looked very different from the one in preview.Turned off and restarted to restore it.

phweyland commented 5 years ago

Turned off and restarted to restore it.

What do you mean? Just turn off lut3d module and enable it again?

chhil commented 5 years ago

Turned off and restarted to restore it.

What do you mean? Just turn off lut3d module and enable it again?

Turn off opencl.

herbert-50 commented 5 years ago

Here is the Haltcut.png: Fuji Neopan 1600 2

Yes - Version rc1 is wrong. Its rc0 (see also screenshot)!

phweyland commented 5 years ago

Does that happen only with back & white clut or with color clut also ?

flannelhead commented 5 years ago

Also colorful ones, for me at least

phweyland commented 5 years ago

I can reproduce the issue too, at least for some fuji black & white clut. I've noted that when I switch dt from full screen to resizable window, or when I switch darkroom from 100% to fit to screen, I get normal the right image (or something which looks right) half a second or a second before the wrong image is back (kind of glitch). Do you have this effect too ?

junkyardsparkle commented 5 years ago

I only tried this quickly with one image (and the haldclut supplied by OP) and saw the difference between preview and main image immediately. After changing the 'application color space' values in the dropdown, it seems that the images are in sync with the two linear colorspaces, but changing to REC 709 or sRGB changes only the preview, not the main image... so it's probably related to a mismatch in working color spaces somewhere, I guess.

junkyardsparkle commented 5 years ago

This wasn't happening with a clean profile for me, with its default setting of 'OpenCL scheduling profile' ('very fast GPU'), but it happens when I switch that setting to 'default' (what I use normally). @herbert-50 My graphics card is similar to yours, could you try changing that setting under 'core options' in preferences and see if it has any effect? This changes the way resources are used between the preview and main view.

herbert-50 commented 5 years ago

I've tried the setting 'very fast GPU'. In this case the picture looks bad in the preview window too.

If I use the setting 'multiply GPUs' and shift the darktable window between my displays (they have different color-profiles, so darktable has to redraw the preview and the main view), the result differs. Sometime the main view looks good and sometimes the preview looks good and sometimes both.

If I use 'defaut' the preview looks good and the main window looks bad.

I think the result depends on the device, which darktable uses to calculate. If the CPU is used, because the GPU is looked, it's ok. But if the GPU is used, it's bad.

With 'darktable -d opencl' I can see, which device is used. If device 0 (GPU) is used for a window, the result is bad.

Perhaps the 3d lut code for the GPU is different from the CPU Code and has a bug!?

junkyardsparkle commented 5 years ago

Perhaps the 3d lut code for the GPU is different from the CPU Code and has a bug!?

Yes, specifically the OpenCL code only seems to use the linear colorspaces, but never the gamma ones.

junkyardsparkle commented 5 years ago

So... it looks like changing the working profile to a matching one in the input color profile module mitigates this. I don't know if that's the intended workflow or not (I've never touched the lut 3D module before today), but it's a workaround for now, at least.

EDIT: maybe it doesn't actually... the interraction with the working profile is more complicated than I thought... I give up. :-)

EDIT 2: Taking another look now, I think I was right the first time... and testing with a no-op identity haldclut doesn't show the issues, so I think the reason that some LUTs were showing more than others probably was a matter of how "strong" of an effect they had, since this effect was effectively being distorted by being applied in the wrong color space.

junkyardsparkle commented 5 years ago

I've noted that when I switch dt from full screen to resizable window, or when I switch darkroom from 100% to fit to screen, I get normal the right image (or something which looks right) half a second or a second before the wrong image is back (kind of glitch). Do you have this effect too ?

@phweyland Yes, I see this too... so I guess the cache is being correctly generated in any case.

phweyland commented 5 years ago

testing with a no-op identity haldclut doesn't show the issues

It seems to me that, in some cases, the switch from rec2020 (working profile) to lut profile doesn't happen. In those cases, the lut transform is done on rec2020.

Trying to compile dt backward to find the breaking point (on 20th October that was ok).

phweyland commented 5 years ago

@aurelienpierre, Hi Aurélien, I think we need some help here. This issue occurs on commit b8e90ea while it is not present 2 commits before

These 2 commits seem to be important changes about opencl standards.

As far as I can understand the lut3d issue seems to be related to that the color space conversion is not always correct (instead of switching to sRGB or REC.709, the switch is done to lin sRGB/REC.709). lut3d uses the routine dt_ioppr_transform_image_colorspace_rgb_cl(). It seems not to return any error however.

Do you have any suggestion / explanation which could help solving that ? Thanks

TurboGit commented 5 years ago

@phweyland : PR #3316 fix this issue. Can you try and see if there is no other regression? thanks.

phweyland commented 5 years ago

@TurboGit, I've fetch your PR (#3316). Everything seems ok and I haven't seen any other issue. Thanks

herbert-50 commented 5 years ago

Yes - this works fine :1st_place_medal:

Thanks @all.

aurelienpierre commented 5 years ago

@aurelienpierre, Hi Aurélien, I think we need some help here. This issue occurs on commit b8e90ea while it is not present 2 commits before

These 2 commits seem to be important changes about opencl standards.

As far as I can understand the lut3d issue seems to be related to that the color space conversion is not always correct (instead of switching to sRGB or REC.709, the switch is done to lin sRGB/REC.709). lut3d uses the routine dt_ioppr_transform_image_colorspace_rgb_cl(). It seems not to return any error however.

Do you have any suggestion / explanation which could help solving that ? Thanks

I commented the TRC related lines to narrow the search while looking for the origin of the black output bug and forgot to restore them later. Sorry about that and thanks @TurboGit