darktable-org / darktable

darktable is an open source photography workflow application and raw developer
https://www.darktable.org
GNU General Public License v3.0
9.71k stars 1.14k forks source link

Zooming out produces black artefacts #15416

Closed kofa73 closed 12 months ago

kofa73 commented 1 year ago

Describe the bug

When using ctrl+scroll to zoom out, if rotate and perspective is used with cropping, black areas show up.

Steps to reproduce

  1. Get the image and sidecar from https://tech.kovacs-telekes.org/files/2023-10-14-zoom-out/
  2. Zoom in/out using Ctrl+wheel. For maximum effect, zoom in strongly, move the preview area to the edge of the image, then zoom in.
  3. Observe the artefacts.

Expected behavior

Show no artefacts.

Logfile | Screenshot | Screencast

https://tech.kovacs-telekes.org/files/2023-10-14-zoom-out/capture.mp4

Commit

No response

Where did you install darktable from?

self compiled

darktable version

4.5.0+908~g84906440ed

What OS are you using?

Linux

What is the version of your OS?

Ubuntu 23.04

Describe your system?

X11, Nvidia

Are you using OpenCL GPU in darktable?

Yes

If yes, what is the GPU card and driver?

NVidia 1060/6GB, 525.125.06

Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip

Same behaviour with OpenCL on and off. With OpenCL off, I get a huge log on the console, starting with:

157.1727 [dt_interpolation_compute_pixel4c] problem at (607,252) in 25fx348  1157.1728 [dt_interpolation_compute_pixel4c] problem at (608,252) in 25fx348  1157.1728 [dt_interpolation_compute_pixel4c] problem at (609,251) in 25fx348  1157.1728 [dt_interpolation_compute_pixel4c] problem at (607,253) in 25fx348
1157.1728 [dt_interpolation_compute_pixel4c] problem at (608,253) in 25fx348  1157.1728 [dt_interpolation_compute_pixel4c] problem at (609,253) in 25fx348  1157.1728 [dt_interpolation_compute_pixel4c] problem at (607,254) in 25fx348  1157.1729 [dt_interpolation_compute_pixel4c] problem at (608,254) in 25fx348 
jenshannoschwalm commented 1 year ago
  1. Clearly a log formatting error here (missing newline and printing hex)
  2. I recently added that debug log info, until then we silently wrote a zero. For some reason we are out of valid input data to do the pixelwise interpolation.

I tried here but can't reproduce so far. Could you check:

  1. issue depending on any of the interpolators? (we had issues with lanc3 until recently)
  2. depending on active modules (ashift or crop)?
  3. depending on darkroom main window size or the border size?
kofa73 commented 1 year ago

This is how the log looks like on the screen: image

Interpolators were the default: image

I get the same with both set to bilateral: image

Same issue if I resize the window (from maximised on a 4K screen to small): image

Simply turning off ashift (that's rotate and perspective, right?) produces: image

Then, zooming in/out: image

There are no new log entries in the terminal, though.

Turning rotate and perspective back on, with no cropping in the module produces interpolation error messages on the console. Zooming out fully only shows part of the image (compare the navigation preview with the editor preview -- this is with zoomed out as far as Ctrl+scroll goes): image

kofa73 commented 1 year ago

I'll try to bisect later today.

pehar1 commented 1 year ago

It took a while until I could reproduce... , about one minute of zooming and panning. I could reproduce occasionally (very rare) if

If I do not commit the "Largest area" cropping in "Rotate and perspective" and CTRL-scroll out, I get a very stange effect (possibly related) :

Set image size to "Fit" (ALT-3) issue15416-fit

Move the mouse cursor clearly in the direction of the upper left corner, then CTRL-scroll out issue15416-cursor-top-left

Set image size to "Fit" (ALT-3) again, then move the mouse cursor clearly in the direction of the bottom right corner, then CTRL-scroll out issue15416-cursor-bottom-right

I could not reproduce if

kofa73 commented 1 year ago

Thanks for confirming! Strangely though, I can reproduce this any time, with small or big window, with or without OpenCL. Running a bisect now, starting from 4.4.2.

jenshannoschwalm commented 1 year ago

Ok, i can reproduce the log data (only reported if non-opencl).

I still can't confirm the artefacts though.

Please note while bisecting, don't care about the logs, that has been added recently.

kofa73 commented 1 year ago
kofa@eagle:~/darktable$ git bisect good
09fa2c909b8a00a9648ed17d0550a2867a2b10e7 is the first bad commit
commit 09fa2c909b8a00a9648ed17d0550a2867a2b10e7
Author: Diederik ter Rahe <dterrahe@yahoo.com>
Date:   Mon Oct 2 17:02:12 2023 -0400

    concentrate zoom handling in develop.c
jenshannoschwalm commented 1 year ago

@dterrahe any quick idea on this ?

dterrahe commented 1 year ago

Steps to reproduce

  1. Get the image and sidecar from https://tech.kovacs-telekes.org/files/2023-10-14-zoom-out/

Would you be able to provide a more minimal xmp (that doesn't give me a missing lut file error) to help speed up reproduction? Unless of course this only happens with slow xmps (using multiple d&s instances).

@dterrahe any quick idea on this ?

My first guess, if this only happens when the cursor is outside the image area while the zoom is adjusted, that the code that tries to keep the same point under the cursor is at fault. Or that the image doesn't get correctly repositioned afterwards. I'm currently working on deep further adjustments in this area, so rather than trying to fix this in code I'm about to replace anyway, I'd like to have a quick test case that I can use to verify that the new code doesn't suffer the same problem.

jenshannoschwalm commented 1 year ago

Another idea here would be bad roi_in or roi_out resulting from slightly different scales. That might lead to out of bounds of output data ... i also couldn't reproduce yet, the logs are just misleading (as long as roi_in is correct but we can't test that in the interpolation function)

pehar1 commented 1 year ago

Removed all the "heavy" modules, removed "LUT 3D", removed "Color look up table", compressed history stack on kofa's xmp. Can still reproduce. Moving the mouse cursor outside the center (near edges) I can reproduce easier. The attached xmp is not zipped, extension zip only added to be able to upload. 2023-10-14_08-19-06_P1020106_01.RW2.xmp.zip

edit : if it is of interest : Ubuntu 22.04, dt 4.5.0+911-g4e44bca430 Memory : MemTotal: 65756236 kB
Graphics Card : Product Name : NVIDIA GeForce RTX 2060
Graphics Card : Driver Version : 470.199.02
OpenCL installed : Device OpenCL C Version OpenCL C 1.2
OpenCL activated : yes
Xorg : Version: 1:7.7+23ubuntu2
Desktop : LXQt
GTK+ : 3.24.33
gcc (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0
CMAKE_BUILD_TYPE : release

kofa73 commented 1 year ago

@dterrahe, @pehar1 : Sorry about the heavy modules. I've removed the old sidecar and uploaded a minimal one. As @pehar1 , the problem can be reproduced with the minimal set, too.

dterrahe commented 1 year ago

@pehar1 thanks a lot!

Submitting clear bug reports like this, with easy ways to reproduce, whenever they encounter strange behavior in a development version, is one of the most helpful assistance users can provide to the project. You've directly helped to make the next release better.

I found the issue and implemented a fix that solves it with my testing. It will be included in a large intrusive PR that I'll hopefully submit today, but if it looks like that will take some time to merge I will also submit a separate PR just addressing this. Your feedback on the big PR would be appreciated though... ;-)

dterrahe commented 1 year ago

Sorry @kofa73 ! I didn't mean to not mention you and hadn't seen your update before pressing send. What I said above of course was meant for both of you (and all other proactive testers). Many thanks again!

kofa73 commented 1 year ago

Sorry @kofa73 ! I didn't mean to not mention you and hadn't seen your update before pressing send. What I said above of course was meant for both of you (and all other proactive testers). Many thanks again!

No worries, this is not a competition -- and @pehar1 did improve on the original report - thanks! It's us users who have to thanks the devs (and it is in our own interest that reports are detailed, so fixes come quicker). :)

todd-prior commented 1 year ago

@dterrahe Just more of a comment but this issue made me think of it.... For some time now I have noticed what appears to be visible tiling or segmented processing in tiles of my screen while ctrl mouse scrolling... I have not changed any hardware. It could I suppose be some part of the more recent changes to opencl... i guess I should try this without.... In anycase I didn't note it in the past....I have a 3060TI and it never seemed to show this.... Its not an issue the image corrects but it is sort of visually distracting... is anyone aware of a change in the processing of screen previews that might have introduced that?? I'll try to capture it and take a small video and start a new topic just curious....

kofa73 commented 1 year ago

@todd-prior : I think you're seeing a new feature: https://github.com/darktable-org/darktable/blob/master/RELEASE_NOTES.md#performance-improvements

When panning or zooming, a low resolution placeholder used to be shown until the image was fully recalculated. Now the part of the previous high quality preview that is still visible is moved or resized and only the edges are temporarily shown in low quality.

It does look a bit strange at times, even though it's supposed to be better than before.

todd-prior commented 1 year ago

@todd-prior : I think you're seeing a new feature: https://github.com/darktable-org/darktable/blob/master/RELEASE_NOTES.md#performance-improvements

It does look a bit strange at times, even though it's supposed to be better than before.

Hey thanks for digging that up. When it first started I thought there was maybe something wrong or some code that needed ironed out... That does sound like what I am seeing..... you are a great resouce as always...thx again

dterrahe commented 1 year ago

Just more of a comment but this issue made me think of it.... For some time now I have noticed what appears to be visible tiling or segmented processing in tiles of my screen while ctrl mouse scrolling

Hi @todd-prior you are not the first to have remarked on this. I am genuinely curious and hope you will honestly answer the next questions:

User feedback is always useful, even if after the fact, because often getting a user opinion beforehand is impossible because the intended change is hard to describe and a lot of opinion is necessarily based on feel. And if this change was objectively bad, it can still be reversed.

even though it's supposed to be better than before

@kofa73 the "supposed" suggests you don't necessarily agree, so I would really appreciate if you were willing to answer the above questions as well.

Thanks!

todd-prior commented 1 year ago

Thanks for your reply and input. The very short answer is that I think I do remember and I think it was sort of accepted as it is normal for other programs to sort of freeze or blur and then update with the final image.... likely years of this behaviour... so the random tiles initially was a bit of a concern as I thought someting was wrong with my system or settings.... I will go home tonight and scroll around on a bunch of images.... perhaps the previous behaviour was not too bad for me as my 3060TI is fairly quick at updating but for others maybe this new approach is a better option??

dterrahe commented 1 year ago

@kofa73 @pehar1 Would you be able to give #15436 a try please? It isn't complete yet (slidershow and, according to CI, export) doesn't work yet, so it is just about seeing how zooming/panning and switching between cropped/uncropped works for you.

kofa73 commented 1 year ago

Thanks! I can test it in the afternoon.

On Tue, 17 Oct 2023, 01:29 dterrahe, @.***> wrote:

@kofa73 https://github.com/kofa73 @pehar1 https://github.com/pehar1 Would you be able to give #15436 https://github.com/darktable-org/darktable/pull/15436 a try please? It isn't complete yet (slidershow and, according to CI, export) doesn't work yet, so it is just about seeing how zooming/panning and switching between cropped/uncropped works for you.

— Reply to this email directly, view it on GitHub https://github.com/darktable-org/darktable/issues/15416#issuecomment-1765422058, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAYH2C3VB3OL36POTGGJORLX7W7NJAVCNFSM6AAAAAA6ALH76SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRVGQZDEMBVHA . You are receiving this because you were mentioned.Message ID: @.***>

kofa73 commented 1 year ago

@kofa73 @pehar1 Would you be able to give #15436 a try please?

Sorry about the delay, had a long day yesterday. I tested this morning and I can no longer reproduce this in the darkroom -- thanks! Strange: returning the the light table, I get a skull icon (this is with the RW2 and the updated, 'lightweight' _02.xmp from my link in the original post). The file is there, I can open it again. I the light table, I can create a duplicate, which also appears with the skull icon; I can discard the history, it comes up fine (my config is set to use the raw for the thumbs). Copying and pasting the history stack from the original turns the icon into a skull again. Log on the console:

   632.4406 [dt_dev_read_history_ext] temporary history deleted
   632.4406 [dt_iop_reload_defaults] defaults reloaded for filmicrgb
   632.4407 [dt_iop_reload_defaults] defaults reloaded for basecurve
   632.4407 [dt_iop_reload_defaults] defaults reloaded for colormapping
   632.4407 [dt_iop_reload_defaults] defaults reloaded for channelmixerrgb
   632.4410 [dt_iop_reload_defaults] defaults reloaded for colorin
   632.4410 [dt_iop_reload_defaults] defaults reloaded for crop
   632.4410 [dt_iop_reload_defaults] defaults reloaded for exposure
   632.4410 [dt_iop_reload_defaults] defaults reloaded for retouch
   632.4410 [dt_iop_reload_defaults] defaults reloaded for clipping
   632.4410 [dt_iop_reload_defaults] defaults reloaded for flip
   632.4410 [dt_iop_reload_defaults] defaults reloaded for ashift
   632.4410 [dt_iop_reload_defaults] defaults reloaded for cacorrectrgb
   632.4418 [dt_iop_reload_defaults] defaults reloaded for lens
   632.4418 [dt_iop_reload_defaults] defaults reloaded for scalepixels
   632.4418 [dt_iop_reload_defaults] defaults reloaded for rotatepixels
   632.4419 [dt_iop_reload_defaults] defaults reloaded for denoiseprofile
   632.4419 [dt_iop_reload_defaults] defaults reloaded for demosaic
   632.4419 [dt_iop_reload_defaults] defaults reloaded for rawdenoise
   632.4419 [dt_iop_reload_defaults] defaults reloaded for hotpixels
   632.4419 [dt_iop_reload_defaults] defaults reloaded for cacorrect
   632.4419 [dt_iop_reload_defaults] defaults reloaded for highlights
   632.4419 [dt_iop_reload_defaults] defaults reloaded for temperature
   632.4419 [dt_iop_reload_defaults] defaults reloaded for invert
   632.4419 [dt_iop_reload_defaults] defaults reloaded for rawprepare
   632.4420 [dev_insert_module] `rawprepare' inserted to history
   632.4429 [dev_insert_module] `gamma' inserted to history
   632.4430 [dev_insert_module] `temperature' inserted to history
   632.4432 [dev_insert_module] `flip' inserted to history
   632.4451 [dt_dev_read_history_ext] temporary history initialised with default params and presets
   632.4453 [dt_dev_read_history_ext] temporary history merged with image history
   632.4454 [history] successfully loaded module rawprepare from history
                        blendop v. 0:   version WRONG   params WRONG
                        params v. 2:    version ok      params ok
   632.4454 [history] successfully loaded module gamma from history
                        blendop v. 0:   version WRONG   params WRONG
                        params v. 1:    version ok      params ok
   632.4454 [history] successfully loaded module temperature from history
                        blendop v. 0:   version WRONG   params WRONG
                        params v. 3:    version ok      params ok
   632.4454 [history] successfully loaded module channelmixerrgb from history
                        blendop v. 13:  version ok      params ok
                        params v. 3:    version ok      params WRONG
   632.4454 [history] successfully loaded module exposure from history
                        blendop v. 13:  version ok      params ok
                        params v. 6:    version ok      params WRONG
   632.4455 [history] successfully loaded module filmicrgb from history
                        blendop v. 13:  version ok      params ok
                        params v. 6:    version ok      params WRONG
   632.4455 [history] successfully loaded module flip from history
                        blendop v. 13:  version ok      params ok
                        params v. 2:    version ok      params ok
   632.4455 [history] successfully loaded module demosaic from history
                        blendop v. 13:  version ok      params ok
                        params v. 4:    version ok      params ok
   632.4455 [history] successfully loaded module colorout from history
                        blendop v. 13:  version ok      params ok
                        params v. 5:    version ok      params ok
   632.4456 [history] successfully loaded module channelmixerrgb from history
                        blendop v. 13:  version ok      params ok
                        params v. 3:    version ok      params ok
   632.4456 [history] successfully loaded module colorin from history
                        blendop v. 13:  version ok      params ok
                        params v. 7:    version ok      params ok
   632.4456 [history] successfully loaded module highlights from history
                        blendop v. 13:  version ok      params ok
                        params v. 4:    version ok      params ok
   632.4456 [history] successfully loaded module exposure from history
                        blendop v. 13:  version ok      params ok
                        params v. 6:    version ok      params ok
   632.4457 [history] successfully loaded module ashift from history
                        blendop v. 13:  version ok      params ok
                        params v. 5:    version ok      params ok
   632.4505 [pixelpipe] [thumbnail] synch all modules with defaults_params
   632.4505 [iop_validate_params] `temperature' failed for type "float", field: g2
   632.4505 [iop_validate_params] `temperature' failed for type "dt_iop_temperature_params_t"
   632.4810 [iop_validate_params] `borders' failed for type "float", field: aspect
   632.4810 [iop_validate_params] `borders' failed for type "dt_iop_borders_params_t"
   632.4810 [pixelpipe] [thumbnail] synch all modules with history
   632.4810 committed params           [thumbnail]    rawprepare             piece hash=1cadd59ba8a22889, 
   632.4810 committed params           [thumbnail]    gamma                  piece hash=1ae2f37bb34a05, 
   632.4810 [iop_validate_params] `temperature' failed for type "float", field: g2
   632.4810 [iop_validate_params] `temperature' failed for type "dt_iop_temperature_params_t"
   632.4810 committed params           [thumbnail]    temperature            piece hash=ff8ce62993d709be, 
   632.4810 committed params           [thumbnail]    channelmixerrgb        piece hash=1a04683d570a9ddb, 
   632.4810 committed params           [thumbnail]    exposure               piece hash=75bff314bf103168, 
   632.4810 committed params           [thumbnail]    filmicrgb              piece hash=2a6c7c8274663789, 
   632.4810 committed params           [thumbnail]    flip                   piece hash=0, 
   632.4810 committed params           [thumbnail]    demosaic               piece hash=b0702bc252c2d1d2, 
   632.4843 committed params           [thumbnail]    colorout               piece hash=8e5a0f73242a7524, 
   632.4843 committed params           [thumbnail]    channelmixerrgb        piece hash=1a04683d570a9ddb, 
   632.4843 committed params           [thumbnail]    colorin                piece hash=7563a39ceae1e96a, 
   632.4843 committed params           [thumbnail]    highlights             piece hash=0, 
   632.4843 committed params           [thumbnail]    exposure               piece hash=75bff314bf103168, 
   632.4843 committed params           [thumbnail]    ashift                 piece hash=ed2f3aa6194a26e0, 
   632.4843 [dt_imageio_export_with_flags]    632.4843 
   632.4843  rawprepare   632.4843  temperature   632.4843  demosaic   632.4843  ashift   632.4843  exposure   632.4843  colorin   632.4843  channelmixerrgb   632.4843  filmicrgb   632.4844  colorout   632.4844  gamma   632.4844  (10)
   632.4844 [dt_imageio_export] [thumbnail] imgid 3, 3839x1967 --> 1440x737 (scale 0.375098). upscale=no, hq=no
   632.4844 pixelpipe starting on CPU  [thumbnail]                           (   0/   0) 1440x 737 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 
   632.4844 modify roi IN              [thumbnail]    ashift                 (   0/   0) 1494x 843 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 
   632.4844 modify roi IN              [thumbnail]    demosaic               (   0/   0) 3982x2248 scale=1.0000 --> (   0/   0) 1494x 843 scale=0.3751 
   632.4844 modify roi IN              [thumbnail]    rawprepare             (   0/   0) 4142x2248 scale=1.0000 --> (   0/   0) 3982x2248 scale=1.0000 
   632.4844 pixelpipe data: 1:1 copied [thumbnail]                           (   0/   0) 4142x2248 scale=1.0000 --> (   0/   0) 4142x2248 scale=1.0000 bpp=2
   632.4866 pixelpipe process CPU      [thumbnail]    rawprepare             (   0/   0) 4142x2248 scale=1.0000 --> (   0/   0) 3982x2248 scale=1.0000 IOP_CS_RAW
   632.4902 pixelpipe process CPU      [thumbnail]    temperature            (   0/   0) 3982x2248 scale=1.0000 --> (   0/   0) 3982x2248 scale=1.0000 IOP_CS_RAW
   632.4941 pixelpipe process CPU      [thumbnail]    demosaic               (   0/   0) 3982x2248 scale=1.0000 --> (   0/   0) 1494x 843 scale=0.3751 IOP_CS_RAW -> IOP_CS_RGB
   632.5460 clip_and_zoom_roi          [thumbnail]    demosaic               (   0/   0) 3982x2248 scale=1.0000 --> (   0/   0) 1494x 843 scale=0.3751 
   632.5881 pixelpipe process CPU      [thumbnail]    ashift                 (   0/   0) 1494x 843 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 IOP_CS_RGB
   632.5931 pixelpipe process CPU      [thumbnail]    exposure               (   0/   0) 1440x 737 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 IOP_CS_RGB
   632.5941 pixelpipe process CPU      [thumbnail]    colorin                (   0/   0) 1440x 737 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 IOP_CS_RGB -> IOP_CS_LAB
   632.5942 matrix conversion on CPU   [thumbnail]    colorin                (   0/   0) 1440x 737 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 `standard color matrix'
   632.5965 transform colorspace CPU   [thumbnail]    channelmixerrgb        (   0/   0) 1440x 737 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 IOP_CS_LAB -> IOP_CS_RGB
   632.5986 pixelpipe process CPU      [thumbnail]    channelmixerrgb        (   0/   0) 1440x 737 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 IOP_CS_RGB
   632.6059 pixelpipe process CPU      [thumbnail]    filmicrgb              (   0/   0) 1440x 737 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 IOP_CS_RGB
   632.6306 transform colorspace CPU   [thumbnail]    colorout               (   0/   0) 1440x 737 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 IOP_CS_RGB -> IOP_CS_LAB
   632.6327 pixelpipe process CPU      [thumbnail]    colorout               (   0/   0) 1440x 737 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 IOP_CS_LAB -> IOP_CS_RGB
   632.6359 pixelpipe process CPU      [thumbnail]    gamma                  (   0/   0) 1440x 737 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 IOP_CS_RGB
   632.6384 cache report               [thumbnail]                           2 lines (important=0, used=0, invalid=0). Using 284MB, limit=0MB. Hits/run=0.00. Hits/test=0.000
   632.6384 pixelpipe finished         [thumbnail]                           (   0/   0) 1440x 737 scale=0.3751 --> (   0/   0) 1440x 737 scale=0.3751 

   632.6384 [dt_imageio_export_with_flags] no valid output buffer
kofa73 commented 1 year ago
  • were you aware that, before this change, when the amount of zoom was changed, the whole image became temporarily low res, until after you stopped changing the zoom level when a new high resolution image was calculated and shown.

Yes.

  • did you prefer that behavior, even though for most of the screen you would see low res for longer?
  • do you think that was just because of habit or because it was more "even" or some other reason?
  • do you expect to get used to the new behavior and if so, do you think it will eventually feel "better"?

Yes, I think it's party the "uniformity" / "evenness", but also what I am used to. Now that I thought about it more and concentrated on the low res -> high res transition, I think the most important factor is that when I stop dragging the image, there are changes and movement 'in the corner of my eye' (the newly visible parts along the edges). (Sorry about the multiple edits, I only realised the 'corner of my eye' part is what disturbs me most after I experimented and wrote down my thoughts, then thought some more.)