Crash when "scale pixels" module is moved on top of pixelpipe and "rotate and perspective" is used

da-phil commented 3 months ago

Describe the bug

When scalepixels module is used and moved on top of pixel pipeline while using "rotate and perspective" with some vertical or horizontal lens shift darktable crashes.

Motivation for using and moving scalepixels module on top of the pixel pipeline is the following:

To make the final edit fit into a certain aspect ratio and crop. Let's say, I'm editing a 3:2 image and apply some corrections which modify the image area and therefore also the aspect ratio, like horizontal and/or vertical lens shift in the "rotate and perspective" module, or the lens correction module. Afterwards I want to be able to apply a 3:2 crop again, without sacrificing image area for the crop. The "scale pixels" module can be use to either squeeze (>1) or stretch the image vertically to make everything fit into the crop I need. Did this explanation help?
If the module is activated after finishing a picture with several masks and drawings in the retouch module, the image warping is not reflected in all the masks if it stays above demosaicing (original position). Hence I moved it on top of all modules, except the crop module.

The issue seems to happen with OpenCL turned on and off, however turning OpenCL off takes longer to provoke the crash, which I find quite weird. But once I found module settings which makes dt crash, just loading the image while OpenCL is on and off leads to the same crash (munmap_chunk(): invalid pointer)

Steps to reproduce

Open arbitrary image
Enable "rotate and perspective" module with a shift in either vertical or horizontal direction, or both
Enable "scale pixels" module and move all the the way above the tonemapping module (sigma / filmic)
Set scale to either < 1 or > 1
If darktable does not crash immediately, play around with shift parameters in the "rotate and perspective" module until it crashes

Expected behavior

No crash

Logfile | Screenshot | Screencast

I was able to get a meaningful stack trace and noticed NaNs in the dt_interpolation_compute_pixel4c function

Thread 113 (Thread 0x7f1a3cdc5640 (LWP 289682) "worker res 0"):
#0  0x00007f1c5b83636b in dt_interpolation_compute_pixel4c (itor=0x7f1c5bf87420 <dt_interpolator+96>, in=0x7f190e722580, out=0x7f196a7a7b60, x=1214, y=566.554749, width=2046, height=1363, linestride=8184) at /home/phil/code/darktable/src/common/interpolation.c:653
        c = 2
        h = {0.0276787803, 0.0373852439, 0.0473049283, -nan(0x400000)}
        i = 2
        itor_width = 6
        pixel = {0.0116574438, 0.0161002278, -0.00482130749, -nan(0x400000)}
        __PRETTY_FUNCTION__ = "dt_interpolation_compute_pixel4c"
        kernelh = {2.53302934e-11, 1.01321174e-10, 1, 1.01321174e-10, 2.53302934e-11, 1.12579078e-11, 6.33257335e-12, 4.05284703e-12}
        kernelv = {0.020618856, -0.12370304, 0.534325838, 0.678812742, -0.143157423, 0.0275011789, 0.0159715433, -0.0613454357}
        normh = 1
        normv = 0.994398117
        oonorm = 1.00563347
        ix = 1214
        iy = 566
#1  0x00007f1c1c6f3119 in process._omp_fn.0 () at /home/phil/code/darktable/src/iop/scalepixels.c:225
        x = 1214
        y = 566.554749
        i = 1214
        out = 0x7f196a7a7b60
        j = 520
        ivoid = 0x7f190d572080
        ovoid = 0x7f1969767080
        roi_in = 0x7f1c357dfc80
        roi_out = 0x7f1c357e00f0
        ch_width = 8184
        interpolation = 0x7f1c5bf87420 <dt_interpolator+96>
        d = 0x7f1c18002a40
        ch = 1214
        ch_width = 8184
        interpolation = 0x440da3814497c000
        d = 0x7f196a7a7b60

see full trace: darktable_bt_1UBMS2.txt. But there are plenty of other NaNs in the backtrace.

Commit

No response

Where did you obtain darktable from?

self compiled

darktable version

4.9.0+238~g03a771b4ba

What OS are you using?

Linux

What is the version of your OS?

Ubuntu 22.04

Describe your system?

CPU: AMD Ryzen 7 5700x 8-core processor GPU: AMD Radeon RX 5700 XT RAM: 32 GB X11 is in use

Are you using OpenCL GPU in darktable?

Yes

If yes, what is the GPU card and driver?

AMD Radeon RX 5700 XT, ROCR installation, Radeon Software for Linux version 24.10.3 for Ubuntu 22.04.4 HWE / amdgpu-install_6.1.60103

Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip

No response

wpferguson commented 3 months ago

What were you trying to achieve by moving it up in the pixelpipe?

There's a reason behind the pixelpipe ordering and you just discovered it.

da-phil commented 3 months ago

What were you trying to achieve by moving it up in the pixelpipe?

There's a reason behind the pixelpipe ordering and you just discovered it.

See my post above:

Motivation for using and moving scalepixels module on top of the pixel pipeline is the following: If the module is activated after finishing a picture with several masks and drawings in the retouch module, the image warping is not reflected in the masks if it stays above demosaicing. Hence I moved it on top of all modules, except the crop module.

More concretely: after finishing a picture, I might notice that the desired crop does not work out without cropping away important parts of the image. Using the scale pixels module I'm able to squeeze the picture to make it fit into the aspect ratio of my desired crop.

That being said: the only module which causes this crash is "rotate and perspective", all others which might be sensitive above image boundaries (e.g. retouch and lens correction) do not lead to a crash or other issues. That's why I think it's a bug and the pixel pipe should be able to handle this rearrangement, but I'm not an expert on the concepts and limitations of the pixel pipe, so I'm open to discuss my expectations ;)

wpferguson commented 3 months ago

From the user manual....

Some cameras (such as the Nikon D1X) have rectangular instead of the usual square sensor cells. Without correction this would lead to distorted images.

So IIUC you're using the scale pixel module to correct a mask problem?

da-phil commented 3 months ago

Apologies for giving a more precise answer only after sending my original reply, I edited it.

So my motivation is to make the final edit fit into a certain aspect ratio and crop. Let's say, I'm editing a 3:2 image and apply some corrections which modify the image area and therefore also the aspect ratio, like horizontal and/or vertical lens shift in the "rotate and perspective" module, or the lens correction module. Afterwards I want to be able to apply a 3:2 crop again, without sacrificing image area for the crop. The "scale pixels" module can be use to either squeeze (>1) or stretch the image vertically to make everything fit into the crop I need. Did this explanation help?

wpferguson commented 3 months ago

Now I understand.

github-actions[bot] commented 1 month ago

This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

da-phil commented 1 month ago

Unfortunately I also didn't have time to further investigate the issue, but it's certainly to be found in dt_interpolation_compute_pixel4c. Will try to find some time after my vacation.

darktable-org / darktable