search

darktable-org / darktable

darktable is an open source photography workflow application and raw developer
https://www.darktable.org
GNU General Public License v3.0
9.77k stars 1.14k forks source link

Darktable 3.0/Windows 10 crashes on export #4374

Closed grumpazoid closed 4 years ago

grumpazoid commented 4 years ago

Today I created a panorama with hugin on my work laptop (windows 10) and then did some darktable edits.

I have transferred the files onto my main PC but I get a crash on exporting the larger file at 0x0 original resolution. It works at smaller resolutions.

The crash takes various forms: 1.Crash to desktop without any errors. 2.A windows box pops up saying that the graphics driver failed to load. ( I have noticed that open CL was not available on restart and driver not working shown in device manager) Had to reboot.

  1. I get an unhandled exception error.
  2. message telling me a bt file has been written.

Files attached.

I have only recently upgraded from 2.6 to 3.0 on both machines. Graphics gard is GTX 970

darktable_bt_H29JG0.txt crash

johnny-bit commented 4 years ago

The issue happens in src/common/locallaplacian.c in function gauss_reduce_sse2 (at least in your crash report).

Given the outlining of omp it seems that the probelm is manifested in 2nd loop (starting from line 177).

Since code in locallaplacian.c doesn't use OpenCL (but heavily uses pararellism and sse) it might not be affected by OpenCL (at least not directly).

Dear @grumpazoid - can you share your image with us along with XMP that's causing the issue? If not - how bit is your image (width x height) + what modules and modes you have enabled?

Does the crash happen on both machines during export? What specs are your machines (please include as much details as possible: CPU, cores, RAM, GPU + it's ram) Can you use OpenCL on those? Does the issue happen regardless of openCL setting?

What version is your OS? Graphics driver version (especially OpenCL)?

grumpazoid commented 4 years ago

@johnny-bit Image is 16598 x 3421 I will attach a screenshot so you can see modules in use. Crash only happens on my home PC which is the one I recently upgraded from Win7 to 10 and also upgrade of darktable from 2.6 to 3 The laptop has a clean install of v.3 and has no issues. I would be happy to fully reset my home version of DT to factory defaults if I knew how?

Laptop is ASUS gaming PC core i7 9th Gen 16GB DDR4 RTX2060 6GB (screen shots attached) Home PC is 4th Gen i5 4570 with 8GB DDR3 GTX 970 4GB As mentioned After the crash on my home PC the graphics driver seems to stop and OpenCL is not available within DT. I will try disabling openCL tonight and retry the export. Both OS's are Windows 10 with latest updates. Graphics drivers are up to date. Sorry I don't know how to find openCl drivers? dtscreenshot asus cpu asus laptop gpu

grumpazoid commented 4 years ago

I can confirm it still crashes with OpenCL switched off.

grumpazoid commented 4 years ago

The issue happens in src/common/locallaplacian.c in function gauss_reduce_sse2 (at least in your crash report).

Ok..found the cause...it is the local contrast module causing the problem - specifically local laplacian filter. If I change it to bilateral the export works.

So the question is why does it work on one machine and not another? Now I am thinking to try a third windows 10 machine that has a clean install of darktable.

johnny-bit commented 4 years ago

I'm still learning about that stuff but local laplacian is complicated beast.

Which machine crashes? the one with less ram?

darktable on windows does output debugging info. we'll need it in this case and will need a it of more information regarding task context. Here's what I want you to do: try to run darktable in console (cmd.exe should be good enough, I honestly haven't used windows since 2005) and run it like darktable -d all - this should generate tons of info. If it's too much we'll try to trim it down to darktable -d perf -d dev -d opencl -d memory. If you'll manage to run it that way remember to do bare minimum - just export this image (with local laplacian filter on) and once it crashes attach log here.

And if you can - open up task manager and observe graphs during export, are all your cores utilized, is memory usage maxed out etc.

grumpazoid commented 4 years ago

@johnny-bit Yes the lower spec desktop PC with 8GB RAM I can see that all 4 cores reach 100% on export and RAM usage is about 90% I have tried darktable -d all. Darktable runs ok but no new files appear in the .debug folder? Files in there are about 3 months old according to modified date.

johnny-bit commented 4 years ago

the backtrace we're looking for isn't in .debug folder. it should be in C:\Users\[user]\AppData\Local\Temp\

I believe you're running out of RAM to allocate local laplacian pyramids. Are you able to temporarly increase ram amount on your lower spec PC? eg borrow 1 RAM module so you have just a tiny bit more, eg 10-12GB? If that helps then it's memory management issue.

grumpazoid commented 4 years ago

Thanks. Got it now. Attached. darktable_bt_PSPGG0.txt

Unfortunately I only have 2 RAM slots and nothing bigger than 4Gb Simms.

johnny-bit commented 4 years ago

Your error (the one from crash log) seems to be caused by RAM error. Therefore this operation needs more ram ;)

your image si roughly 55 megapixel one and let me tell you - my 24GB machine struggles sometimes with 24megapixel photos. Unfortunately processing is sometimes memory hogging. In your case I'd stick to higher-end one for processing of higher-megapixel images.

pt., 28 lut 2020 o 19:22 grumpazoid notifications@github.com napisał(a):

Thanks. Got it now. Attached. darktable_bt_PSPGG0.txt https://github.com/darktable-org/darktable/files/4268451/darktable_bt_PSPGG0.txt

Unfortunately I only have 2 RAM slots and nothing bigger than 4Gb Simms.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/darktable-org/darktable/issues/4374?email_source=notifications&email_token=ACRRKFMYJE6COCEBSPJ76YTRFFI5BA5CNFSM4K4NFOW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENJURYI#issuecomment-592660705, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACRRKFP4AZDUH56LEQY5NULRFFI5BANCNFSM4K4NFOWQ .

-- Pozdrawiam, Hubert Kowalski

grumpazoid commented 4 years ago

OK. Thanks very much for having a look. Yes, this is probably the biggest picture I have attempted to work with. I stitched 13 photos, each 24 megapixel, together using hugin, resulting in a 14Mb final image. This was donbe on the more powerful laptop. Looks like it may be time for a RAM upgrade to keep my ancient Intel 4th gen desktop PC going!

johnny-bit commented 4 years ago

If Export works on your more powerfull machine (with same settings as on less powerfull one), please close this issue :) Thanks!

jenshannoschwalm commented 4 years ago

That's a very quick-shot response. I would rather concider this as a dt bug resulting from too optimistic memory allocation without error handling.

parafin commented 4 years ago

I'm not sure how much error-handling darktable can do if OS video driver crashes...

grumpazoid commented 4 years ago

I have now tried this the same export on a very old core2duo windows 10 laptop with fresh install of Darktable. It took about 2 minutes and there was lots of disk activity but the export worked!!!!!

So please can someone tell me how I completely remove darktable from my desktop PC and start over with default settings.

grumpazoid commented 4 years ago

I have hopefully solved this. I upgraded to a bigger SSD this weekend so I could play the new Free Call of Duty! I got a paging error and needed to let Windows manage the swap file size as opposed to what the SSD software wanted to do. Result is that export on the same file now works. Hope this helps someone.