Segfault while opening a second RAW loader

[mrbougo]

I can't seem to open two RAW files in the same project. I'm not a regular photoflow user so pardon my inaccuracies and lack of experience with the software.

Steps to reproduce:

1. Get a RAW file, I'm using a CR2 file, for instance this Canon EOS 1000D file from rawsamples.ch
2. Make a copy of it under a different name. In my use case I'm using two different raw files, but the same issue appears when using two copies of the same file (but not when using a single file in two RAW layers).
3. Open one of the copies in photoflow, the layer stack consists of a single RAW loader.
4. Add a new RAW loader, and set the file path to the other copy of the file.
5. Photoflow segfaults.

I am running Arch Linux, with VIPS 7.42.3 from the official Arch repositories.

Here's a gdb backtrace from a debug build (./build.sh debug) of current master, although I don't know whether that's useful. Please let me know if I can provide more information.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe7a81700 (LWP 2539)]
0x00007fffef2ed9f6 in _int_free () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007fffef2ed9f6 in _int_free () from /usr/lib/libc.so.6
#1  0x000000000115f003 in PF::RawImage::RawImage (this=0x7fffe11a52d0, f="") at /tmp/home/photoflow/src/operations/raw_image.cc:212
#2  0x00000000011525aa in PF::RawLoaderPar::build (this=0x7fffe05c0810, in=std::vector of length 1, capacity 1 = {...}, first=0, imap=0x0, omap=0x0, level=@0x7fffe7a8087c: 0) at /tmp/home/photoflow/src/operations/raw_loader.cc:83
#3  0x0000000000e6a837 in PF::OpParBase::build_many (this=0x7fffe05c0810, in=std::vector of length 1, capacity 1 = {...}, first=0, imap=0x0, omap=0x0, level=@0x7fffe7a8087c: 0) at /tmp/home/photoflow/src/base/operation.cc:283
#4  0x0000000000e6fc2c in PF::LayerManager::rebuild_chain (this=0x29749f8, pipeline=0x29907d0, cs=PF::PF_COLORSPACE_RGB, width=100, height=100, list=std::list = {...}, previous_layer=0x29e51e0)
    at /tmp/home/photoflow/src/base/layermanager.cc:938
#5  0x0000000000e70a09 in PF::LayerManager::rebuild (this=0x29749f8, pipeline=0x29907d0, cs=PF::PF_COLORSPACE_RGB, width=100, height=100, area=0x0) at /tmp/home/photoflow/src/base/layermanager.cc:1180
#6  0x0000000000e79a2a in PF::Image::do_update (this=0x29749f0, target_pipeline=0x0) at /tmp/home/photoflow/src/base/image.cc:242
#7  0x0000000000e77176 in PF::ImageProcessor::run (this=0x278d650) at /tmp/home/photoflow/src/base/imageprocessor.cc:200
#8  0x0000000000e76b57 in run_image_processor () at /tmp/home/photoflow/src/base/imageprocessor.cc:39
#9  0x00007ffff6dff965 in ?? () from /usr/lib/libvips.so.40
#10 0x00007ffff67cf625 in ?? () from /usr/lib/libglib-2.0.so.0
#11 0x00007ffff00dd354 in start_thread () from /usr/lib/libpthread.so.0
#12 0x00007fffef35dbfd in clone () from /usr/lib/libc.so.6

Another backtrace using two copies of this Olympus E-M5 image from rawsamples:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe7a81700 (LWP 6065)]
0x000000000115ef64 in PF::RawImage::RawImage (this=0x7fffe1170e00, f="") at /tmp/home/photoflow/src/operations/raw_image.cc:185
185         raw_hist[hist_id] += 1;
(gdb) bt
#0  0x000000000115ef64 in PF::RawImage::RawImage (this=0x7fffe1170e00, f="") at /tmp/home/photoflow/src/operations/raw_image.cc:185
#1  0x00000000011525aa in PF::RawLoaderPar::build (this=0x7fffe056e450, in=std::vector of length 1, capacity 1 = {...}, first=0, imap=0x0, omap=0x0, level=@0x7fffe7a8087c: 0) at /tmp/home/photoflow/src/operations/raw_loader.cc:83
#2  0x0000000000e6a837 in PF::OpParBase::build_many (this=0x7fffe056e450, in=std::vector of length 1, capacity 1 = {...}, first=0, imap=0x0, omap=0x0, level=@0x7fffe7a8087c: 0) at /tmp/home/photoflow/src/base/operation.cc:283
#3  0x0000000000e6fc2c in PF::LayerManager::rebuild_chain (this=0x295ab38, pipeline=0x299c5d0, cs=PF::PF_COLORSPACE_RGB, width=100, height=100, list=std::list = {...}, previous_layer=0x29ef3f0)
    at /tmp/home/photoflow/src/base/layermanager.cc:938
#4  0x0000000000e70a09 in PF::LayerManager::rebuild (this=0x295ab38, pipeline=0x299c5d0, cs=PF::PF_COLORSPACE_RGB, width=100, height=100, area=0x0) at /tmp/home/photoflow/src/base/layermanager.cc:1180
#5  0x0000000000e79a2a in PF::Image::do_update (this=0x295ab30, target_pipeline=0x0) at /tmp/home/photoflow/src/base/image.cc:242
#6  0x0000000000e77176 in PF::ImageProcessor::run (this=0x278d650) at /tmp/home/photoflow/src/base/imageprocessor.cc:200
#7  0x0000000000e76b57 in run_image_processor () at /tmp/home/photoflow/src/base/imageprocessor.cc:39
#8  0x00007ffff6dff965 in ?? () from /usr/lib/libvips.so.40
#9  0x00007ffff67cf625 in ?? () from /usr/lib/libglib-2.0.so.0
#10 0x00007ffff00dd354 in start_thread () from /usr/lib/libpthread.so.0
#11 0x00007fffef35dbfd in clone () from /usr/lib/libc.so.6

Thanks!

[aferrero2707]

Thanks for reporting this issue, I will have a look into it as soon as possible.

Meanwhile, I think it is useful to clarify how RAW images are loaded in photoflow: the "RAW loader" merely reads and decodes the RAW data, but does not apply any post-processing. In order to convert the RAW data into something useful, you need to add a "RAW developer" above the "RAW loader". The advantage of this scheme is that you can save your preferred RAW processing settings into a preset and recall it every time you open a new RAW file (as explained here: http://photoflowblog.blogspot.fr/2014/09/tutorial-how-to-process-raw-image-in.html).

To add a second RAW file above a first one, I recommend to follow those steps:

• create a group layer, and initially set the blend mode of the group layer to "passthrough"
• add the second "RAW loader" layer inside the group
• add a "RAW developer" layer above the newly created "RAW developer"
• change the blend mode of the group layer to "normal", and adjust the opacity to taste if needed. You can also associate a mask to the group layer in order to blend the two RAW files (you can for example associate a gradient, a freehand drawing or a luminosity mask to the group layer mask).

Hope this helps.

[mrbougo]

Ah, yes, I should have mentioned I did give groups of loader + developer a try, with segfault as well. Thanks for the recommendation though, and also for this very attractive piece of software.

I'll be happy to provide more debug output if you somehow don't encounter the issue.

[aferrero2707]

You are welcome!

I'll try to reproduce your issue tomorrow, and I'll post a working PFI if I cannot reproduce the crash.

[aferrero2707]

This should be fixed now in the stable branch. The crashed were the result of two independent issues. See the related commits for details: https://github.com/aferrero2707/PhotoFlow/commit/307295e603f02c32c34305f650d232a6dbafcdde https://github.com/aferrero2707/PhotoFlow/commit/b2c2964d5b0b7d880753b10098b3f4175689be56

Could you test the current stable branch to see if it now works?

Thanks!

[mrbougo]

Thank you, the bug is fixed as far as I can tell!

The latest batch of commits added a new bug, though (commit ad16291 I believe, as it works when reverted). When I add a RAW developer layer on top of a single RAW loader with the above EOS 1000D file, PhotoFlow gets stuck after outputting

PF::OperationConfigGUI::update("RAW developer"): waiting for semaphore

Please let me know if I should file a new report for this, and thanks again for your response!

[aferrero2707]

Ooops, I'll have a look into that as soon as possible, that's really a serious regression!

[aferrero2707]

For the moment I could no reproduce the lock you observed, at least not on my Mint system...

If you have some experience with gdb, here is what you could do to greatly help me pin down the problem:

• start photoflow with gdb
• open the RAW image and add the RAW developer layer, until it gets stuck
• at this point, stop the execution by typing control-z in the gdb terminal

• list the active threads, by typing

  info threads

  in the gdb command prompt

• activate one thread after the other and make a backtrace (by typing this commands in the gbd prompt, and where "n" is the ID of the various threads):

  thread n bt

This way I can see where each thread is blocking, and therefore what is causing the deadlock condition.

Just let me know if you need further help.

Thanks!

[aferrero2707]

UPDATE: I've probably finally managed to reproduce the lock, and it should now be fixed in the "stable" branch.

Could you check that? This also means that there is no need to do the gdb "gymnastics" for the moment...

Thanks!

[mrbougo]

Thanks for the fix! That seems to have fixed the deadlock I reported, but by testing I accidentally found another one. I got it by opening a RAW file, adding a group, moving the RAW loader into the group, adding a RAW developer on top of the group, then moving the raw developer at the top position inside the group. That last move got me a lock.

I can reproduce the issue consistently in a current stable debug build by opening this pfi file for the EOS 1000D raw sample and, while the view is loading, moving the raw developer on top of the loader into the group. After the view is loaded, the move seems to work fine.

I seem to be able to trigger a lock without using groups too, by moving the develop layer around. At least this lock isn't as nasty from a usability point of view.

Here's a backtrace from that

PF::OperationConfigGUI::update("RAW loader"): waiting for semaphore
^Z
Program received signal SIGTSTP, Stopped (user).
0x00007fffef39ece9 in syscall () from /usr/lib/libc.so.6
(gdb) info threads
  Id   Target Id         Frame 
  4    Thread 0x7fffe649f700 (LWP 6345) "pool" 0x00007fffef39ece9 in syscall () from /usr/lib/libc.so.6
  3    Thread 0x7fffe6eb2700 (LWP 6344) "gmain" 0x00007fffef39a18d in poll () from /usr/lib/libc.so.6
  2    Thread 0x7fffe7ab3700 (LWP 6317) "image_processor" 0x00007fffef39ece9 in syscall () from /usr/lib/libc.so.6
* 1    Thread 0x7ffff7f93980 (LWP 6316) "photoflow" 0x00007fffef39ece9 in syscall () from /usr/lib/libc.so.6
(gdb) thread apply all bt

Thread 4 (Thread 0x7fffe649f700 (LWP 6345)):
#0  0x00007fffef39ece9 in syscall () from /usr/lib/libc.so.6
#1  0x00007ffff67ed67a in g_cond_wait_until () from /usr/lib/libglib-2.0.so.0
#2  0x00007ffff677d889 in ?? () from /usr/lib/libglib-2.0.so.0
#3  0x00007ffff677deab in g_async_queue_timeout_pop () from /usr/lib/libglib-2.0.so.0
#4  0x00007ffff67d007c in ?? () from /usr/lib/libglib-2.0.so.0
#5  0x00007ffff67cf625 in ?? () from /usr/lib/libglib-2.0.so.0
#6  0x00007ffff011d4a4 in start_thread () from /usr/lib/libpthread.so.0
#7  0x00007fffef3a312d in clone () from /usr/lib/libc.so.6

Thread 3 (Thread 0x7fffe6eb2700 (LWP 6344)):
#0  0x00007fffef39a18d in poll () from /usr/lib/libc.so.6
#1  0x00007ffff67a8c7c in ?? () from /usr/lib/libglib-2.0.so.0
#2  0x00007ffff67a8d8c in g_main_context_iteration () from /usr/lib/libglib-2.0.so.0
#3  0x00007ffff67a8dc9 in ?? () from /usr/lib/libglib-2.0.so.0
#4  0x00007ffff67cf625 in ?? () from /usr/lib/libglib-2.0.so.0
#5  0x00007ffff011d4a4 in start_thread () from /usr/lib/libpthread.so.0
#6  0x00007fffef3a312d in clone () from /usr/lib/libc.so.6

Thread 2 (Thread 0x7fffe7ab3700 (LWP 6317)):
#0  0x00007fffef39ece9 in syscall () from /usr/lib/libc.so.6
#1  0x00007ffff67ed55f in g_cond_wait () from /usr/lib/libglib-2.0.so.0
#2  0x00007ffff6dff8cd in vips_semaphore_downn () from /usr/lib/libvips.so.40
#3  0x0000000000a900fd in PF::OperationConfigGUI::update (this=0x2a40c50) at /tmp/home/photoflow/src/gui/operation_config_gui.cc:718
#4  0x0000000000e62a33 in PF::LayerManager::rebuild_chain (this=0x2982698, pipeline=0x29e2630, cs=PF::PF_COLORSPACE_RGB, width=100, height=100, list=..., previous_layer=0x0) at /tmp/home/photoflow/src/base/layermanager.cc:1001
#5  0x0000000000e62b11 in PF::LayerManager::rebuild_chain (this=0x2982698, pipeline=0x29e2630, cs=PF::PF_COLORSPACE_RGB, width=100, height=100, list=..., previous_layer=0x0) at /tmp/home/photoflow/src/base/layermanager.cc:1017
#6  0x0000000000e634c9 in PF::LayerManager::rebuild (this=0x2982698, pipeline=0x29e2630, cs=PF::PF_COLORSPACE_RGB, width=100, height=100, area=0x0) at /tmp/home/photoflow/src/base/layermanager.cc:1178
#7  0x0000000000e6948c in PF::Image::do_update (this=0x2982690, target_pipeline=0x0) at /tmp/home/photoflow/src/base/image.cc:263
#8  0x0000000000e66a8e in PF::ImageProcessor::run (this=0x279c650) at /tmp/home/photoflow/src/base/imageprocessor.cc:205
#9  0x0000000000e6642d in run_image_processor () at /tmp/home/photoflow/src/base/imageprocessor.cc:39
#10 0x00007ffff6dff965 in ?? () from /usr/lib/libvips.so.40
#11 0x00007ffff67cf625 in ?? () from /usr/lib/libglib-2.0.so.0
#12 0x00007ffff011d4a4 in start_thread () from /usr/lib/libpthread.so.0
#13 0x00007fffef3a312d in clone () from /usr/lib/libc.so.6

Thread 1 (Thread 0x7ffff7f93980 (LWP 6316)):
#0  0x00007fffef39ece9 in syscall () from /usr/lib/libc.so.6
#1  0x00007ffff67eca3c in ?? () from /usr/lib/libglib-2.0.so.0
#2  0x0000000000a91d23 in PF::Image::lock (this=0x2982690) at /tmp/home/photoflow/src/gui/../base/image.hh:156
#3  0x0000000000e42e77 in PF::LayerTreeModel::drag_data_received_vfunc (this=0x29dbab0, dest=..., selection_data=...) at /tmp/home/photoflow/src/gui/layertree_dnd.cc:424
#4  0x00007ffff37cab32 in Gtk::TreeDragDest_Class::drag_data_received_vfunc_callback(_GtkTreeDragDest*, _GtkTreePath*, _GtkSelectionData*) () from /usr/lib/libgtkmm-2.4.so.1
#5  0x00007ffff2e73810 in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#6  0x00007ffff37e1e80 in Gtk::Widget_Class::drag_data_received_callback(_GtkWidget*, _GdkDragContext*, int, int, _GtkSelectionData*, unsigned int, unsigned int) () from /usr/lib/libgtkmm-2.4.so.1
#7  0x00007ffff2d73e67 in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#8  0x00007ffff6a7d2f5 in g_closure_invoke () from /usr/lib/libgobject-2.0.so.0
#9  0x00007ffff6a8ef22 in ?? () from /usr/lib/libgobject-2.0.so.0
#10 0x00007ffff6a97688 in g_signal_emit_valist () from /usr/lib/libgobject-2.0.so.0
#11 0x00007ffff6a97e3a in g_signal_emit_by_name () from /usr/lib/libgobject-2.0.so.0
#12 0x00007ffff2ea3a29 in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#13 0x00007ffff6a7d2f5 in g_closure_invoke () from /usr/lib/libgobject-2.0.so.0
#14 0x00007ffff6a8f02c in ?? () from /usr/lib/libgobject-2.0.so.0
#15 0x00007ffff6a97688 in g_signal_emit_valist () from /usr/lib/libgobject-2.0.so.0
#16 0x00007ffff6a97e3a in g_signal_emit_by_name () from /usr/lib/libgobject-2.0.so.0
#17 0x00007ffff2dd28b3 in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#18 0x00007ffff2dd42f8 in gtk_selection_convert () from /usr/lib/libgtk-x11-2.0.so.0
#19 0x00007ffff2e74231 in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#20 0x00007ffff37e1d14 in Gtk::Widget_Class::drag_drop_callback(_GtkWidget*, _GdkDragContext*, int, int, unsigned int) () from /usr/lib/libgtkmm-2.4.so.1
#21 0x00007ffff2d71f99 in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#22 0x00007ffff6a7d2f5 in g_closure_invoke () from /usr/lib/libgobject-2.0.so.0
#23 0x00007ffff6a8ef22 in ?? () from /usr/lib/libgobject-2.0.so.0
#24 0x00007ffff6a97195 in g_signal_emit_valist () from /usr/lib/libgobject-2.0.so.0
#25 0x00007ffff6a97e3a in g_signal_emit_by_name () from /usr/lib/libgobject-2.0.so.0
#26 0x00007ffff2ea66f7 in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#27 0x00007ffff2ea4700 in ?? () from /usr/lib/libgtk-x11-2.0.so.0
#28 0x00007ffff2d70593 in gtk_main_do_event () from /usr/lib/libgtk-x11-2.0.so.0
#29 0x00007ffff1d282cc in ?? () from /usr/lib/libgdk-x11-2.0.so.0
#30 0x00007ffff67a89fd in g_main_context_dispatch () from /usr/lib/libglib-2.0.so.0
#31 0x00007ffff67a8ce0 in ?? () from /usr/lib/libglib-2.0.so.0
#32 0x00007ffff67a9002 in g_main_loop_run () from /usr/lib/libglib-2.0.so.0
#33 0x00007ffff2d6f467 in gtk_main () from /usr/lib/libgtk-x11-2.0.so.0
#34 0x00007ffff377e94d in Gtk::Main::run(Gtk::Window&) () from /usr/lib/libgtkmm-2.4.so.1
#35 0x0000000000a88410 in main (argc=1, argv=0x7fffffffe218) at /tmp/home/photoflow/src/main.cc:236

[aferrero2707]

Thanks for the detailed backtrace, that will be really helpful! I've already some idea of what could be wrong, will try to look into that later. However, the fix might not be as trivial as the last one...

I'll still leave this issue opened until everything is fixed.

[mrbougo]

Great! Thanks again for your perseverance, I greatly appreciate it :smile:

[aferrero2707]

Should be fixed in 0.2.1, closing issue.