Seg fault with darktable-generate-cache

[chhil]

darktable 3.3.0+1705~g2bdb83188 With 5K images and I constantly change between collections, the lighttable view was taking long to draw the images (5 in a row). It starts off fast but slows down over time.

Decided to build the cache and see if helped.

mchhil@xyz:~/darktable$ /opt/darktable/bin/darktable-generate-cache
creating complete lighttable thumbnail cache
creating cache directories
creating cache directory '/home/mchhil/.cache/darktable/mipmaps-aa9d315b7cd299e1db70d11c4a2ce8afc56563cb.d/0'
creating cache directory '/home/mchhil/.cache/darktable/mipmaps-aa9d315b7cd299e1db70d11c4a2ce8afc56563cb.d/1'
creating cache directory '/home/mchhil/.cache/darktable/mipmaps-aa9d315b7cd299e1db70d11c4a2ce8afc56563cb.d/2'
image 1/5078 (0.02%) (id:1)
image 2/5078 (0.04%) (id:2)
image 3/5078 (0.06%) (id:3)
image 4/5078 (0.08%) (id:4)
image 5/5078 (0.10%) (id:5)
image 6/5078 (0.12%) (id:6)
image 7/5078 (0.14%) (id:7)
image 8/5078 (0.16%) (id:8)
image 9/5078 (0.18%) (id:9)
image 10/5078 (0.20%) (id:10)
....
....
....
image 791/5078 (15.58%) (id:791)
image 792/5078 (15.60%) (id:792)
image 793/5078 (15.62%) (id:793)
**Segmentation fault (core dumped)**

This is what my cache directory structure looks like

\home\mchhil.cache\darktable\mipmaps-aa9d315b7cd299e1db70d11c4a2ce8afc56563cb.d\5 \home\mchhil.cache\darktable\mipmaps-aa9d315b7cd299e1db70d11c4a2ce8afc56563cb.d\3 \home\mchhil.cache\darktable\mipmaps-aa9d315b7cd299e1db70d11c4a2ce8afc56563cb.d\0 \home\mchhil.cache\darktable\mipmaps-aa9d315b7cd299e1db70d11c4a2ce8afc56563cb.d\1 \home\mchhil.cache\darktable\mipmaps-aa9d315b7cd299e1db70d11c4a2ce8afc56563cb.d\2

Folders 3 and 5 were from before I ran generate cache based on timestamps.

No backtrace file available.

[chhil]

I removed all mipmap folders and tried again. Seg fault at same place.

image 793/5078 (15.62%) (id:793)
**Segmentation fault (core dumped)**

[TurboGit]

would be nice to know what the xmp for image 793 contains.

[chhil]

How is 793 related to data in the library.db?

Some query I can run?

[chhil]

Based on the info in images.id

I have shared the xmp corresponding to id=793

P1082279.RW2.xmp.txt

[chhil]

Just in case , I have provided id 794 as it may have crashed processing that one. P1082280.RW2.xmp.txt

[chhil]

If it helps, when I run darktable-generate-cache using vscode debugging on linux with breakpoint set on exceptions...it crashes after this...

libdarktable.so!dt_interpolation_resample_sse._omp_fn.1() (\home\mchhil\darktable\src\common\interpolation.c:1727)
libgomp.so.1![Unknown/Just-In-Time compiled code] (Unknown Source:0)
libpthread.so.0!start_thread(void * arg) (\build\glibc-ZN95T4\glibc-2.31\nptl\pthread_create.c:477)
libc.so.6!clone() (\build\glibc-ZN95T4\glibc-2.31\sysdeps\unix\sysv\linux\x86_64\clone.S:95)

this is darktable 3.3.0+1705~g2bdb83188 reporting a segfault:

This is what got written to the backtrace file.

warning: Currently logging to /tmp/darktable_bt_68ROU0.txt.  Turn the logging off and on to make the new setting effective.
/home/mchhil/darktable/build/share/darktable/gdb_commands:2: Error in sourced command file:
No stack.

[chhil]

Sorry, I am new to the debugging environment...

This may be the stacktrace that wil interest the devs.

libdarktable.so!_mm_add_ps(__m128 __A) (\usr\lib\gcc\x86_64-linux-gnu\9\include\xmmintrin.h:198)
libdarktable.so!dt_interpolation_resample_sse._omp_fn.1() (\home\mchhil\darktable\src\common\interpolation.c:1727)
libgomp.so.1!GOMP_parallel (Unknown Source:0)
libdarktable.so!dt_interpolation_resample_sse(const dt_iop_roi_t * const roi_in, const dt_iop_roi_t * const roi_in, const int32_t in_stride, const float * const in, const int32_t out_stride, const dt_iop_roi_t * const roi_out, float * out, const struct dt_interpolation * itor) (\home\mchhil\darktable\src\common\interpolation.c:1669)
libdarktable.so!dt_interpolation_resample(const struct dt_interpolation * itor, float * out, const dt_iop_roi_t * const roi_out, const int32_t out_stride, const float * const in, const dt_iop_roi_t * const roi_in, const int32_t in_stride) (\home\mchhil\darktable\src\common\interpolation.c:1782)
libdarktable.so!dt_iop_clip_and_zoom(float * out, const float * const in, const dt_iop_roi_t * const roi_out, const dt_iop_roi_t * const roi_in, const int32_t out_stride, const int32_t in_stride) (\home\mchhil\darktable\src\develop\imageop_math.c:144)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1241)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1270)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1270)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1270)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1270)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1270)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1270)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1145)
libdarktable.so!dt_dev_pixelpipe_process_rec(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, void ** output, void ** cl_mem_output, dt_iop_buffer_dsc_t ** out_format, const dt_iop_roi_t * roi_out, GList * modules, int pos) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:1270)
libdarktable.so!dt_dev_pixelpipe_process_rec_and_backcopy(int pos, GList * pieces, GList * modules, const dt_iop_roi_t * roi_out, dt_iop_buffer_dsc_t ** out_format, void ** cl_mem_output, void ** output, dt_develop_t * dev, dt_dev_pixelpipe_t * pipe) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:2179)
libdarktable.so!dt_dev_pixelpipe_process(dt_dev_pixelpipe_t * pipe, dt_develop_t * dev, int x, int y, int width, int height, float scale) (\home\mchhil\darktable\src\develop\pixelpipe_hb.c:2267)
libdarktable.so!dt_imageio_export_with_flags(const int32_t imgid, const char * filename, dt_imageio_module_format_t * format, dt_imageio_module_data_t * format_params, const gboolean ignore_exif, const gboolean display_byteorder, const gboolean high_quality, const gboolean upscale, const gboolean thumbnail_export, const char * filter, const gboolean copy_metadata, const gboolean export_masks, dt_colorspaces_color_profile_type_t icc_type, const gchar * icc_filename, dt_iop_color_intent_t icc_intent, dt_imageio_module_storage_t * storage, dt_imageio_module_data_t * storage_params, int num, int total, dt_export_metadata_t * metadata) (\home\mchhil\darktable\src\common\imageio.c:764)
libdarktable.so!_init_8(const dt_mipmap_size_t size, const uint32_t imgid, dt_colorspaces_color_profile_type_t * color_space, float * iscale, uint32_t * height, uint32_t * width, uint8_t * buf) (\home\mchhil\darktable\src\common\mipmap_cache.c:1255)
libdarktable.so!dt_mipmap_cache_get_with_caller(dt_mipmap_cache_t * cache, dt_mipmap_buffer_t * buf, const uint32_t imgid, const dt_mipmap_size_t mip, const dt_mipmap_get_flags_t flags, const char mode, const char * file, int line) (\home\mchhil\darktable\src\common\mipmap_cache.c:822)
generate_thumbnail_cache(const int32_t max_imgid, const int32_t min_imgid, const dt_mipmap_size_t max_mip, const dt_mipmap_size_t min_mip) (\home\mchhil\darktable\src\generate-cache\main.c:112)
main(int argc, char ** arg) (\home\mchhil\darktable\src\generate-cache\main.c:253)

[TurboGit]

Looks like an out of bound index ! What if you open the image in dt directly ? Does it crash too ?

[TurboGit]

This could also be because the xmp is corrupted somehow.

[TurboGit]

@chhil : since you can rebuild dt, can you try this patch:

diff --git a/src/common/interpolation.c b/src/common/interpolation.c
index e0f3b7581..ef1ca3eb1 100644
--- a/src/common/interpolation.c
+++ b/src/common/interpolation.c
@@ -1723,7 +1723,8 @@ static void dt_interpolation_resample_sse(const struct dt_interpolation *itor, f
           const size_t baseidx = (size_t)hindex[hiidx++] * 4;
           const float htap = hkernel[hkidx++];
           const __m128 vhtap = _mm_set_ps1(htap);
-          vhs = _mm_add_ps(vhs, _mm_mul_ps(*(__m128 *)&i[baseidx], vhtap));
+          const __m128 ival = _mm_loadu_ps(&i[baseidx]);
+          vhs = _mm_add_ps(vhs, _mm_mul_ps(ival, vhtap));
         }

This is in common/interpolation.c line 1727

EDIT: Sorry I meant _mm_loadu_ps() above, just edited the patch.

[chhil]

Crashes Dt when image is opened. Removed the xmp and DT still crashes.

I worked around the issue, by renaming the file and adding a _1 to it, deleted the xmp file and re imported it.

since you can rebuild dt, can you try this patch:

I can try it, will revert the filename and see if I can still reproduce it.

[TurboGit]

Crashes Dt when image is opened. Removed the xmp and DT still crashes.

Yes because the XMP is not used but the data in the db.

[TurboGit]

I worked around the issue, by renaming the file and adding a _1 to it, deleted the xmp file and re imported it.

You've removed the image and imported a fresh copy. Right ?

[TurboGit]

I can try it, will revert the filename and see if I can still reproduce it.

Would be nice, I'm wondering if this is not a non aligned memory access. TIA.

[chhil]

You've removed the image and imported a fresh copy. Right ?

Renamed the file, its basically treating it as a new one without an xmp and then imported this single image.

Would be nice, I'm wondering if this is not a non aligned memory access. TIA.

Its in progress, need to wait till image 793 is reached.

[TurboGit]

Renamed the file, its basically treating it as a new one without an xmp and then imported this single image.

Ok, clear now. Thanks for clarifications.

[chhil]

Renamed the file to what was in the db for id 793 Placed the xmp that was present when it was failing.

Now for some reason I started gettting excpetions for earlier id'd 400+ and for each its throwing the following exception. It says tiff but the files are panasonic raws rw2 files.

libstdc++.so.6!__cxa_throw (Unknown Source:0)
libdarktable.so!rawspeed::ThrowException<rawspeed::TiffParserException>(const char * fmt) (\usr\include\c++\9\array:234)
libdarktable.so!rawspeed::TiffIFD::TiffIFD(rawspeed::TiffIFD * const this, rawspeed::TiffIFD * parent_, rawspeed::NORangesSet * ifds,  data, uint32_t offset) (\home\mchhil\darktable\src\external\rawspeed\src\librawspeed\tiff\TiffIFD.cpp:121)
libdarktable.so!std::make_unique<rawspeed::TiffIFD, rawspeed::TiffIFD*, std::set<rawspeed::Buffer, rawspeed::RangesOverlapCmp<rawspeed::Buffer>, std::allocator<rawspeed::Buffer> >*&, rawspeed::ByteStream&, unsigned int>() (\usr\include\c++\9\bits\unique_ptr.h:856)
libdarktable.so!rawspeed::TiffIFD::parseIFDEntry(rawspeed::TiffIFD * const this, rawspeed::NORangesSet * ifds, rawspeed::ByteStream * bs) (\home\mchhil\darktable\src\external\rawspeed\src\librawspeed\tiff\TiffIFD.cpp:80)
libdarktable.so!rawspeed::TiffIFD::TiffIFD(rawspeed::TiffIFD * const this, rawspeed::TiffIFD * parent_, rawspeed::NORangesSet * ifds,  data, uint32_t offset) (\home\mchhil\darktable\src\external\rawspeed\src\librawspeed\tiff\TiffIFD.cpp:124)
libdarktable.so!rawspeed::TiffRootIFD::TiffRootIFD(uint32_t offset, const rawspeed::DataBuffer & data, rawspeed::NORangesSet * ifds, rawspeed::TiffIFD * parent_, rawspeed::TiffRootIFD * const this) (\home\mchhil\darktable\src\external\rawspeed\src\librawspeed\tiff\TiffIFD.h:133)
libdarktable.so!std::make_unique<rawspeed::TiffRootIFD, rawspeed::TiffIFD*, std::set<rawspeed::Buffer, rawspeed::RangesOverlapCmp<rawspeed::Buffer>, std::allocator<rawspeed::Buffer> >*&, rawspeed::ByteStream&, unsigned int>() (\usr\include\c++\9\bits\unique_ptr.h:857)
libdarktable.so!rawspeed::TiffIFD::parseMakerNote(rawspeed::TiffIFD * const this, rawspeed::NORangesSet * ifds, rawspeed::TiffEntry * t) (\home\mchhil\darktable\src\external\rawspeed\src\librawspeed\tiff\TiffIFD.cpp:201)
libdarktable.so!rawspeed::TiffIFD::parseIFDEntry(rawspeed::TiffIFD * const this, rawspeed::NORangesSet * ifds, rawspeed::ByteStream * bs) (\home\mchhil\darktable\src\external\rawspeed\src\librawspeed\tiff\TiffIFD.cpp:73)
libdarktable.so!rawspeed::TiffIFD::TiffIFD(rawspeed::TiffIFD * const this, rawspeed::TiffIFD * parent_, rawspeed::NORangesSet * ifds,  data, uint32_t offset) (\home\mchhil\darktable\src\external\rawspeed\src\librawspeed\tiff\TiffIFD.cpp:124)
libdarktable.so!std::make_unique<rawspeed::TiffIFD, rawspeed::TiffRootIFD*, std::set<rawspeed::Buffer, rawspeed::RangesOverlapCmp<rawspeed::Buffer>, std::allocator<rawspeed::Buffer> >*, rawspeed::ByteStream&, unsigned int&>() (\usr\include\c++\9\bits\unique_ptr.h:856)
libdarktable.so!rawspeed::TiffParser::parse(rawspeed::TiffIFD * parent,  data) (\home\mchhil\darktable\src\external\rawspeed\src\librawspeed\parsers\TiffParser.cpp:82)
libdarktable.so!rawspeed::TiffParser::getDecoder(rawspeed::TiffParser * const this, const rawspeed::CameraMetaData * meta) (\home\mchhil\darktable\src\external\rawspeed\src\librawspeed\parsers\TiffParser.cpp:62)
libdarktable.so!rawspeed::RawParser::getDecoder(rawspeed::RawParser * const this, const rawspeed::CameraMetaData * meta) (\home\mchhil\darktable\src\external\rawspeed\src\librawspeed\parsers\RawParser.cpp:69)
libdarktable.so!dt_imageio_open_rawspeed(dt_image_t * img, const char * filename, dt_mipmap_buffer_t * mbuf) (\home\mchhil\darktable\src\common\imageio_rawspeed.cc:134)
libdarktable.so!dt_imageio_open(dt_image_t * img, const char * filename, dt_mipmap_buffer_t * buf) (\home\mchhil\darktable\src\common\imageio.c:1182)
libdarktable.so!dt_mipmap_cache_get_with_caller(dt_mipmap_cache_t * cache, dt_mipmap_buffer_t * buf, const uint32_t imgid, const dt_mipmap_size_t mip, const dt_mipmap_get_flags_t flags, const char mode, const char * file, int line) (\home\mchhil\darktable\src\common\mipmap_cache.c:784)
libdarktable.so!_dt_dev_load_raw(const uint32_t imgid, dt_develop_t * dev) (\home\mchhil\darktable\src\develop\develop.c:643)
libdarktable.so!dt_dev_load_image(dt_develop_t * dev, const uint32_t imgid) (\home\mchhil\darktable\src\develop\develop.c:697)

[TurboGit]

You have properly updated the rawspeed sub-module ?

$ git submodule update

The good news is that it seems I have the fix. I'll issue a PR, for the other you'll probably to create a new issue report.

Maybe there is a rawspeed regression on .RW2 ?

[chhil]

$ git submodule update

Yes.

Maybe there is a rawspeed regression on .RW2 ?

Unfortunately, I would have no way of knowing. I am just poking around as I can see a lot of times an issue won't be reproducible and needs better debug info.

As for the cache generation, I am running it as a regular executable now and it seems to have reached 600. Still sometime to go for 729.

[chhil]

Sorry for the earlier comment.

It seg faulted on 793.

[TurboGit]

It seg faulted on 793.

Too bad, just to be sure you have use the _mm_loadu_ps (I edited the patch minute after I published it, just to be sure).

If yes, then can you run under gdb again... Maybe there is another similar crash in another place.

[chhil]

[TurboGit]

Ok, so that's either a out of bound access or a corrupted XMP. I'll close my PR.

[chhil]

@TurboGit

If interested the raw and xmp are located below. https://www.dropbox.com/sh/g6ef49bcfsp52z6/AAD2dlsZ4Kxx4ENQnTc6-FNFa?dl=0

You can import it and try opening the image in DT for crash or generate the cache for the specific id in your database.

[jenshannoschwalm]

I vaguely remember such issues while working on image export for thumbs when cropping was involved.

[TurboGit]

I have debugged this a bit. There was two instances of demosaic in the XMP meaning that you have changed the also probably. Removing the last instance of demosaic fixed the issue. Do you remember what it was ? There was a new demosaic algo proposed recently and not integrated, if you tried it now that it is not merged it could explain the issue...

[jenshannoschwalm]

Strange. I can confirm @TurboGits analysis, setting the history< to before that second demosaicer is ok.

But i tested this on my rcd demosaicer branch which has not changed the DT_IOP_DEMOSAIC_RCD at all.

Have you been using the passthrough color demosaicer maybe?

[jenshannoschwalm]

If i run this using -d all i get this before setting history to the problematic demosaic history entry

^^^^^ modulegroups
          rawprepare 1
         temperature 3
            demosaic 8
                flip 16
            exposure 21
             colorin 28
     channelmixerrgb 29
           filmicrgb 48
            colorout 71
          finalscale 73, hidden
         overexposed 74, hidden
      rawoverexposed 75, hidden
               gamma 79, hidden
vvvvv

when clicking on that i get this

          rawprepare 1
         temperature 3
                flip 16
            exposure 21
             colorin 28
     channelmixerrgb 29
           filmicrgb 48
            colorout 71
          finalscale 73, hidden
         overexposed 74, hidden
      rawoverexposed 75, hidden
               gamma 79, hidden
vvvvv

which is clearly wrong as there is no demosaicer any mor

[chhil]

I don't recollect changing any demosaic setting.. I was copying pasting history to reproduce an issue i was having. There could be a possibility where history was copied from fuji edited image to a panasonic one.

[jenshannoschwalm]

Hhm, i will think and test ...

[jenshannoschwalm]

There is something more wrong with the xmp.

validation check failed in _iop_validate_params for type "float", field: g2
validation check failed in _iop_validate_params for type "dt_iop_temperature_params_t"

[johnny-bit]

wooo! i knew @dterrahe's work on validate params would pay :D

[jenshannoschwalm]

Ok, that xmp is really full of surprises :-) @TurboGit

1. I took the original xmp and set the history end to 11 to make sure the image loads fine.
2. I added some debugging code in the demosaic code reporting about sensor mismatches when committing parameters and indeed - as @chhil remembered - the problematic history step originated from an xtrans sensor now applied to a bayer image. That could be fixed in that code although i could not yet trigger any problem.
3. The big surprise was, if i have a look at the database the demosaic module was disabled. And this made me looking really stupid as the xmp clearly showed that and the debugging output also :-(

So - these questions arise

1. who / what condition lead to disabling a always-on module
2. If a module is always-on we should make sure there is one in history

[chhil]

Guys, thank you for your perseverance and getting to the bottom of it.

[TurboGit]

1. who / what condition lead to disabling a always-on module

That's the real question and the bug we need to be chasing indeed... But I have no idea :( Tried some scenario involving jpeg & history copy but without success !

[jenshannoschwalm]

I also tried combinations of non-raws / raws and combinations of simply wrong copy&paste.

It's very difficult to understand also because the xmp does not contain the dt version. That info might be helping in future cases like this.

So the pr is more of a life-belt but as we know all boats & ships must have them too :-)

[chhil]

Image 793 that was creating problems, no longer does so with the current master.

image 791/5556 (14.24%) (id:791) image 792/5556 (14.25%) (id:792) image 793/5556 (14.27%) (id:793) [dt_dev_pixelpipe_synch] alway-on moduledemosaic' found as disabled in history` image 794/5556 (14.29%) (id:794) image 795/5556 (14.31%) (id:795)

[TurboGit]

Perfect, the message is clear yet no idea on how to create such broken history.

[chhil]

@jenshannoschwalm Would you like me to run any form of test?

As I was going through the whole thread again, is it possible the history append v overwrite mode would have an impact on 2 demosaics being present in the xmp? I do append sometimes and sometimes overwrite and mistakenly at times I forget what setting was set in the lighttable history append/overwrite mode. It would be nice to have some indicator if I am in append or overwrite mode.

[ptilopteri]

the menu for history clearly shows the mode, append or overwrite, but you must look at it.

[chhil]

True, but when you are spending most of the time in darkroom the oversight is very possible. Let me see if I can assign the options a short-cut key.

[ptilopteri]

we all spend "most of our time" in darkroom mode. but copying history should be performed in lighttable view where it's controls exist.

if you are directly applying a copy of the history stack to the current image in darkroom, why not make a style and copy the style. a style does not need to be a permanent entity, they can be deleted. and you can assign a short cut key to them.

[chhil]

but copying history should be performed in lighttable view where it's controls exist.

Darkroom filmstrip allows for ctrl+c and ctrl+shift+c for a reason, I prefer doing it there and tweak it further.

why not make a style and copy the style.

Workflow does not require a style it's an overkill. If I see 5 images that need a similar treatmen then I copy and paste history on the filmstrip after editing one and history will vary across the set of images that are being copied from so making "n" styles is not feasible for this.

[jenshannoschwalm]

I think you have been reporting here and in #7060 three different problems

1. history stack data might have wrong en- disabled status (reported for demosaic). The keyboard shortcut problem is now fixed in master and there is a pr at least 'fixing' that at runtime.

2. you have reported "crippled" xmp files that could not be reimported because an existing xmp suggests valid information but some necessary modules are missing. That would be a blocker for 3.4 i guess.

3. some sort of file access problem that seems to be specific for your system (WSL)

For 1. and 2. we still have not found a way to trigger such a condition. It could of course be related to wrongful implementation of copy & paste of history. It would be very welcome to describe a way to reproduce TYA @TurboGit

[chhil]

Is there a query I can run that can tell me which images have demosaic turned off?