search

darktable-org / darktable

darktable is an open source photography workflow application and raw developer
https://www.darktable.org
GNU General Public License v3.0
9.77k stars 1.14k forks source link

git master : Debian 12 Bookworm : Building darktable triggers internal compiler error #14701

Closed KarlMagnusLarsson closed 1 year ago

KarlMagnusLarsson commented 1 year ago

Describe the bug

have done apt-get full-upgrade to Debian 12 Bookworm, from Debian 11.

When I try to build darktable git master, I trigger a gcc internal compiler error.

Steps to reproduce

git clone --recurse-submodules https://github.com/darktable-org/darktable.git
cd darktable
git submodule update --init
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX="$HOME/darktable-debian12/install" -DCMAKE_BUILD_TYPE=Release ..
make

Then I get internal compiler error: Illegal instruction

[ 20%] Building C object bin/CMakeFiles/lib_darktable.dir/common/iop_order.c.o
during GIMPLE pass: graphite
/home/magnus/camera2/test2/darktable-debian12/darktable/src/common/iop_order.c: In function ‘_insert_before.isra’:
/home/magnus/camera2/test2/darktable-debian12/darktable/src/common/iop_order.c:404:15: internal compiler error: Illegal instruction
  404 | static GList *_insert_before(GList *iop_order_list,
      |               ^~~~~~~~~~~~~~
0x7f1fc963ef8f ???
    ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x7f1fc962a189 __libc_start_call_main
    ../sysdeps/nptl/libc_start_call_main.h:58
0x7f1fc962a244 __libc_start_main_impl
    ../csu/libc-start.c:381
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
See <file:///usr/share/doc/gcc-12/README.Bugs> for instructions.
make[2]: *** [bin/CMakeFiles/lib_darktable.dir/build.make:726: bin/CMakeFiles/lib_darktable.dir/common/iop_order.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:2473: bin/CMakeFiles/lib_darktable.dir/all] Error 2
make: *** [Makefile:156: all] Error 2

Expected behavior

Darktable should complete the compile step.

Logfile | Screenshot | Screencast

No response

Commit

release-4.3.0-2518-gccb3310858

Where did you install darktable from?

self compiled

darktable version

release-4.3.0-2518-gccb3310858

What OS are you using?

Linux

What is the version of your OS?

Debian 12.0

Describe your system?

Are you using OpenCL GPU in darktable?

Yes

If yes, what is the GPU card and driver?

VIDIA Corporation TU104GL [Quadro RTX 4000], Nvidia Driver Version: 525.105.17-1 amd64

Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip

No response

TurboGit commented 1 year ago

As posted on pxls.us:

I was about to build darktable without issue on my side. As I’m up-to-date on Debian/sid I’m basically using bookworm:

$ gcc --version gcc (Debian 12.2.0-14) 12.2.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

TurboGit commented 1 year ago

Also tested with RelWithDebInfo & Release mode. No issue on my side.

KarlMagnusLarsson commented 1 year ago

Compiles with clang.

jpellegrini commented 1 year ago

I also have been unable to build darktable with GCC for some months now, and have been using clang instead. The error I get is precisely the one shown here. It looks like a bug in GCC, perhaps (?)

@TurboGit I am also using Debian sid, and due to a hardware failure, I have just re-installed the system, so this is a fresh Debian stable install that was upgraded to sid... Should be the same as yours! I suppose it may be something hardware or firmware specific (gcc-12 + Debian version of firmware blobs + specific hardware triggering the problem). I have here a AMD FX-8320E CPU; the firmware packages on my system (including, unfortunately, some non-free, which I really didn't want to have installed) are:

dpkg -l \*firmware\*|grep ii
ii  alsa-firmware-loaders  1.2.5-3      amd64        ALSA software loaders for specific hardware
ii  firmware-amd-graphics  20230210-5   all          Binary firmware for AMD/ATI graphics chips
ii  firmware-linux         20230210-5   all          Binary firmware for various drivers in the Linux kernel (metapackage)
ii  firmware-linux-free    20200122-1   all          Binary firmware for various drivers in the Linux kernel
ii  firmware-linux-nonfree 20230210-5   all          Binary firmware for various drivers in the Linux kernel (metapackage)
ii  firmware-misc-nonfree  20230210-5   all          Binary firmware for various drivers in the Linux kernel

The commands used to compile dt were:

rm -rf build
./build.sh --prefix /usr/local --build-type Release --install --sudo

I'm not sure what other information would be relevant.

LebedevRI commented 1 year ago

@KarlMagnusLarsson @jpellegrini What is your $ lscpu? That error almost surely means that said GCC itself has been built in a way that requires newer CPU than the one you have.

KarlMagnusLarsson commented 1 year ago

I am using Intel® Core™ i7-3770K × 8 on ASUSTeK COMPUTER INC. P8Z77-V (computer build 2013, so semi-old).

Debian 11 + selfcompiled gcc 12.2.0 (gnu/gcc standard code) worked fine when it comes to building darktable. Debian 12 + gcc (Debian 12.2.0-14) 12.2.0 does not compile darktable code (error as listed in issue). Debian 12 + Debian clang version 14.0.6 works fine. I can start darktable and import pictures.

I have tried to install this Debian package: intel-microcode/stable,now 3.20230512.1 amd64 [installed] , but this did not help gcc.

jpellegrini commented 1 year ago

@LebedevRI -- here it is:

$ lscpu

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         48 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  8
  On-line CPU(s) list:   0-7
Vendor ID:               AuthenticAMD
  Model name:            AMD FX-8320E Eight-Core Processor
    CPU family:          21
    Model:               2
    Thread(s) per core:  2
    Core(s) per socket:  4
    Socket(s):           1
    Stepping:            0
    Frequency boost:     enabled
    CPU(s) scaling MHz:  92%
    CPU max MHz:         3200.0000
    CPU min MHz:         1400.0000
    BogoMIPS:            6429.77
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq moni
                         tor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb cpb hw_pstate s
                         sbd ibpb vmmcall bmi1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold
Virtualization features: 
  Virtualization:        AMD-V
Caches (sum of all):     
  L1d:                   128 KiB (8 instances)
  L1i:                   256 KiB (4 instances)
  L2:                    8 MiB (4 instances)
  L3:                    8 MiB (1 instance)
NUMA:                    
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-7
Vulnerabilities:         
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Not affected
  Retbleed:              Mitigation; untrained return thunk; SMT vulnerable
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
  Srbds:                 Not affected
  Tsx async abort:       Not affected
LebedevRI commented 1 year ago

@jpellegrini when GCC crashes with internal compiler error: Illegal instruction, do any messages appear in $ dmesg? If so, can you post them here?

KarlMagnusLarsson commented 1 year ago

Hello @LebedevRI. The output of lscpu looks like this:

# lscpu
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         36 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  8
  On-line CPU(s) list:   0-7
Vendor ID:               GenuineIntel
  BIOS Vendor ID:        Intel
  Model name:            Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz
    BIOS Model name:     Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz To Be Filled By O.E.M. CPU @ 3.5GHz
    BIOS CPU family:     191
    CPU family:          6
    Model:               58
    Thread(s) per core:  2
    Core(s) per socket:  4
    Socket(s):           1
    Stepping:            9
    CPU(s) scaling MHz:  50%
    CPU max MHz:         3900.0000
    CPU min MHz:         1600.0000
    BogoMIPS:            7020.23
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe sysc
                         all nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes6
                         4 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx f16c rdrand lah
                         f_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida a
                         rat pln pts md_clear flush_l1d
Virtualization features: 
  Virtualization:        VT-x
Caches (sum of all):     
  L1d:                   128 KiB (4 instances)
  L1i:                   128 KiB (4 instances)
  L2:                    1 MiB (4 instances)
  L3:                    8 MiB (1 instance)
NUMA:                    
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-7
Vulnerabilities:         
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
  Mds:                   Mitigation; Clear CPU buffers; SMT vulnerable
  Meltdown:              Mitigation; PTI
  Mmio stale data:       Unknown: No mitigations
  Retbleed:              Not affected
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
  Srbds:                 Vulnerable: No microcode
  Tsx async abort:       Not affected

Regarding this:

That error almost surely means that said GCC itself has been built in a way that requires newer CPU than the one you have.

OK, I see. That could explain why gcc (Debian 12.2.0-14) 12.2.0 does not work. But I also tried to compile and bootstrap my own gnu/gcc 12.2.0 and use that to compile darktable and I get exactly the same error with my self compiled gcc 12.2.0.

jpellegrini commented 1 year ago

@jpellegrini when GCC crashes with internal compiler error: Illegal instruction, do any messages appear in $ dmesg? If so, can you post them here?

Nope. Not a single line is added to the dmesg output after compilation...

LebedevRI commented 1 year ago

@jpellegrini when GCC crashes with internal compiler error: Illegal instruction, do any messages appear in $ dmesg? If so, can you post them here?

Nope. Not a single line is added to the dmesg output after compilation...

Unfortunate. Can you then manually running the compilation command that fails (make VERBOSE=1 would show it), but under gdb, i.e.:

$ gdb /path/to/gcc
(gdb) run <rest of the run line>

and once it crashes, do

(gdb) layout asm

and show the output?

KarlMagnusLarsson commented 1 year ago

@LebedevRI

gdb /home/magnus/camera2/test2/darktable-gcc12/bin/gcc

and

run /home/magnus/camera2/test2/darktable-gcc12/bin/gcc -DAVIF_DLL -DDT_HAVE_SIGNAL_TRACE -DGDK_DISABLE_DEPRECATED -DGDK_VERSION_MIN_REQUIRED=GDK_VERSION_3_24 -DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_MIN_REQUIRED -DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_56 -DGTK_DISABLE_DEPRECATED -DGTK_DISABLE_SINGLE_INCLUDES -DHAVE_BUILTIN_CPU_SUPPORTS -DHAVE_CONFIG_H -DHAVE_GAME -DHAVE_GMIC -DHAVE_GPHOTO2 -DHAVE_GRAPHICSMAGICK -DHAVE_ICU -DHAVE_IMATH -DHAVE_ISO_CODES -DHAVE_KWALLET -DHAVE_LIBAVIF=1 -DHAVE_LIBEXIV2_WITH_ISOBMFF=1 -DHAVE_LIBHEIF=1 -DHAVE_LIBJXL -DHAVE_LIBRAW=1 -DHAVE_LIBSECRET -DHAVE_MAP -DHAVE_OPENCL -DHAVE_OPENEXR -DHAVE_OPENJPEG -DHAVE_OSMGPSMAP_110_OR_NEWER -DHAVE_OSMGPSMAP_NEWER_THAN_110 -DHAVE_PRINT -DHAVE_SQLITE_324_OR_NEWER -DHAVE_VISIBILITY -DHAVE_WEBP -DLIBHEIF_EXPORTS -DLIBRAW_NODLL -DNATIVE_ARCH -DSQLITE_CORE -DSQLITE_ENABLE_ICU -DUSE_COLORDGTK -DUSE_LUA -D_XOPEN_SOURCE=700 -Dlib_darktable_EXPORTS -I/home/magnus/camera2/test2/darktable-gcc12/darktable/build/bin -I/home/magnus/camera2/test2/darktable-gcc12/darktable/src -I/home/magnus/camera2/test2/darktable-gcc12/darktable/src/external/LuaAutoC -I/home/magnus/camera2/test2/darktable-gcc12/darktable/src/external/whereami/src -I/home/magnus/camera2/test2/darktable-gcc12/darktable/src/external/LibRaw -isystem /home/magnus/camera2/test2/darktable-gcc12/darktable/src/external -isystem /home/magnus/camera2/test2/darktable-gcc12/darktable/src/external/OpenCL -isystem /usr/include/glib-2.0 -isystem /usr/lib/x86_64-linux-gnu/glib-2.0/include -isystem /usr/include/gtk-3.0 -isystem /usr/include/pango-1.0 -isystem /usr/include/harfbuzz -isystem /usr/include/freetype2 -isystem /usr/include/libpng16 -isystem /usr/include/libmount -isystem /usr/include/blkid -isystem /usr/include/fribidi -isystem /usr/include/cairo -isystem /usr/include/pixman-1 -isystem /usr/include/gdk-pixbuf-2.0 -isystem /usr/include/gio-unix-2.0 -isystem /usr/include/atk-1.0 -isystem /usr/include/at-spi2-atk/2.0 -isystem /usr/include/at-spi-2.0 -isystem /usr/include/dbus-1.0 -isystem /usr/lib/x86_64-linux-gnu/dbus-1.0/include -isystem /usr/include/libxml2 -isystem /usr/include/lensfun -isystem /usr/include/librsvg-2.0 -isystem /usr/include/json-glib-1.0 -isystem /usr/include/openjpeg-2.5 -isystem /usr/include/libsecret-1 -isystem /usr/include/GraphicsMagick -isystem /usr/include/lua5.4 -isystem /usr/include/osmgpsmap-1.0 -isystem /usr/include/libsoup-2.4 -isystem /usr/include/colord-1 -isystem /home/magnus/camera2/test2/darktable-gcc12/include/Imath -isystem /home/magnus/camera2/test2/darktable-gcc12/include/OpenEXR -Wall -Wformat -Wformat-security -Wshadow -Wtype-limits -Wvla -Wold-style-declaration -Wmaybe-uninitialized -Wno-unknown-pragmas -Wno-error=varargs -Wno-format-truncation -Wno-error=address-of-packed-member -fopenmp -march=native -msse2 -g -mfpmath=sse -O3 -DNDEBUG -O3 -ffast-math -fno-finite-math-only -fexpensive-optimizations -fPIC -Werror -Wfatal-errors -std=c99 -MD -MT bin/CMakeFiles/lib_darktable.dir/common/iop_order.c.o -MF CMakeFiles/lib_darktable.dir/common/iop_order.c.o.d -o CMakeFiles/lib_darktable.dir/common/iop_order.c.o -c /home/magnus/camera2/test2/darktable-gcc12/darktable/src/common/iop_order.c

(gdb) layout asm gives me this:

Screenshot from 2023-06-12 20-21-33

My apologies for the screenshot. I am sure there is a better way.

This is using my own gcc 12.2.0 from gnu/gcc code. The debian gcc 12.2.0 did not have any symbols in it.

jpellegrini commented 1 year ago

(gdb) layout asm

I did all that, but gdb said there was no assembly to show. The command line that I used (which I copied from the compilation process) did have the -g flag... Not sure what happens.

LebedevRI commented 1 year ago

Hm, that is not it. https://gcc.gnu.org/wiki/DebuggingGCC suggests gcc <parameters> -wrapper gdb,--args, can you try that instead?

KarlMagnusLarsson commented 1 year ago

/home/magnus/camera2/test2/darktable-gcc12/bin/gcc -DAVIF_DLL -DDT_HAVE_SIGNAL_TRACE -DGDK_DISABLE_DEPRECATED -DGDK_VERSION_MIN_REQUIRED=GDK_VERSION_3_24 -DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_MIN_REQUIRED -DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_56 -DGTK_DISABLE_DEPRECATED -DGTK_DISABLE_SINGLE_INCLUDES -DHAVE_BUILTIN_CPU_SUPPORTS -DHAVE_CONFIG_H -DHAVE_GAME -DHAVE_GMIC -DHAVE_GPHOTO2 -DHAVE_GRAPHICSMAGICK -DHAVE_ICU -DHAVE_IMATH -DHAVE_ISO_CODES -DHAVE_KWALLET -DHAVE_LIBAVIF=1 -DHAVE_LIBEXIV2_WITH_ISOBMFF=1 -DHAVE_LIBHEIF=1 -DHAVE_LIBJXL -DHAVE_LIBRAW=1 -DHAVE_LIBSECRET -DHAVE_MAP -DHAVE_OPENCL -DHAVE_OPENEXR -DHAVE_OPENJPEG -DHAVE_OSMGPSMAP_110_OR_NEWER -DHAVE_OSMGPSMAP_NEWER_THAN_110 -DHAVE_PRINT -DHAVE_SQLITE_324_OR_NEWER -DHAVE_VISIBILITY -DHAVE_WEBP -DLIBHEIF_EXPORTS -DLIBRAW_NODLL -DNATIVE_ARCH -DSQLITE_CORE -DSQLITE_ENABLE_ICU -DUSE_COLORDGTK -DUSE_LUA -D_XOPEN_SOURCE=700 -Dlib_darktable_EXPORTS -I/home/magnus/camera2/test2/darktable-gcc12/darktable/build/bin -I/home/magnus/camera2/test2/darktable-gcc12/darktable/src -I/home/magnus/camera2/test2/darktable-gcc12/darktable/src/external/LuaAutoC -I/home/magnus/camera2/test2/darktable-gcc12/darktable/src/external/whereami/src -I/home/magnus/camera2/test2/darktable-gcc12/darktable/src/external/LibRaw -isystem /home/magnus/camera2/test2/darktable-gcc12/darktable/src/external -isystem /home/magnus/camera2/test2/darktable-gcc12/darktable/src/external/OpenCL -isystem /usr/include/glib-2.0 -isystem /usr/lib/x86_64-linux-gnu/glib-2.0/include -isystem /usr/include/gtk-3.0 -isystem /usr/include/pango-1.0 -isystem /usr/include/harfbuzz -isystem /usr/include/freetype2 -isystem /usr/include/libpng16 -isystem /usr/include/libmount -isystem /usr/include/blkid -isystem /usr/include/fribidi -isystem /usr/include/cairo -isystem /usr/include/pixman-1 -isystem /usr/include/gdk-pixbuf-2.0 -isystem /usr/include/gio-unix-2.0 -isystem /usr/include/atk-1.0 -isystem /usr/include/at-spi2-atk/2.0 -isystem /usr/include/at-spi-2.0 -isystem /usr/include/dbus-1.0 -isystem /usr/lib/x86_64-linux-gnu/dbus-1.0/include -isystem /usr/include/libxml2 -isystem /usr/include/lensfun -isystem /usr/include/librsvg-2.0 -isystem /usr/include/json-glib-1.0 -isystem /usr/include/openjpeg-2.5 -isystem /usr/include/libsecret-1 -isystem /usr/include/GraphicsMagick -isystem /usr/include/lua5.4 -isystem /usr/include/osmgpsmap-1.0 -isystem /usr/include/libsoup-2.4 -isystem /usr/include/colord-1 -isystem /home/magnus/camera2/test2/darktable-gcc12/include/Imath -isystem /home/magnus/camera2/test2/darktable-gcc12/include/OpenEXR -Wall -Wformat -Wformat-security -Wshadow -Wtype-limits -Wvla -Wold-style-declaration -Wmaybe-uninitialized -Wno-unknown-pragmas -Wno-error=varargs -Wno-format-truncation -Wno-error=address-of-packed-member -fopenmp -march=native -msse2 -g -mfpmath=sse -O3 -DNDEBUG -O3 -ffast-math -fno-finite-math-only -fexpensive-optimizations -fPIC -Werror -Wfatal-errors -std=c99 -MD -MT bin/CMakeFiles/lib_darktable.dir/common/iop_order.c.o -MF CMakeFiles/lib_darktable.dir/common/iop_order.c.o.d -o CMakeFiles/lib_darktable.dir/common/iop_order.c.o -c /home/magnus/camera2/test2/darktable-gcc12/darktable/src/common/iop_order.c -wrapper gdb,--args home/magnus/camera2/test2/darktable-gcc12/bin/cc1

(gdb) run

Starting program: /home/magnus/camera2/test2/darktable-gcc12/libexec/gcc/x86_64-pc-linux-gnu/12.2.0/cc1 -quiet -I /home/magnus/camera2/test2/darktable-gcc12/darktable/build/bin -I /home/magnus/camera2/test2/darktable-gcc12/darktable/src -I /home/magnus/camera2/test2/darktable-gcc12/darktable/src/external/LuaAutoC -I /home/magnus/camera2/test2/darktable-gcc12/darktable/src/external/whereami/src -I /home/magnus/camera2/test2/darktable-gcc12/darktable/src/external/LibRaw -imultiarch x86_64-linux-gnu -MD CMakeFiles/lib_darktable.dir/common/iop_order.c.d -MF CMakeFiles/lib_darktable.dir/common/iop_order.c.o.d -MT bin/CMakeFiles/lib_darktable.dir/common/iop_order.c.o -D_REENTRANT -D AVIF_DLL -D DT_HAVE_SIGNAL_TRACE -D GDK_DISABLE_DEPRECATED -D GDK_VERSION_MIN_REQUIRED=GDK_VERSION_3_24 -D GLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_MIN_REQUIRED -D GLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_56 -D GTK_DISABLE_DEPRECATED -D GTK_DISABLE_SINGLE_INCLUDES -D HAVE_BUILTIN_CPU_SUPPORTS -D HAVE_CONFIG_H -D HAVE_GAME -D HAVE_GMIC -D HAVE_GPHOTO2 -D HAVE_GRAPHICSMAGICK -D HAVE_ICU -D HAVE_IMATH -D HAVE_ISO_CODES -D HAVE_KWALLET -D HAVE_LIBAVIF=1 -D HAVE_LIBEXIV2_WITH_ISOBMFF=1 -D HAVE_LIBHEIF=1 -D HAVE_LIBJXL -D HAVE_LIBRAW=1 -D HAVE_LIBSECRET -D HAVE_MAP -D HAVE_OPENCL -D HAVE_OPENEXR -D HAVE_OPENJPEG -D HAVE_OSMGPSMAP_110_OR_NEWER -D HAVE_OSMGPSMAP_NEWER_THAN_110 -D HAVE_PRINT -D HAVE_SQLITE_324_OR_NEWER -D HAVE_VISIBILITY -D HAVE_WEBP -D LIBHEIF_EXPORTS -D LIBRAW_NODLL -D NATIVE_ARCH -D SQLITE_CORE -D SQLITE_ENABLE_ICU -D USE_COLORDGTK -D USE_LUA -D _XOPEN_SOURCE=700 -D lib_darktable_EXPORTS -D NDEBUG -isystem /home/magnus/camera2/test2/darktable-gcc12/darktable/src/external -isystem /home/magnus/camera2/test2/darktable-gcc12/darktable/src/external/OpenCL -isystem /usr/include/glib-2.0 -isystem /usr/lib/x86_64-linux-gnu/glib-2.0/include -isystem /usr/include/gtk-3.0 -isystem /usr/include/pango-1.0 -isystem /usr/include/harfbuzz -isystem /usr/include/freetype2 -isystem /usr/include/libpng16 -isystem /usr/include/libmount -isystem /usr/include/blkid -isystem /usr/include/fribidi -isystem /usr/include/cairo -isystem /usr/include/pixman-1 -isystem /usr/include/gdk-pixbuf-2.0 -isystem /usr/include/gio-unix-2.0 -isystem /usr/include/atk-1.0 -isystem /usr/include/at-spi2-atk/2.0 -isystem /usr/include/at-spi-2.0 -isystem /usr/include/dbus-1.0 -isystem /usr/lib/x86_64-linux-gnu/dbus-1.0/include -isystem /usr/include/libxml2 -isystem /usr/include/lensfun -isystem /usr/include/librsvg-2.0 -isystem /usr/include/json-glib-1.0 -isystem /usr/include/openjpeg-2.5 -isystem /usr/include/libsecret-1 -isystem /usr/include/GraphicsMagick -isystem /usr/include/lua5.4 -isystem /usr/include/osmgpsmap-1.0 -isystem /usr/include/libsoup-2.4 -isystem /usr/include/colord-1 -isystem /home/magnus/camera2/test2/darktable-gcc12/include/Imath -isystem /home/magnus/camera2/test2/darktable-gcc12/include/OpenEXR /home/magnus/camera2/test2/darktable-gcc12/darktable/src/common/iop_order.c -march=ivybridge -mmmx -mpopcnt -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -mavx -mno-avx2 -mno-sse4a -mno-fma4 -mno-xop -mno-fma -mno-avx512f -mno-bmi -mno-bmi2 -maes -mpclmul -mno-avx512vl -mno-avx512bw -mno-avx512dq -mno-avx512cd -mno-avx512er -mno-avx512pf -mno-avx512vbmi -mno-avx512ifma -mno-avx5124vnniw -mno-avx5124fmaps -mno-avx512vpopcntdq -mno-avx512vbmi2 -mno-gfni -mno-vpclmulqdq -mno-avx512vnni -mno-avx512bitalg -mno-avx512bf16 -mno-avx512vp2intersect -mno-3dnow -mno-adx -mno-abm -mno-cldemote -mno-clflushopt -mno-clwb -mno-clzero -mcx16 -mno-enqcmd -mf16c -mfsgsbase -mfxsr -mno-hle -msahf -mno-lwp -mno-lzcnt -mno-movbe -mno-movdir64b -mno-movdiri -mno-mwaitx -mno-pconfig -mno-pku -mno-prefetchwt1 -mno-prfchw -mno-ptwrite -mno-rdpid -mrdrnd -mno-rdseed -mno-rtm -mno-serialize -mno-sgx -mno-sha -mno-shstk -mno-tbm -mno-tsxldtrk -mno-vaes -mno-waitpkg -mno-wbnoinvd -mxsave -mno-xsavec -mxsaveopt -mno-xsaves -mno-amx-tile -mno-amx-int8 -mno-amx-bf16 -mno-uintr -mno-hreset -mno-kl -mno-widekl -mno-avxvnni -mno-avx512fp16 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=ivybridge -quiet -dumpdir CMakeFiles/lib_darktable.dir/common/ -dumpbase iop_order.c.c -dumpbase-ext .c -msse2 -mfpmath=sse -g -O3 -O3 -Wall -Wformat=1 -Wformat-security -Wshadow -Wtype-limits -Wvla -Wold-style-declaration -Wmaybe-uninitialized -Wno-unknown-pragmas -Wno-error=varargs -Wformat-truncation=0 -Wno-error=address-of-packed-member -Werror -Wfatal-errors -std=c99 -fopenmp -ffast-math -fno-finite-math-only -fexpensive-optimizations -fPIC -o /tmp/cccAqOcr.s [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction. 0x00007ffff7ce565a in isl_hash_table_init () from /lib/x86_64-linux-gnu/libisl.so.23

layout asm Screenshot from 2023-06-12 21-00-57

jpellegrini commented 1 year ago

Ok, I did 

gcc-12  -DAVIF_DLL -DDT_HAVE_SIGNAL_TRACE -DGDK_DISABLE_DEPRECATED -DGDK_VERSION_MIN_REQUIRED=GDK_VERSION_3_24 -DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_MIN_REQUIRED -DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_56 -DGTK_DISABLE_DEPRECATED -DGTK_DISABLE_SINGLE_INCLUDES -DHAVE_BUILTIN_CPU_SUPPORTS -DHAVE_CONFIG_H -DHAVE_GAME -DHAVE_GPHOTO2 -DHAVE_GRAPHICSMAGICK -DHAVE_ICU -DHAVE_IMATH -DHAVE_ISO_CODES -DHAVE_KWALLET -DHAVE_LIBAVIF=1 -DHAVE_LIBEXIV2_WITH_ISOBMFF=1 -DHAVE_LIBHEIF=1 -DHAVE_LIBJXL -DHAVE_LIBRAW=1 -DHAVE_LIBSECRET -DHAVE_MAP -DHAVE_OPENCL -DHAVE_OPENEXR -DHAVE_OPENJPEG -DHAVE_OSMGPSMAP_110_OR_NEWER -DHAVE_OSMGPSMAP_NEWER_THAN_110 -DHAVE_PRINT -DHAVE_SQLITE_324_OR_NEWER -DHAVE_VISIBILITY -DHAVE_WEBP -DLIBHEIF_EXPORTS -DLIBRAW_NODLL -DNATIVE_ARCH -DSQLITE_CORE -DSQLITE_ENABLE_ICU -DUSE_COLORDGTK -DUSE_LUA -D_XOPEN_SOURCE=700 -Dlib_darktable_EXPORTS -I/home/jeronimo/pkg/darktable/build/bin -I/home/jeronimo/pkg/darktable/src -I/home/jeronimo/pkg/darktable/src/external/LuaAutoC -I/home/jeronimo/pkg/darktable/src/external/whereami/src -I/home/jeronimo/pkg/darktable/src/external/LibRaw -isystem /home/jeronimo/pkg/darktable/src/external -isystem /home/jeronimo/pkg/darktable/src/external/OpenCL -isystem /usr/include/glib-2.0 -isystem /usr/lib/x86_64-linux-gnu/glib-2.0/include -isystem /usr/include/gtk-3.0 -isystem /usr/include/pango-1.0 -isystem /usr/include/harfbuzz -isystem /usr/include/freetype2 -isystem /usr/include/libpng16 -isystem /usr/include/libmount -isystem /usr/include/blkid -isystem /usr/include/fribidi -isystem /usr/include/cairo -isystem /usr/include/pixman-1 -isystem /usr/include/gdk-pixbuf-2.0 -isystem /usr/include/gio-unix-2.0 -isystem /usr/include/atk-1.0 -isystem /usr/include/at-spi2-atk/2.0 -isystem /usr/include/at-spi-2.0 -isystem /usr/include/dbus-1.0 -isystem /usr/lib/x86_64-linux-gnu/dbus-1.0/include -isystem /usr/include/libxml2 -isystem /usr/include/lensfun -isystem /usr/include/librsvg-2.0 -isystem /usr/include/json-glib-1.0 -isystem /usr/include/openjpeg-2.5 -isystem /usr/include/libsecret-1 -isystem /usr/include/GraphicsMagick -isystem /usr/include/lua5.4 -isystem /usr/include/osmgpsmap-1.0 -isystem /usr/include/libsoup-2.4 -isystem /usr/include/colord-1 -isystem /usr/include/Imath -isystem /usr/include/OpenEXR -Wall -Wformat -Wformat-security -Wshadow -Wtype-limits -Wvla -Wold-style-declaration -Wmaybe-uninitialized -Wno-unknown-pragmas -Wno-error=varargs -Wno-format-truncation -Wno-error=address-of-packed-member -fopenmp -march=native -msse2 -g -mfpmath=sse -O3 -DNDEBUG -O3 -ffast-math -fno-finite-math-only -fexpensive-optimizations -fPIC   -Werror -Wfatal-errors -std=c99 -MD -MT bin/CMakeFiles/lib_darktable.dir/common/iop_order.c.o -MF CMakeFiles/lib_darktable.dir/common/iop_order.c.o.d -o CMakeFiles/lib_darktable.dir/common/iop_order.c.o -c /home/jeronimo/pkg/darktable/src/common/iop_order.c -wrapper gdb,--args

Then run:

(gdb) run
Starting program: /usr/lib/gcc/x86_64-linux-gnu/12/cc1 -quiet -I /home/jeronimo/pkg/darktable/build/bin -I /home/jeronimo/pkg/darktable/src -I /home/jeronimo/pkg/darktable/src/external/LuaAutoC -I /home/jeronimo/pkg/darktable/src/external/whereami/src -I /home/jeronimo/pkg/darktable/src/external/LibRaw -imultiarch x86_64-linux-gnu -MD CMakeFiles/lib_darktable.dir/common/iop_order.c.d -MF CMakeFiles/lib_darktable.dir/common/iop_order.c.o.d -MT bin/CMakeFiles/lib_darktable.dir/common/iop_order.c.o -D_REENTRANT -D AVIF_DLL -D DT_HAVE_SIGNAL_TRACE -D GDK_DISABLE_DEPRECATED -D GDK_VERSION_MIN_REQUIRED=GDK_VERSION_3_24 -D GLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_MIN_REQUIRED -D GLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_56 -D GTK_DISABLE_DEPRECATED -D GTK_DISABLE_SINGLE_INCLUDES -D HAVE_BUILTIN_CPU_SUPPORTS -D HAVE_CONFIG_H -D HAVE_GAME -D HAVE_GPHOTO2 -D HAVE_GRAPHICSMAGICK -D HAVE_ICU -D HAVE_IMATH -D HAVE_ISO_CODES -D HAVE_KWALLET -D HAVE_LIBAVIF=1 -D HAVE_LIBEXIV2_WITH_ISOBMFF=1 -D HAVE_LIBHEIF=1 -D HAVE_LIBJXL -D HAVE_LIBRAW=1 -D HAVE_LIBSECRET -D HAVE_MAP -D HAVE_OPENCL -D HAVE_OPENEXR -D HAVE_OPENJPEG -D HAVE_OSMGPSMAP_110_OR_NEWER -D HAVE_OSMGPSMAP_NEWER_THAN_110 -D HAVE_PRINT -D HAVE_SQLITE_324_OR_NEWER -D HAVE_VISIBILITY -D HAVE_WEBP -D LIBHEIF_EXPORTS -D LIBRAW_NODLL -D NATIVE_ARCH -D SQLITE_CORE -D SQLITE_ENABLE_ICU -D USE_COLORDGTK -D USE_LUA -D _XOPEN_SOURCE=700 -D lib_darktable_EXPORTS -D NDEBUG -isystem /home/jeronimo/pkg/darktable/src/external -isystem /home/jeronimo/pkg/darktable/src/external/OpenCL -isystem /usr/include/glib-2.0 -isystem /usr/lib/x86_64-linux-gnu/glib-2.0/include -isystem /usr/include/gtk-3.0 -isystem /usr/include/pango-1.0 -isystem /usr/include/harfbuzz -isystem /usr/include/freetype2 -isystem /usr/include/libpng16 -isystem /usr/include/libmount -isystem /usr/include/blkid -isystem /usr/include/fribidi -isystem /usr/include/cairo -isystem /usr/include/pixman-1 -isystem /usr/include/gdk-pixbuf-2.0 -isystem /usr/include/gio-unix-2.0 -isystem /usr/include/atk-1.0 -isystem /usr/include/at-spi2-atk/2.0 -isystem /usr/include/at-spi-2.0 -isystem /usr/include/dbus-1.0 -isystem /usr/lib/x86_64-linux-gnu/dbus-1.0/include -isystem /usr/include/libxml2 -isystem /usr/include/lensfun -isystem /usr/include/librsvg-2.0 -isystem /usr/include/json-glib-1.0 -isystem /usr/include/openjpeg-2.5 -isystem /usr/include/libsecret-1 -isystem /usr/include/GraphicsMagick -isystem /usr/include/lua5.4 -isystem /usr/include/osmgpsmap-1.0 -isystem /usr/include/libsoup-2.4 -isystem /usr/include/colord-1 -isystem /usr/include/Imath -isystem /usr/include/OpenEXR /home/jeronimo/pkg/darktable/src/common/iop_order.c -march=bdver2 -mmmx -mpopcnt -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -mavx -mno-avx2 -msse4a -mfma4 -mxop -mfma -mno-avx512f -mbmi -mno-bmi2 -maes -mpclmul -mno-avx512vl -mno-avx512bw -mno-avx512dq -mno-avx512cd -mno-avx512er -mno-avx512pf -mno-avx512vbmi -mno-avx512ifma -mno-avx5124vnniw -mno-avx5124fmaps -mno-avx512vpopcntdq -mno-avx512vbmi2 -mno-gfni -mno-vpclmulqdq -mno-avx512vnni -mno-avx512bitalg -mno-avx512bf16 -mno-avx512vp2intersect -mno-3dnow -mno-adx -mabm -mno-cldemote -mno-clflushopt -mno-clwb -mno-clzero -mcx16 -mno-enqcmd -mf16c -mno-fsgsbase -mfxsr -mno-hle -msahf -mno-lwp -mlzcnt -mno-movbe -mno-movdir64b -mno-movdiri -mno-mwaitx -mno-pconfig -mno-pku -mno-prefetchwt1 -mprfchw -mno-ptwrite -mno-rdpid -mno-rdrnd -mno-rdseed -mno-rtm -mno-serialize -mno-sgx -mno-sha -mno-shstk -mtbm -mno-tsxldtrk -mno-vaes -mno-waitpkg -mno-wbnoinvd -mxsave -mno-xsavec -mno-xsaveopt -mno-xsaves -mno-amx-tile -mno-amx-int8 -mno-amx-bf16 -mno-uintr -mno-hreset -mno-kl -mno-widekl -mno-avxvnni -mno-avx512fp16 --param l1-cache-size=16 --param l1-cache-line-size=64 --param l2-cache-size=2048 -mtune=bdver2 -quiet -dumpdir CMakeFiles/lib_darktable.dir/common/ -dumpbase iop_order.c.c -dumpbase-ext .c -msse2 -mfpmath=sse -g -O3 -O3 -Wall -Wformat=1 -Wformat-security -Wshadow -Wtype-limits -Wvla -Wold-style-declaration -Wmaybe-uninitialized -Wno-unknown-pragmas -Wno-error=varargs -Wformat-truncation=0 -Wno-error=address-of-packed-member -Werror -Wfatal-errors -std=c99 -fopenmp -ffast-math -fno-finite-math-only -fexpensive-optimizations -fPIC -fasynchronous-unwind-tables -o /tmp/ccCYGzcj.s

I got something a bit different from @KarlMagnusLarsson :

Program received signal SIGILL, Illegal instruction.
0x00007ffff7ce5675 in isl_hash_table_init () from /lib/x86_64-linux-gnu/libisl.so.23
(gdb) where
#0  0x00007ffff7ce5675 in isl_hash_table_init () from /lib/x86_64-linux-gnu/libisl.so.23
#1  0x00007ffff7cd12a7 in isl_ctx_alloc_with_options () from /lib/x86_64-linux-gnu/libisl.so.23
#2  0x00000000017f4c38 in graphite_transform_loops() ()
#3  0x00000000017f6271 in ?? ()
#4  0x0000000000b1ed5b in execute_one_pass(opt_pass*) ()
#5  0x0000000000b1f5b0 in ?? ()
#6  0x0000000000b1f5c2 in ?? ()
#7  0x0000000000b1f5c2 in ?? ()
#8  0x0000000000b1f5c2 in ?? ()
#9  0x0000000000b1f5e9 in execute_pass_list(function*, opt_pass*) ()
#10 0x00000000007e8968 in cgraph_node::expand() ()
#11 0x00000000007e9e6f in ?? ()
#12 0x00000000007ec378 in symbol_table::finalize_compilation_unit() ()
#13 0x0000000000bedb90 in ?? ()
#14 0x0000000000674ebe in toplev::main(int, char**) ()
#15 0x000000000067655b in main ()

Then, layout asm shows (not the same instruction, it seems!):

gdb-output

LebedevRI commented 1 year ago

@KarlMagnusLarsson yup, there you go: it's blsr, which is a BMI1 instruction: https://www.felixcloutier.com/x86/blsr And both your lscpu (lack of bmi in the output), and -mno-bmi -mno-bmi2 in that command line, tell you that said instruction set is not supported on your CPU.

Why GCC is built in a way that requires BMI1 is their (or rather, debian's) bug. (it is likely built with -march=x86-64-v2, which requires BMI1)

LebedevRI commented 1 year ago

@jpellegrini for you it's shlx, which is from BMI2 instruction set, which your CPU does not support as per lscpu et al. So gcc/libisl is built with -march=x86-64-v3 ? That is downright stupid.

You all should really report this to debian gcc maintainers...

LebedevRI commented 1 year ago

@jpellegrini @KarlMagnusLarsson thank you!

jpellegrini commented 1 year ago

You all should really report this to debian gcc maintainers...

Sure! I just download the sources of GCC-12, and it seems that the Debian build script tries to guess the flags, and somehow got it wrong.

KarlMagnusLarsson commented 1 year ago

@LebedevRI, thank you for the feedback.

Why GCC is built in a way that requires BMI1 is their (or rather, debian's) bug. (it is likely built with -march=x86-64-v2, which requires BMI1)

Which -march= works for my Intel® Core™ i7-3770K × 8?

I am asking, since I can rebuild my own gcc-12.2.0 from gnu/gcc source. I am not using Debian's build script.

LebedevRI commented 1 year ago

-march=native is always a safe bet.

KarlMagnusLarsson commented 1 year ago

export CFLAGS="-march=native" export CXXFLAGS="-march=native" when building own gcc-12-2-0 did not help. Same error when using resulting compiler.

../configure \ --prefix=/home/magnus/camera2/test2/darktable-gcc12 \ --disable-multilib \ --with-system-zlib \ --enable-default-pie \ --enable-default-ssp \ --enable-languages=c,c++

during GIMPLE pass: graphite
/home/magnus/camera2/test2/darktable-gcc12/darktable/src/common/iop_profile.c: In function ‘_transform_from_to_rgb_lab_lcms2’:
/home/magnus/camera2/test2/darktable-gcc12/darktable/src/common/iop_profile.c:72:13: internal compiler error: Illegal instruction
   72 | static void _transform_from_to_rgb_lab_lcms2(const float *const image_in,
LebedevRI commented 1 year ago

You probably want to either just remove libisl23 apt package, or at least ensure (LD_LIBRARY_PATH?) that your new-built gcc is using it's own, newly-built libisl.so*, not the system one.

KarlMagnusLarsson commented 1 year ago

You probably want to either just remove libisl23 apt package, or at least ensure (LD_LIBRARY_PATH?) that your new-built gcc is using it's own, newly-built libisl.so*, not the system one.

Hello, @LebedevRI, I do not think there is a gcc internal libisl available. I already point start of LD_LIBRARY_PATH to my own gcc libs. I need isl for Graphite optimizations. What I could do is to build my own local isl and point to that.

KarlMagnusLarsson commented 1 year ago

@LebedevRI , @jpellegrini Works for me.

  1. Build gcc-12-2-0 from gnu/gcc code with export CFLAGS="-march=native" and export CXXFLAGS="-march=native" to avoid the BMI1 instruction pointed out by @LebedevRI. I bootstrapped local gcc from debian gcc 12.2.0.
  2. Build ISL to get a local libisl to avoid the shlx instruction pointed out by @LebedevRI in libisl23.so from Debian. EDIT:same flags as for gcc or unset flags work. I did ./autogen.sh, configure, make. I built libisl v0.26 with my local gcc compiler.

With my local gcc and local libisl I can build and run darktable.

doko42 commented 1 year ago

The Debian GCC packages for x86_64-linux are built without any arch settings, meaning x86-86-v1. You can see how GCC is configured by calling gcc -v.

The dt build script doesn't work for me, asking for LLVM as well, so I cannot check.

What I already see in the build script so far is: -- Performing Test _MSSE2 -- Performing Test _MSSE2 - Success -- Building SSE2-optimized codepaths: ON

so dt is playing around with some build flags on it's own.

Please point me to the command line and the preprocessed source which is supposed to generate any instructions for a higher baseline.

LebedevRI commented 1 year ago

@doko42 hi! Please note that the "illegal instruction" is encountered in libisl, as you can see in https://github.com/darktable-org/darktable/issues/14701#issuecomment-1587908160 and https://github.com/darktable-org/darktable/issues/14701#issuecomment-1587908160, this really can't be caused by whatever flags darktable itself compiles with.

LebedevRI commented 1 year ago

@doko42, FWIW, i can "reproduce" on a clean debian sid:

$ wget -q http://ftp.us.debian.org/debian/pool/main/i/isl/libisl23_0.26-2_amd64.deb
$ dpkg-deb -xv libisl23_0.26-2_amd64.deb zzzz
./
./usr/
./usr/lib/
./usr/lib/x86_64-linux-gnu/
./usr/lib/x86_64-linux-gnu/libisl.so.23.3.0
./usr/share/
./usr/share/doc/
./usr/share/doc/libisl23/
./usr/share/doc/libisl23/changelog.Debian.gz
./usr/share/doc/libisl23/changelog.gz
./usr/share/doc/libisl23/copyright
./usr/share/gdb/
./usr/share/gdb/auto-load/
./usr/share/gdb/auto-load/usr/
./usr/share/gdb/auto-load/usr/lib/
./usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/
./usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libisl.so.23.3.0-gdb.py
./usr/lib/x86_64-linux-gnu/libisl.so.23
$ objdump --disassemble=isl_hash_table_init zzzz/usr/lib/x86_64-linux-gnu/libisl.so.23.3.0

zzzz/usr/lib/x86_64-linux-gnu/libisl.so.23.3.0:     file format elf64-x86-64

Disassembly of section .init:

Disassembly of section .plt:

Disassembly of section .plt.got:

Disassembly of section .text:

00000000000e99a0 <isl_hash_table_init@@Base>:
   e99a0:       48 85 f6                test   %rsi,%rsi
   e99a3:       74 6e                   je     e9a13 <isl_hash_table_init@@Base+0x73>
   e99a5:       c5 f9 6f 0d 03 7e 0c    vmovdqa 0xc7e03(%rip),%xmm1        # 1b17b0 <isl_cell_foreach_simplex@@Base+0x38f0>
   e99ac:       00 
   e99ad:       c5 f9 6e c2             vmovd  %edx,%xmm0
   e99b1:       53                      push   %rbx
   e99b2:       48 89 f3                mov    %rsi,%rbx
   e99b5:       ba ab aa aa aa          mov    $0xaaaaaaab,%edx
   e99ba:       c4 e2 79 3d c1          vpmaxsd %xmm1,%xmm0,%xmm0
   e99bf:       c5 f9 7e c0             vmovd  %xmm0,%eax
   e99c3:       8d 04 85 04 00 00 00    lea    0x4(,%rax,4),%eax
   e99ca:       48 0f af c2             imul   %rdx,%rax
   e99ce:       48 c1 e8 21             shr    $0x21,%rax
   e99d2:       ff c8                   dec    %eax
   e99d4:       0f 1f 40 00             nopl   0x0(%rax)
   e99d8:       89 c2                   mov    %eax,%edx
   e99da:       c4 e2 78 f3 c8          blsr   %eax,%eax
   e99df:       75 f7                   jne    e99d8 <isl_hash_table_init@@Base+0x38>
   e99e1:       01 d2                   add    %edx,%edx
   e99e3:       c7 43 04 00 00 00 00    movl   $0x0,0x4(%rbx)
   e99ea:       be 01 00 00 00          mov    $0x1,%esi
   e99ef:       f3 0f bc d2             tzcnt  %edx,%edx
   e99f3:       89 13                   mov    %edx,(%rbx)
   e99f5:       c4 e2 69 f7 f6          shlx   %edx,%esi,%esi
   e99fa:       ba 10 00 00 00          mov    $0x10,%edx
   e99ff:       48 63 f6                movslq %esi,%rsi
   e9a02:       e8 49 cf f7 ff          call   66950 <isl_calloc_or_die@plt>
   e9a07:       48 83 f8 01             cmp    $0x1,%rax
   e9a0b:       48 89 43 08             mov    %rax,0x8(%rbx)
   e9a0f:       5b                      pop    %rbx
   e9a10:       19 c0                   sbb    %eax,%eax
   e9a12:       c3                      ret
   e9a13:       b8 ff ff ff ff          mov    $0xffffffff,%eax
   e9a18:       c3                      ret

Disassembly of section .fini:
$
LebedevRI commented 1 year ago

@doko42 here's the bug you requested: https://buildd.debian.org/status/fetch.php?pkg=isl&arch=amd64&ver=0.26-2&stamp=1686530707&raw=0

checking whether C compiler accepts -march=haswell... yes
checking for gcc architecture flag... -march=haswell
checking whether C compiler accepts -g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -O3 -fomit-frame-pointer -malign-double -fstrict-aliasing -ffast-math -march=haswell... yes
/bin/bash ./libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H   -I. -I. -I./include -Iinclude/  -Wdate-time -D_FORTIFY_SOURCE=2  -g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -O3 -fomit-frame-pointer -malign-double -fstrict-aliasing -ffast-math -march=haswell -c -o isl_hash.lo isl_hash.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I. -I./include -Iinclude/ -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -O3 -fomit-frame-pointer -malign-double -fstrict-aliasing -ffast-math -march=haswell -c isl_hash.c  -fPIC -DPIC -o .libs/isl_hash.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I. -I./include -Iinclude/ -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -O3 -fomit-frame-pointer -malign-double -fstrict-aliasing -ffast-math -march=haswell -c isl_hash.c -o isl_hash.o >/dev/null 2>&1

Notice anything funny there? :)

KarlMagnusLarsson commented 1 year ago

@LebedevRI wrote:

hi! Please note that the "illegal instruction" is encountered in libisl,

That is correct, as originally pointed out by @LebedevRI in (libisl issue, shlx instruction x86-64-v3?), but there also seem to be an issue with debian supplied gcc 12.2.0 as well, as pointed out by @LebedevRI in (gcc issue, blsr instruction x86-64-v2?).

In my case I had to fix both gcc (compile with native) and a newly compiled libisl in workaround. It is quite possible @jpellegrini had only one issue, with libisl, but I seem to have an additional gcc issue as well.

LebedevRI commented 1 year ago

@KarlMagnusLarsson there is only a single issue. If you look at both of those backtraces, you can notice that they happen in the same exact function in the same exact library, and just a few instructions apart.

jpellegrini commented 1 year ago

It is quite possible @jpellegrini had only one issue, with libisl, but I seem to have an additional gcc issue as well.

Yes, only the isl issue with me, probably because my CPU is different from yours...

jpellegrini commented 1 year ago

and just a few instructions apart

Because one of the CPUs tested supports BMI1 but not BMI2, and the other supports neither (right?)

KarlMagnusLarsson commented 1 year ago

@KarlMagnusLarsson there is only a single issue. If you look at both of those backtraces, you can notice that they happen in the same exact function in the same exact library, and just a few instructions apart.

Aha, OK. Thank you @LebedevRI for pointing that out. I somehow worked around this regarding darktable code. The workaround is much easier if I just need a new libisl. I will try that.

@doko42 wrote:

The dt build script doesn't work for me, asking for LLVM as well, so I cannot check.

You do not need the build script. LLVM is supposed to be optional dependency for dt. This works fine for dt source:

git clone --recurse-submodules https://github.com/darktable-org/darktable.git
cd darktable
git submodule update --init
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX="<install dir>" -DCMAKE_BUILD_TYPE=Release ..
make
doko42 commented 1 year ago

thanks for tracking that down. You need to configure isl --without-gcc-arch. Currently building for Debian unstable.

LebedevRI commented 1 year ago

thanks for tracking that down. You need to configure isl --without-gcc-arch. Currently building for Debian unstable.

I'm surprised there is no required lintian check for this -- passing -march=<march> with <march> other than generic/x86-64-v1/??? should really raise a packaging warning.

doko42 commented 1 year ago

lintian is not doing any analysis of the build logs

KarlMagnusLarsson commented 1 year ago

Hello @LebedevRI , @jpellegrini and @doko42, only building new ISL library works for me.

No need to mess with own gcc build at all. I thought I had two separate issues one in Debian gcc and another in debian libisl23. I do not. Issue is confined to libisl23 from Debian bookworm. I have this:

apt list libisl*
Listing... Done
libisl-dev/stable,now 0.25-1 amd64 [installed]
libisl23/stable,now 0.25-1 amd64 [installed,automatic]

I build ISL like shown below. I do not set any particular CFLAGS or CXXFLAGS. I use gcc (Debian 12.2.0-14) 12.2.0 standard gcc from Debian 12 Bookworm. I then make sure that cmake and darktable finds my new libisl during configure of darktable and during execution of darktable.

git clone https://repo.or.cz/isl.git
cd isl
git checkout isl-0.26
./autogen.sh
./configure --prefix="$HOME/camera2/test2/darktable-isl"
make -j8
make check
make install

With this local libisl I can build and run darktable from git master.

jpellegrini commented 1 year ago

Hello @KarlMagnusLarsson @LebedevRI @doko42 -- the new version of libisl is available on sid, and... It works!!! :grin: I have successfully compiled darktable with GCC 12. :)

BodoHobbit commented 1 year ago

@KarlMagnusLarsson @jpellegrini @LebedevRI @doko42 Thanks a lot for finding out the root cause of this issue. I just installed libisl from unstable (sid) and compilation worked without any issues.

jpellegrini commented 1 year ago

I just installed libisl from unstable (sid) and compilation worked without any issues.

Yes -- I opened a bug in the gcc package (reassigned to libisl): https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1038603 and they fixed it quite quickly!

KarlMagnusLarsson commented 1 year ago

I will stay on Debian 12, stable. I will have to wait for this libisl issue to be fixed in stable. I compile my own libisl meanwhile.

KarlMagnusLarsson commented 9 months ago

Continuing the discussion from Debian 12 Bookworm : Building darktable triggers internal compiler error:

This is now fixed, also in Debian stable Bookworm, after upgrade 12.4 -> 12.5: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1038603 in libisl23/stable,now 0.25-1.1 amd64.

It has been a long wait for me using libisl-0.26 compiled from source, meanwhile. However, now I can compile darktable source using regular libisl23/stable,now 0.25-1.1 amd64 from Debian.

Thank you for the issue report and subsequent comments @jpellegrini, @LebedevRI and all.

LebedevRI commented 9 months ago

6+ months to get this fixed. WTF debian.