search

darktable-org / darktable

darktable is an open source photography workflow application and raw developer
https://www.darktable.org
GNU General Public License v3.0
9.77k stars 1.14k forks source link

Compiler segfaults on src/common/iop_profile.c on github runner macos-13 #15945

Closed zisoft closed 8 months ago

zisoft commented 10 months ago

I am testing the new github runner macos-13 since several weeks now and cannot get dt to compile.

The build process reproducible segfaults the compiler (Xcode current version 15.1) on src/common/iop_profile.c:

/Applications/Xcode_15.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -DAVIF_DLL -DGDK_DISABLE_DEPRECATED -DGDK_VERSION_MIN_REQUIRED=GDK_VERSION_3_24 -DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_MIN_REQUIRED -DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_56 -DGTK_DISABLE_DEPRECATED -DGTK_DISABLE_SINGLE_INCLUDES -DHAVE_BUILTIN_CPU_SUPPORTS -DHAVE_CONFIG_H -DHAVE_GAME -DHAVE_GPHOTO2 -DHAVE_ICU -DHAVE_IMAGEMAGICK -DHAVE_IMATH -DHAVE_ISO_CODES -DHAVE_KWALLET -DHAVE_LIBAVIF=1 -DHAVE_LIBEXIV2_WITH_ISOBMFF=1 -DHAVE_LIBHEIF=1 -DHAVE_LIBJXL -DHAVE_LIBRAW=1 -DHAVE_LIBSECRET -DHAVE_LIBSHARPYUV=1 -DHAVE_MAP -DHAVE_OPENCL -DHAVE_OPENEXR -DHAVE_OPENJPEG -DHAVE_OSMGPSMAP_110_OR_NEWER -DHAVE_OSMGPSMAP_NEWER_THAN_110 -DHAVE_P11KIT -DHAVE_PRINT -DHAVE_SQLITE_324_OR_NEWER -DHAVE_VISIBILITY -DHAVE_WEBP -DLIBHEIF_EXPORTS -DLIBRAW_NODLL -DMAC_INTEGRATION -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16 -DOS_OBJECT_USE_OBJC=0 -DSQLITE_CORE -DSQLITE_ENABLE_ICU -DUSE_LUA -D_XOPEN_SOURCE=700 -Dlib_darktable_EXPORTS -I/Users/runner/work/darktable/darktable/src/build/bin -I/Users/runner/work/darktable/darktable/src/src -I/Users/runner/work/darktable/darktable/src/src/external/LuaAutoC -I/Users/runner/work/darktable/darktable/src/src/external/whereami/src -I/Users/runner/work/darktable/darktable/src/src/external/LibRaw -isystem /Users/runner/work/darktable/darktable/src/src/external -isystem /Users/runner/work/darktable/darktable/src/src/external/OpenCL -isystem /usr/local/include -isystem /usr/local/include/glib-2.0 -isystem /usr/local/include/../lib/glib-2.0/include -isystem /usr/local/Cellar/gtk+3/3.24.39/include/gtk-3.0 -isystem /usr/local/Cellar/glib/2.78.3/include/gio-unix-2.0 -isystem /usr/local/Cellar/cairo/1.18.0/include -isystem /usr/local/Cellar/libepoxy/1.5.10/include -isystem /usr/local/Cellar/pango/1.50.14/include/pango-1.0 -isystem /usr/local/Cellar/harfbuzz/8.3.0/include/harfbuzz -isystem /usr/local/Cellar/fribidi/1.0.13/include/fribidi -isystem /usr/local/Cellar/graphite2/1.3.14/include -isystem /usr/local/Cellar/at-spi2-core/2.50.0/include/atk-1.0 -isystem /usr/local/Cellar/cairo/1.18.0/include/cairo -isystem /usr/local/Cellar/fontconfig/2.14.2/include -isystem /usr/local/opt/freetype/include/freetype2 -isystem /usr/local/Cellar/libxext/1.3.5/include -isystem /usr/local/Cellar/libxrender/0.9.11/include -isystem /usr/local/Cellar/libx11/1.8.7/include -isystem /usr/local/Cellar/libxcb/1.16/include -isystem /usr/local/Cellar/libxau/1.0.11/include -isystem /usr/local/Cellar/libxdmcp/1.1.4/include -isystem /usr/local/Cellar/pixman/0.42.2/include/pixman-1 -isystem /usr/local/Cellar/gdk-pixbuf/2.42.10_1/include/gdk-pixbuf-2.0 -isystem /usr/local/opt/libpng/include/libpng16 -isystem /usr/local/Cellar/libtiff/4.6.0/include -isystem /usr/local/opt/zstd/include -isystem /usr/local/Cellar/xz/5.4.5/include -isystem /usr/local/Cellar/jpeg-turbo/3.0.0/include -isystem /usr/local/Cellar/glib/2.78.3/include -isystem /usr/local/Cellar/glib/2.78.3/include/glib-2.0 -isystem /usr/local/Cellar/glib/2.78.3/lib/glib-2.0/include -isystem /usr/local/opt/gettext/include -isystem /usr/local/Cellar/pcre2/10.42/include -isystem /usr/local/Cellar/xorgproto/2023.2/include -isystem /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/usr/include/ffi -isystem /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/usr/include/libxml2 -isystem /usr/local/Cellar/lensfun/0.3.4/include/lensfun -isystem /usr/local/include/pango-1.0 -isystem /usr/local/include/librsvg-2.0 -isystem /usr/local/include/json-glib-1.0 -isystem /usr/local/Cellar/openjpeg/2.5.0_1/include/openjpeg-2.5 -isystem /usr/local/Cellar/libsecret/0.21.2/include/libsecret-1 -isystem /usr/local/Cellar/libgcrypt/1.10.3/include -isystem /usr/local/opt/libgpg-error/include -isystem /usr/local/include/gtkmacintegration -isystem /usr/local/Cellar/p11-kit/0.25.3/include/p11-kit-1 -isystem /usr/local/Cellar/imagemagick@6/6.9.13-1/include/ImageMagick-6 -isystem /usr/local/include/lua -isystem /usr/local/Cellar/osm-gps-map/1.2.0_2/include/osmgpsmap-1.0 -isystem /usr/local/Cellar/libsoup@2/2.74.2_1/include/libsoup-2.4 -isystem /usr/local/Cellar/libpsl/0.21.2_2/include -isystem /usr/local/Cellar/icu4c/73.2/include -isystem /usr/local/include/Imath -isystem /usr/local/include/OpenEXR -isystem /usr/local/opt/icu4c/include -fopenmp-version=45 -D_DARWIN_C_SOURCE -Wall -Wformat -Wformat-security -Wshadow -Wtype-limits -Wvla -Wthread-safety -Wno-unknown-pragmas -Wno-error=varargs -Wno-error=address-of-packed-member -Xclang -fopenmp -mtune=generic -msse2 -g -O3 -DNDEBUG -O3 -ffast-math -fno-finite-math-only -std=c99 -isysroot /Applications/Xcode_15.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk -mmacosx-version-min=13.5 -fPIC   -Werror -Wfatal-errors -Wno-error=pass-failed -MD -MT bin/CMakeFiles/lib_darktable.dir/common/iop_profile.c.o -MF bin/CMakeFiles/lib_darktable.dir/common/iop_profile.c.o.d -o bin/CMakeFiles/lib_darktable.dir/common/iop_profile.c.o -c /Users/runner/work/darktable/darktable/src/src/common/iop_profile.c
clang: fatal error: unable to execute command: Segmentation fault: 11
Apple clang version 15.0.0 (clang-1500.1.0.2.5)
Target: x86_64-apple-darwin22.6.0
Thread model: posix
InstalledDir: /Applications/Xcode_15.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

full log

I cannot reproduce that on my local machine, it is only on the github runner. Maybe the compiler experts can check if there is something critical with iop_profile.c

The build process runs without errors with Xcode 14

zisoft commented 9 months ago

Still present with macos-13 runner and Xcode 15.2: log

LebedevRI commented 8 months ago

Hm, that's not good.

Could you try limiting the amount of build jobs, for a start, down to 1? Put -j1 in https://github.com/darktable-org/darktable/blob/7fcbdba13e1133a080abd6355ff3d081b590e189/.ci/ci-script.sh#L34

Can you reproduce with a brew-provided clang-15 / clang-16 / clang-17?

zisoft commented 8 months ago

I am now able to reproduce the crash on my local machine. The file compiles fine without -DHAVE_OPENCL.

Digging further...

LebedevRI commented 8 months ago

I'd next try applying creduce to that file. Though, if you've never did that before, that is a big ask...

zisoft commented 8 months ago

If I comment out the following two function calls:

https://github.com/darktable-org/darktable/blob/9c8cad544862fac6e561d4a601a7ca37794d0302/src/common/iop_profile.c#L1622-L1626

and

https://github.com/darktable-org/darktable/blob/9c8cad544862fac6e561d4a601a7ca37794d0302/src/common/iop_profile.c#L1817-L1821

the compiler is happy.

zisoft commented 8 months ago

it is indeed the __DT_CLONE_TARGETS__ in front of that function definitions

LebedevRI commented 8 months ago

Is __DT_CLONE_TARGETS__ really not empty there, is it not ARM64/AArch64 (not to be confused with AMD64)?

zisoft commented 8 months ago 
#if __has_attribute(target_clones) && !defined(_WIN32) && !defined(NATIVE_ARCH)
# if defined(__amd64__) || defined(__amd64) || defined(__x86_64__) || defined(__x86_64)
#define __DT_CLONE_TARGETS__ __attribute__((target_clones("default", "sse2", "sse3", "sse4.1", "sse4.2", "popcnt", "avx", "avx2", "avx512f", "fma4")))
# elif defined(__PPC64__)
/* __PPC64__ is the only macro tested for in is_supported_platform.h, other macros would fail there anyway. */
#define __DT_CLONE_TARGETS__ __attribute__((target_clones("default","cpu=power9")))
# else
#define __DT_CLONE_TARGETS__
# endif
#else
#define __DT_CLONE_TARGETS__
#endif

__x86_64__ is true

LebedevRI commented 8 months ago

Well, it's mac, so unlike normal distributions, it's not like there's a zoo of CPU versions to support, i think? So is there even a need for __DT_CLONE_TARGETS__ there, or even -DBINARY_PACKAGE_BUILD=ON?

zisoft commented 8 months ago

Looks good on my machine with

#if __has_attribute(target_clones) && !defined(_WIN32) && !defined(NATIVE_ARCH) && !defined(__APPLE__)

Let's see what github runner macos-13 gives.

EDIT: success!

LebedevRI commented 8 months ago

Nice!