search

AcademySoftwareFoundation / openexr

The OpenEXR project provides the specification and reference implementation of the EXR file format, the professional-grade image storage format of the motion picture industry.
http://www.openexr.com/
BSD 3-Clause "New" or "Revised" License
1.63k stars 620 forks source link

Regression in v3.2.1: no longer finds -ldeflate on FreeBSD, breaks build #1571

Open mandree opened 1 year ago

mandree commented 1 year ago

FreeBSD builds, which install libdeflate as .so in /usr/local/lib, and use the system libdeflate (not the embedding approach), fail to build as of v3.2.1. The cmake configure phase completes successfully:

-- Found PkgConfig: /usr/local/bin/pkgconf (found version "2.0.3") 
-- Checking for module 'libdeflate'
--   Found libdeflate, version 1.19
-- Using libdeflate from /usr/local/lib/libdeflate.so
-- Using Imath from /usr/local/lib/cmake/Imath

But the build then barfs:

FAILED: src/lib/OpenEXRCore/libOpenEXRCore-3_2.so.31.3.2.1 
: && /usr/local/libexec/ccache/cc -fPIC -O2 -pipe -march=znver1  -fstack-protector-strong -fno-strict-aliasing -O2 -pipe -march=znver1  -fstack-protector-strong -fno-strict-aliasing  -DNDEBUG  -lpthread -fstack-protector-strong -shared -Wl,-soname,libOpenEXRCore-3_2.so.31 -o src/lib/OpenEXRCore/libOpenEXRCore-3_2.so.31.3.2.1 src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_rle.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_zip.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_pxr24.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_b44.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_b44_table.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_piz.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_dwa.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_huf.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/attributes.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/string.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/string_vector.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/float_vector.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/channel_list.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/opaque.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/preview.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/base.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/context.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/memory.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_structs.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/part.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/part_attr.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/std_attr.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/parse_header.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/write_header.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/chunk.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/coding.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/compression.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/decoding.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/encoding.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/pack.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/unpack.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/validation.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/debug.c.o  -Wl,-rpath,/usr/local/lib:  /usr/local/lib/libImath-3_1.so.29.8.0  /usr/lib/libm.so  -ldeflate  -lm && :
ld: error: unable to find library -ldeflate
cc: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

FreeBSD requires some -L/usr/local/lib for -ldeflate, and pkgconf also emits it.

This issue appears to be caused by the second hunk of this patch:

https://github.com/AcademySoftwareFoundation/openexr/pull/1561/files#diff-d20456c2960f2c3e9dbea28785de9afbf98e37a3ccef8043a4848bce979535c3

Reverting this hunk changes the above to (note we have the absolute /usr/local/lib/libdeflate.so instead of -ldeflate), which succeeds:

[  0% 1/252] : && /usr/local/libexec/ccache/cc -fPIC -O2 -pipe -march=znver1  -fstack-protector-strong -fno-strict-aliasing -O2 -pipe -march=znver1  -fstack-protector-strong -fno-strict-aliasing  -DNDEBUG  -lpthread -fstack-protector-strong -shared -Wl,-soname,libOpenEXRCore-3_2.so.31 -o src/lib/OpenEXRCore/libOpenEXRCore-3_2.so.31.3.2.1 src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_rle.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_zip.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_pxr24.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_b44.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_b44_table.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_piz.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_dwa.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_huf.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/attributes.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/string.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/string_vector.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/float_vector.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/channel_list.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/opaque.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/preview.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/base.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/context.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/memory.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/internal_structs.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/part.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/part_attr.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/std_attr.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/parse_header.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/write_header.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/chunk.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/coding.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/compression.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/decoding.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/encoding.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/pack.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/unpack.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/validation.c.o src/lib/OpenEXRCore/CMakeFiles/OpenEXRCore.dir/debug.c.o  -Wl,-rpath,/usr/local/lib:  /usr/local/lib/libImath-3_1.so.29.8.0  /usr/lib/libm.so  -lm  /usr/local/lib/libdeflate.so && :
mandree commented 1 year ago

One small workaround for FreeBSD 13.2 is to patch cmake/OpenEXRSetup.cmake on line 218 like this:

-  set(EXR_DEFLATE_LIB ${deflate_LIBRARIES})
+  set(EXR_DEFLATE_LIB ${deflate_LINK_LIBRARIES})
cary-ilm commented 1 year ago

Thanks for the catch and the suggested fix, that does seem correct to me. Our CI doesn't currently validate all these cases as it should. Let's give this another day or so and I'll push out another release.

mandree commented 1 year ago

Cary, I am sorry to say I appear to have misled you, reading https://github.com/AcademySoftwareFoundation/openexr/pull/1572#issue-1916654793 - my apologies for being unclear about what FreeBSD usually does, and what deflate_LINK_LIBRARIES contains.

deflate_LINK_LIBRARIES, after cmake's configure run, on FreeBSD ports, will contain the absolute path of libdeflate.so (by default, /usr/local/lib/libdeflate.so) but NOT the -L flag. The latter is part of deflate_LDFLAGS, see below. This is what I get in CMakeCache.txt on FreeBSD 13.2 with the workaround shown above (I think it should not influence CMakeCache.txt):

$ grep deflate work/.build/CMakeCache.txt  | grep -v ^//
OPENEXR_DEFLATE_REPO:STRING=https://github.com/ebiggers/libdeflate.git
pkgcfg_lib_deflate_deflate:FILEPATH=/usr/local/lib/libdeflate.so
__pkg_config_arguments_deflate:INTERNAL=IMPORTED_TARGET;GLOBAL;libdeflate
__pkg_config_checked_deflate:INTERNAL=1
deflate_CFLAGS:INTERNAL=-I/usr/local/include
deflate_CFLAGS_I:INTERNAL=
deflate_CFLAGS_OTHER:INTERNAL=
deflate_FOUND:INTERNAL=1
deflate_INCLUDEDIR:INTERNAL=/usr/local/include
deflate_INCLUDE_DIRS:INTERNAL=/usr/local/include
deflate_LDFLAGS:INTERNAL=-L/usr/local/lib;-ldeflate
deflate_LDFLAGS_OTHER:INTERNAL=
deflate_LIBDIR:INTERNAL=/usr/local/lib
deflate_LIBRARIES:INTERNAL=deflate
deflate_LIBRARY_DIRS:INTERNAL=/usr/local/lib
deflate_LIBS:INTERNAL=
deflate_LIBS_L:INTERNAL=
deflate_LIBS_OTHER:INTERNAL=
deflate_LIBS_PATHS:INTERNAL=
deflate_MODULE_NAME:INTERNAL=libdeflate
deflate_PREFIX:INTERNAL=/usr/local
deflate_STATIC_CFLAGS:INTERNAL=-I/usr/local/include
deflate_STATIC_CFLAGS_I:INTERNAL=
deflate_STATIC_CFLAGS_OTHER:INTERNAL=
deflate_STATIC_INCLUDE_DIRS:INTERNAL=/usr/local/include
deflate_STATIC_LDFLAGS:INTERNAL=-L/usr/local/lib;-ldeflate
deflate_STATIC_LDFLAGS_OTHER:INTERNAL=
deflate_STATIC_LIBDIR:INTERNAL=
deflate_STATIC_LIBRARIES:INTERNAL=deflate
deflate_STATIC_LIBRARY_DIRS:INTERNAL=/usr/local/lib
deflate_STATIC_LIBS:INTERNAL=
deflate_STATIC_LIBS_L:INTERNAL=
deflate_STATIC_LIBS_OTHER:INTERNAL=
deflate_STATIC_LIBS_PATHS:INTERNAL=
deflate_VERSION:INTERNAL=1.19
deflate_libdeflate_INCLUDEDIR:INTERNAL=
deflate_libdeflate_LIBDIR:INTERNAL=
deflate_libdeflate_PREFIX:INTERNAL=
deflate_libdeflate_VERSION:INTERNAL=
pkgcfg_lib_deflate_deflate-ADVANCED:INTERNAL=1
cary-ilm commented 1 year ago

I'm unfortunately not going to have any further time to look into this in the next few days. If anyone out there can contribute a fix, I'd appreciate it so much I'll buy you a milkshake.

The fix in #1572 doesn't resolve the issue and presumably shouldn't be merged, although the discussion there makes some good suggestions, I just don't have the time now to turn that into a workable PR.

mgraham-wetafx commented 1 year ago

If anyone out there can contribute a fix, I'd appreciate it so much I'll buy you a milkshake

We've also had the same issue internally here at Weta using Ubuntu with an existing libdelflate install (provided via CMAKE_PREFIX_PATH/PKG_CONFIG_PATH).

Will be pushing up a patch soon, and looking forward to that milkshake!