search

python-hyper / brotlicffi

Python bindings to the Brotli compression library
MIT License
148 stars 27 forks source link

build `brotlicffi` against shared system libbrotli #195

Closed wuch-g2v closed 12 months ago

wuch-g2v commented 12 months ago

How to build build brotlicffi modules DSO against shared system libbrotli? 🤔

sethmlarson commented 12 months ago

I believe you set USE_SHARED_BROTLI=1 as an environment variable and then install from source pip install --no-binary=brotlicffi ...

wuch-g2v commented 12 months ago

Hmm looking on the https://github.com/python-hyper/brotlicffi/blob/353f6f16f0a1871d761fa2f68cd2efedc35afc9a/setup.py#L43 USE_SHARED_BROTLI=1 it is default behaviour and there is no any other place where -lbrotlienc is passed to cython.

wuch-g2v commented 12 months ago

Just tested that and looks like when this env variable is used build ends with

[builder@builder brotlicffi]$ ldd _brotlicffi.abi3.so
        linux-vdso.so.1 (0x00007ffd4a5fb000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f14a646f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f14a6662000)

Checking build log I see that empty 

+ USE_SHARED_BROTLI=0 /usr/bin/python3 -sBm build -w --no-isolation
* Getting build dependencies for wheel...
* Building wheel...
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-38
creating build/lib.linux-x86_64-cpython-38/brotlicffi
copying src/brotlicffi/__init__.py -> build/lib.linux-x86_64-cpython-38/brotlicffi
copying src/brotlicffi/_api.py -> build/lib.linux-x86_64-cpython-38/brotlicffi
copying src/brotlicffi/_build.py -> build/lib.linux-x86_64-cpython-38/brotlicffi
running build_clib
building 'libbrotli' library
creating build/temp.linux-x86_64-cpython-38
/usr/bin/gcc-ar rcs build/temp.linux-x86_64-cpython-38/liblibbrotli.a     <<<< HERE
running build_ext
generating cffi module 'build/temp.linux-x86_64-cpython-38/_brotlicffi.c'
building '_brotlicffi' extension
creating build/temp.linux-x86_64-cpython-38/build
creating build/temp.linux-x86_64-cpython-38/build/temp.linux-x86_64-cpython-38
/usr/bin/gcc -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -s
pecs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -fno-
semantic-interposition -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -spec
s=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fa
synchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -fno-sem
antic-interposition -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-ha
rdened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fsta
ck-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -fno-semantic-interposition -O2 -g -gre
cord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-st
rong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protec
tion -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -fPIC -Ilibbrotli/c -Ilibbrotli/c/include -Ilibbrotli/c/common -I/usr/include/python3.8 -c build/temp.linux-x86_64-c
python-38/_brotlicffi.c -o build/temp.linux-x86_64-cpython-38/build/temp.linux-x86_64-cpython-38/_brotlicffi.o
/usr/bin/gcc -shared -Wl,--gc-sections -Wl,--as-needed -flto=auto -flto-partition=none -fuse-linker-plugin -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--build-id=sha1 -Wl,-
-gc-sections -Wl,--as-needed -flto=auto -flto-partition=none -fuse-linker-plugin -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--build-id=sha1 -Wl,--gc-sections -Wl,--as-need
ed -flto=auto -flto-partition=none -fuse-linker-plugin -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--build-id=sha1 -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-s
ecurity -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin
-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto
=auto -flto-partition=none build/temp.linux-x86_64-cpython-38/build/temp.linux-x86_64-cpython-38/_brotlicffi.o -L/usr/lib64 -Lbuild/temp.linux-x86_64-cpython-38 -llibbrotli -llibbrotli -o b
uild/lib.linux-x86_64-cpython-38/brotlicffi/_brotlicffi.abi3.so
wuch-g2v commented 11 months ago

Are you able at least provide some hints what needs to be altered to be able link against system brotli libraries instead bundled one? 🤔