python extension rebuilt during meson's install phase

[sharkcz]

Looks like the python extension is rebuilt when running meson's install command. Which is generally wrong and in a package build it means that the rebuild during installation won't use the proper compiler and linker flags from the compile phase, but rather the defaults. Running

git clean -xfd
meson setup . redhat-linux-build -Dstatic-build=false
meson compile -C redhat-linux-build -j 16 --verbose
rm -rf /tmp/dtc.ppc64le
DESTDIR=/tmp/dtc.ppc64le meson install -C redhat-linux-build --no-rebuild

produces

...
[dan@talos dtc]$ rm -rf /tmp/dtc.ppc64le; DESTDIR=/tmp/dtc.ppc64le /usr/bin/meson install -C redhat-linux-build --no-rebuild
Installing libfdt/libfdt.so.1.7.0 to /tmp/dtc.ppc64le/usr/local/lib64
Installing libfdt/libfdt.a to /tmp/dtc.ppc64le/usr/local/lib64
Installing convert-dtsv0 to /tmp/dtc.ppc64le/usr/local/bin
Installing dtc to /tmp/dtc.ppc64le/usr/local/bin
Installing fdtdump to /tmp/dtc.ppc64le/usr/local/bin
Installing fdtget to /tmp/dtc.ppc64le/usr/local/bin
Installing fdtput to /tmp/dtc.ppc64le/usr/local/bin
Installing fdtoverlay to /tmp/dtc.ppc64le/usr/local/bin
Installing /mnt/dan/dtc/libfdt/fdt.h to /tmp/dtc.ppc64le/usr/local/include
Installing /mnt/dan/dtc/libfdt/libfdt.h to /tmp/dtc.ppc64le/usr/local/include
Installing /mnt/dan/dtc/libfdt/libfdt_env.h to /tmp/dtc.ppc64le/usr/local/include
Installing /mnt/dan/dtc/redhat-linux-build/meson-private/libfdt.pc to /tmp/dtc.ppc64le/usr/local/lib64/pkgconfig
Installing /mnt/dan/dtc/dtdiff to /tmp/dtc.ppc64le/usr/local/bin
Installing symlink pointing to libfdt.so.1.7.0 to /tmp/dtc.ppc64le/usr/local/lib64/libfdt.so.1
Installing symlink pointing to libfdt.so.1 to /tmp/dtc.ppc64le/usr/local/lib64/libfdt.so
Running custom install script '/mnt/dan/dtc/pylibfdt/../setup.py --top-builddir /mnt/dan/dtc/redhat-linux-build install --prefix=/usr/local --root=$DESTDIR'
running install
/usr/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
running build
running build_py
running build_ext
building '_libfdt' extension
swigging /mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt.i to /mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt_wrap.c
swig -python -I/mnt/dan/dtc/pylibfdt/../libfdt -o /mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt_wrap.c /mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt.i
gcc -fno-strict-overflow -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -fexceptions -fexceptions -fexceptions -fPIC -DPY_SSIZE_T_CLEAN -I/mnt/dan/dtc/pylibfdt/../libfdt -I/usr/include/python3.12 -c /mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt_wrap.c -o build/temp.linux-ppc64le-cpython-312/mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt_wrap.o
creating build/lib.linux-ppc64le-cpython-312
gcc -shared build/temp.linux-ppc64le-cpython-312/mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt_wrap.o -L/mnt/dan/dtc/redhat-linux-build/libfdt -L/usr/lib64 -lfdt -o build/lib.linux-ppc64le-cpython-312/_libfdt.cpython-312-powerpc64le-linux-gnu.so
copying /mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt.py -> build/lib.linux-ppc64le-cpython-312
running install_lib
creating /tmp/dtc.ppc64le/usr/local/lib64/python3.12
creating /tmp/dtc.ppc64le/usr/local/lib64/python3.12/site-packages
copying build/lib.linux-ppc64le-cpython-312/_libfdt.cpython-312-powerpc64le-linux-gnu.so -> /tmp/dtc.ppc64le/usr/local/lib64/python3.12/site-packages
copying build/lib.linux-ppc64le-cpython-312/libfdt.py -> /tmp/dtc.ppc64le/usr/local/lib64/python3.12/site-packages
byte-compiling /tmp/dtc.ppc64le/usr/local/lib64/python3.12/site-packages/libfdt.py to libfdt.cpython-312.pyc
running install_egg_info
running egg_info
creating /mnt/dan/dtc/pylibfdt/libfdt.egg-info
writing /mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt.egg-info/PKG-INFO
writing dependency_links to /mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt.egg-info/dependency_links.txt
writing top-level names to /mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt.egg-info/top_level.txt
writing manifest file '/mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt.egg-info/SOURCES.txt'
reading manifest file '/mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt.egg-info/SOURCES.txt'
writing manifest file '/mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt.egg-info/SOURCES.txt'
Copying /mnt/dan/dtc/pylibfdt/../pylibfdt/libfdt.egg-info to /tmp/dtc.ppc64le/usr/local/lib64/python3.12/site-packages/libfdt-0.0.0-py3.12.egg-info
running install_scripts

[dgibson]

Looks like a real bug, unfortunately setuptools confuses the hell out of me, so I don't really know how to fix it. @blmaier @sjg20 any ideas?

[blmaier]

Are you using the DTC 1.7.0 tag or the main branch? What Meson version?

I tried running the steps provided but on my Fedora 40 machine it doesn't rebuild the python library.

$ DESTDIR=./dest meson install -C build --no-rebuild
Installing libfdt/libfdt.so.1.7.0 to /home/brandon/work/dtc/build/dest/usr/local/lib64
Installing libfdt/libfdt.a to /home/brandon/work/dtc/build/dest/usr/local/lib64
Installing convert-dtsv0 to /home/brandon/work/dtc/build/dest/usr/local/bin
Installing dtc to /home/brandon/work/dtc/build/dest/usr/local/bin
Installing fdtdump to /home/brandon/work/dtc/build/dest/usr/local/bin
Installing fdtget to /home/brandon/work/dtc/build/dest/usr/local/bin
Installing fdtput to /home/brandon/work/dtc/build/dest/usr/local/bin
Installing fdtoverlay to /home/brandon/work/dtc/build/dest/usr/local/bin
Installing /home/brandon/work/dtc/libfdt/fdt.h to /home/brandon/work/dtc/build/dest/usr/local/include
Installing /home/brandon/work/dtc/libfdt/libfdt.h to /home/brandon/work/dtc/build/dest/usr/local/include
Installing /home/brandon/work/dtc/libfdt/libfdt_env.h to /home/brandon/work/dtc/build/dest/usr/local/include
Installing /home/brandon/work/dtc/build/meson-private/libfdt.pc to /home/brandon/work/dtc/build/dest/usr/local/lib64/pkgconfig
Installing /home/brandon/work/dtc/dtdiff to /home/brandon/work/dtc/build/dest/usr/local/bin
Installing symlink pointing to libfdt.so.1.7.0 to /home/brandon/work/dtc/build/dest/usr/local/lib64/libfdt.so.1
Installing symlink pointing to libfdt.so.1 to /home/brandon/work/dtc/build/dest/usr/local/lib64/libfdt.so
Running custom install script '/home/brandon/work/dtc/pylibfdt/../setup.py --quiet --top-builddir /home/brandon/work/dtc/build install --prefix=/usr/local --root=$DESTDIR'
/usr/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()

[sharkcz]

I am on the main branch, meson meson-1.4.1-1.fc40.noarch, the host is ppc64le, but it shouldn't really matter :-)

[sharkcz]

And I made the python setup.py script non-quiet (see #134) with

diff --git a/pylibfdt/meson.build b/pylibfdt/meson.build
index 978c53f..a043d0a 100644
--- a/pylibfdt/meson.build
+++ b/pylibfdt/meson.build
@@ -1,5 +1,5 @@
 setup_py = find_program('../setup.py')
-setup_py = [setup_py, '--quiet', '--top-builddir', meson.project_build_root()]
+setup_py = [setup_py, '--top-builddir', meson.project_build_root()]

 pylibfdt = custom_target(
   'pylibfdt',

[sharkcz]

Isn't the rebuild output just hidden in your case?

[blmaier]

@sharkcz you're right, I hadn't removed --quiet, I do see this behavior now.

I opened #136 which should work around the issue.