search

mreineck / ducc

Fork of https://gitlab.mpcdf.mpg.de/mtr/ducc to simplify external contributions
GNU General Public License v2.0
13 stars 12 forks source link

Cannot build in editable mode with gcc13 #22

Open JBorrow opened 10 months ago

JBorrow commented 10 months ago

Trying to build in editable mode (pip3 install -e .) breaks:

pip3 install -e .
Obtaining file:///Users/borrow-adm/Documents/SimonsObs/ducc
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: numpy>=1.17.0 in /Users/borrow-adm/.pyenv/versions/3.9.18/lib/python3.9/site-packages (from ducc0==0.32.0) (1.26.0)
Building wheels for collected packages: ducc0
  Building editable for ducc0 (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building editable for ducc0 (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [171 lines of output]
      /private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/config/_apply_pyprojecttoml.py:73: _WouldIgnoreField: `dependencies` defined outside of `pyproject.toml` would be ignored.
      !!

              ********************************************************************************
              ##########################################################################
              # configuration would be ignored/result in error due to `pyproject.toml` #
              ##########################################################################

              The following seems to be defined outside of `pyproject.toml`:

              `dependencies = ['numpy>=1.17.0']`

              According to the spec (see the link below), however, setuptools CANNOT
              consider this value unless `dependencies` is listed as `dynamic`.

              https://packaging.python.org/en/latest/specifications/declaring-project-metadata/

              For the time being, `setuptools` will still consider the given value (as a
              **transitional** measure), but please note that future releases of setuptools will
              follow strictly the standard.

              To prevent this warning, you can list `dependencies` under `dynamic` or alternatively
              remove the `[project]` table from your file and rely entirely on other means of
              configuration.

              By 2023-Oct-30, you need to update your project and remove deprecated calls
              or your builds will no longer be supported.
              ********************************************************************************

      !!
        _handle_missing_dynamic(dist, project_table)

      Build environment:
      Platform:      macOS-13.6-arm64-arm-64bit
      Machine:       arm64
      System:        Darwin
      Architecture:  ('64bit', '')

      running editable_wheel
      creating /private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-wheel-ckiw077o/.tmp-565l2_9i/ducc0.egg-info
      writing /private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-wheel-ckiw077o/.tmp-565l2_9i/ducc0.egg-info/PKG-INFO
      writing dependency_links to /private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-wheel-ckiw077o/.tmp-565l2_9i/ducc0.egg-info/dependency_links.txt
      writing requirements to /private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-wheel-ckiw077o/.tmp-565l2_9i/ducc0.egg-info/requires.txt
      writing top-level names to /private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-wheel-ckiw077o/.tmp-565l2_9i/ducc0.egg-info/top_level.txt
      writing manifest file '/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-wheel-ckiw077o/.tmp-565l2_9i/ducc0.egg-info/SOURCES.txt'
      reading manifest file '/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-wheel-ckiw077o/.tmp-565l2_9i/ducc0.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      no previously-included directories found matching '.'
      adding license file 'LICENSE'
      writing manifest file '/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-wheel-ckiw077o/.tmp-565l2_9i/ducc0.egg-info/SOURCES.txt'
      creating '/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-wheel-ckiw077o/.tmp-565l2_9i/ducc0-0.32.0.dist-info'
      creating /private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-wheel-ckiw077o/.tmp-565l2_9i/ducc0-0.32.0.dist-info/WHEEL
      running build_ext
      building 'ducc0' extension
      creating /var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/tmp_9qtjrrd.build-temp/python
      /opt/homebrew/Cellar/gcc/13.2.0/bin/gcc-13 -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -DOPENSSL_NO_SSL3 -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -DPKGNAME=ducc0 -DPKGVERSION=0.32.0 -I. -I./src/ -I/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/pybind11/include -I/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/pybind11/include -I/Users/borrow-adm/.pyenv/versions/3.9.18/include/python3.9 -c python/ducc.cc -o /var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/tmp_9qtjrrd.build-temp/python/ducc.o -std=c++17 -fvisibility=hidden -g0 -ffast-math -O3 -march=native -mmacosx-version-min=10.14 -pthread
      In file included from python/ducc.cc:13:
      ./src/ducc0/wgridder/wgridder.cc: In function 'auto ducc0::detail_gridder::get_nminmax_rectangle(double, double, double, double)':
      ./src/ducc0/wgridder/wgridder.cc:125:34: note: parameter passing for argument of type 'std::pair<double, double>' when C++17 is enabled changed to match C++14 in GCC 10.1
        125 |   return make_pair(nm1min, nm1max);
            |                                  ^
      In file included from /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/string:51,
                       from /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/bits/locale_classes.h:40,
                       from /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/bits/ios_base.h:41,
                       from /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/ios:44,
                       from /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/istream:40,
                       from /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/sstream:40,
                       from ./src/ducc0/infra/string_utils.cc:33,
                       from python/ducc.cc:1:
      In static member function 'static _Up* std::__copy_move<_IsMove, true, std::random_access_iterator_tag>::__copy_m(_Tp*, _Tp*, _Up*) [with _Tp = const long unsigned int; _Up = long unsigned int; bool _IsMove = false]',
          inlined from '_OI std::__copy_move_a2(_II, _II, _OI) [with bool _IsMove = false; _II = const long unsigned int*; _OI = long unsigned int*]' at /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/bits/stl_algobase.h:506:30,
          inlined from '_OI std::__copy_move_a1(_II, _II, _OI) [with bool _IsMove = false; _II = const long unsigned int*; _OI = long unsigned int*]' at /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/bits/stl_algobase.h:533:42,
          inlined from '_OI std::__copy_move_a(_II, _II, _OI) [with bool _IsMove = false; _II = __gnu_cxx::__normal_iterator<const long unsigned int*, vector<long unsigned int> >; _OI = long unsigned int*]' at /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/bits/stl_algobase.h:540:31,
          inlined from '_OI std::copy(_II, _II, _OI) [with _II = __gnu_cxx::__normal_iterator<const long unsigned int*, vector<long unsigned int> >; _OI = long unsigned int*]' at /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/bits/stl_algobase.h:633:7,
          inlined from 'static _ForwardIterator std::__uninitialized_copy<true>::__uninit_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = __gnu_cxx::__normal_iterator<const long unsigned int*, std::vector<long unsigned int> >; _ForwardIterator = long unsigned int*]' at /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/bits/stl_uninitialized.h:147:27,
          inlined from '_ForwardIterator std::uninitialized_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = __gnu_cxx::__normal_iterator<const long unsigned int*, vector<long unsigned int> >; _ForwardIterator = long unsigned int*]' at /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/bits/stl_uninitialized.h:185:15,
          inlined from '_ForwardIterator std::__uninitialized_copy_a(_InputIterator, _InputIterator, _ForwardIterator, allocator<_Tp>&) [with _InputIterator = __gnu_cxx::__normal_iterator<const long unsigned int*, vector<long unsigned int> >; _ForwardIterator = long unsigned int*; _Tp = long unsigned int]' at /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/bits/stl_uninitialized.h:373:37,
          inlined from 'std::vector<_Tp, _Alloc>::vector(const std::vector<_Tp, _Alloc>&) [with _Tp = long unsigned int; _Alloc = std::allocator<long unsigned int>]' at /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/bits/stl_vector.h:603:31,
          inlined from 'shp ducc0::detail_misc_utils::noncritical_shape(const shp&, size_t) [with shp = std::vector<long unsigned int>]' at ./src/ducc0/infra/misc_utils.h:84:23:
      /opt/homebrew/Cellar/gcc/13.2.0/include/c++/13/bits/stl_algobase.h:437:30: warning: 'void* __builtin_memmove(void*, const void*, long unsigned int)' forming offset 8 is out of the bounds [0, 8] [-Warray-bounds=]
        437 |             __builtin_memmove(__result, __first, sizeof(_Tp) * _Num);
            |             ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      /opt/homebrew/Cellar/gcc/13.2.0/bin/gcc-13 -bundle -undefined dynamic_lookup -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/borrow-adm/.pyenv/versions/3.9.18/lib -Wl,-rpath,/Users/borrow-adm/.pyenv/versions/3.9.18/lib -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/borrow-adm/.pyenv/versions/3.9.18/lib -Wl,-rpath,/Users/borrow-adm/.pyenv/versions/3.9.18/lib -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib /var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/tmp_9qtjrrd.build-temp/python/ducc.o -L/Users/borrow-adm/.pyenv/versions/3.9.18/lib -o /var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/tmptst4h18z.build-lib/ducc0.cpython-39-darwin.so -mmacosx-version-min=10.14 -pthread
      ld: warning: duplicate -rpath '/Users/borrow-adm/.pyenv/versions/3.9.18/lib' ignored
      ld: warning: duplicate -rpath '/opt/homebrew/lib' ignored
      ld: warning: ignoring duplicate libraries: '-lgcc'
      0  0x102207648  __assert_rtn + 72
      1  0x10213bfac  ld::AtomPlacement::findAtom(unsigned char, unsigned long long, ld::AtomPlacement::AtomLoc const*&, long long&) const + 1204
      2  0x102151924  ld::InputFiles::SliceParser::parseObjectFile(mach_o::Header const*) const + 15164
      3  0x10215ee30  ld::InputFiles::parseAllFiles(void (ld::AtomFile const*) block_pointer)::$_7::operator()(unsigned long, ld::FileInfo const&) const + 420
      4  0x197a38440  _dispatch_client_callout2 + 20
      5  0x197a4d544  _dispatch_apply_invoke_and_wait + 224
      6  0x197a4c84c  _dispatch_apply_with_attr_f + 1180
      7  0x197a4ca38  dispatch_apply + 96
      8  0x1021d93b8  ld::AtomFileConsolidator::parseFiles(bool) + 292
      9  0x10217a170  main + 9048
      ld: Assertion failed: (resultIndex < sectData.atoms.size()), function findAtom, file Relocations.cpp, line 1336.
      collect2: error: ld returned 1 exit status
      Traceback (most recent call last):
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/_distutils/unixccompiler.py", line 266, in link
          self.spawn(linker + ld_args)
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/_distutils/ccompiler.py", line 1041, in spawn
          spawn(cmd, dry_run=self.dry_run, **kwargs)
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/_distutils/spawn.py", line 70, in spawn
          raise DistutilsExecError(
      distutils.errors.DistutilsExecError: command '/opt/homebrew/Cellar/gcc/13.2.0/bin/gcc-13' failed with exit code 1

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 156, in run
          self._create_wheel_file(bdist_wheel)
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 345, in _create_wheel_file
          files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 268, in _run_build_commands
          self._run_build_subcommands()
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 295, in _run_build_subcommands
          self.run_command(name)
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 88, in run
          _build_ext.run(self)
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 249, in build_extension
          _build_ext.build_extension(self, ext)
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 572, in build_extension
          self.compiler.link_shared_object(
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/_distutils/ccompiler.py", line 752, in link_shared_object
          self.link(
        File "/private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/_distutils/unixccompiler.py", line 268, in link
          raise LinkError(msg)
      distutils.errors.LinkError: command '/opt/homebrew/Cellar/gcc/13.2.0/bin/gcc-13' failed with exit code 1
      /private/var/folders/k1/fld2m5s54fd9nd8qy_wdtvf00000gp/T/pip-build-env-olpyuy67/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py:988: _DebuggingTips: Problem in editable installation.
      !!

              ********************************************************************************
              An error happened while installing `ducc0` in editable mode.

              The following steps are recommended to help debug this problem:

              - Try to install the project normally, without using the editable mode.
                Does the error still persist?
                (If it does, try fixing the problem before attempting the editable mode).
              - If you are using binary extensions, make sure you have all OS-level
                dependencies installed (e.g. compilers, toolchains, binary libraries, ...).
              - Try the latest version of setuptools (maybe the error was already fixed).
              - If you (or your project dependencies) are using any setuptools extension
                or customization, make sure they support the editable mode.

              After following the steps above, if the problem still persists and
              you think this is related to how setuptools handles editable installations,
              please submit a reproducible example
              (see https://stackoverflow.com/help/minimal-reproducible-example) to:

                  https://github.com/pypa/setuptools/issues

              See https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.
              ********************************************************************************

      !!
        cmd_obj.run()
      error: command '/opt/homebrew/Cellar/gcc/13.2.0/bin/gcc-13' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building editable for ducc0
Failed to build ducc0
ERROR: Could not build wheels for ducc0, which is required to install pyproject.toml-based projects
JBorrow commented 10 months ago

Build does not succeed in standard (non-editable) mode, either. Sorry for that confusion.

JBorrow commented 10 months ago

Library builds fine with Apple clang version 15.0.0 (clang-1500.0.40.1)

mreineck commented 9 months ago

Hi, and sorry for the long delay!

I cannot reproduce this one locally,since I don't have access to Apple hardware, but perhaps there is some explanation here: https://developer.apple.com/forums/thread/737707

If I understand correctly, this is due to a bug in Apple's linker and should go away at some point.