search

conda-forge / petsc-feedstock

A conda-smithy repository for petsc.
BSD 3-Clause "New" or "Revised" License
8 stars 23 forks source link

scotch7.0.2 broke macOS #180

Closed guyer closed 9 months ago

guyer commented 9 months ago

Solution to issue cannot be found in the documentation.

Issue

Since merging #176, petsc4py raises a linkage error on macOS (both osx-64 and osx-arm64). linux-64 is fine.

$ python -c "from petsc4py import PETSc"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/guyer/mambaforge-arm/envs/scotch_eq_7/lib/python3.12/site-packages/petsc4py/PETSc.py", line 3, in <module>
    PETSc = ImportPETSc(ARCH)
            ^^^^^^^^^^^^^^^^^
  File "/Users/guyer/mambaforge-arm/envs/scotch_eq_7/lib/python3.12/site-packages/petsc4py/lib/__init__.py", line 29, in ImportPETSc
    return Import('petsc4py', 'PETSc', path, arch)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/guyer/mambaforge-arm/envs/scotch_eq_7/lib/python3.12/site-packages/petsc4py/lib/__init__.py", line 95, in Import
    module = import_module(pkg, name, path, arch)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/guyer/mambaforge-arm/envs/scotch_eq_7/lib/python3.12/site-packages/petsc4py/lib/__init__.py", line 71, in import_module
    module = importlib.util.module_from_spec(spec)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: dlopen(/Users/guyer/mambaforge-arm/envs/scotch_eq_7/lib/python3.12/site-packages/petsc4py/lib/PETSc.cpython-312-darwin.so, 0x0002): symbol not found in flat namespace '_SCOTCH_errorPrint'

Environment created with

conda create --channel conda-forge --name scotch_eq_7 "scotch=7" petsc4py

Installed packages

# packages in environment at /Users/guyer/mambaforge-arm/envs/scotch_eq_7:
#
# Name                    Version                   Build  Channel
bzip2                     1.0.8                h93a5062_5    conda-forge
c-ares                    1.23.0               h93a5062_0    conda-forge
ca-certificates           2023.11.17           hf0a4a13_0    conda-forge
fftw                      3.3.10          mpi_openmpi_haef8dc3_8    conda-forge
gmp                       6.3.0                h965bd2d_0    conda-forge
hdf5                      1.14.3          mpi_openmpi_h20f603a_0    conda-forge
hypre                     2.28.0          mpi_openmpi_haba3941_0    conda-forge
icu                       73.2                 hc8870d7_0    conda-forge
krb5                      1.21.2               h92f50d5_0    conda-forge
libaec                    1.1.2                h13dd4ca_1    conda-forge
libblas                   3.9.0           20_osxarm64_openblas    conda-forge
libcblas                  3.9.0           20_osxarm64_openblas    conda-forge
libcurl                   8.5.0                h2d989ff_0    conda-forge
libcxx                    16.0.6               h4653b0c_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libexpat                  2.5.0                hb7217d7_1    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0           13_2_0_hd922786_1    conda-forge
libgfortran5              13.2.0               hf226fd6_1    conda-forge
libhwloc                  2.9.3           default_h4394839_1009    conda-forge
libiconv                  1.17                 he4db4b2_0    conda-forge
liblapack                 3.9.0           20_osxarm64_openblas    conda-forge
libnghttp2                1.58.0               ha4dd798_1    conda-forge
libopenblas               0.3.25          openmp_h6c19121_0    conda-forge
libptscotch               7.0.2                h4ec5e76_0    conda-forge
libscotch                 7.0.2                h80c162b_0    conda-forge
libsqlite                 3.44.2               h091b4b1_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libxml2                   2.11.6               h0d0cfa8_0    conda-forge
libzlib                   1.2.13               h53f4e23_5    conda-forge
llvm-openmp               17.0.6               hcd81f8e_0    conda-forge
metis                     5.1.1                h965bd2d_2    conda-forge
mpfr                      4.2.1                h9546428_0    conda-forge
mpi                       1.0                     openmpi    conda-forge
mumps-include             5.2.1               hce30654_13    conda-forge
mumps-mpi                 5.2.1               h00b355f_13    conda-forge
ncurses                   6.4                  h463b476_2    conda-forge
numpy                     1.26.2          py312h5d55045_0    conda-forge
openmpi                   4.1.6              h526c993_101    conda-forge
openssl                   3.2.0                h0d3ecfb_1    conda-forge
parmetis                  4.0.3             h6eb5794_1005    conda-forge
petsc                     3.20.2          real_hccb9628_102    conda-forge
petsc4py                  3.20.1          real_h17b5e51_100    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
ptscotch                  7.0.2                hc1c4572_0    conda-forge
python                    3.12.0          h47c9636_0_cpython    conda-forge
python_abi                3.12                    4_cp312    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
scalapack                 2.2.0                h515df86_1    conda-forge
scotch                    7.0.2                hc1c4572_0    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
suitesparse               5.10.1               h88be0ae_2    conda-forge
superlu                   5.2.2                hc615359_0    conda-forge
tbb                       2021.11.0            h6aa02a4_0    conda-forge
tk                        8.6.13               h5083fa2_1    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
yaml                      0.2.5                h3422bc3_2    conda-forge
zlib                      1.2.13               h53f4e23_5    conda-forge
zstd                      1.5.5                h4f39d0f_0    conda-forge

Environment info

active environment : scotch_eq_7
    active env location : /Users/guyer/mambaforge-arm/envs/scotch_eq_7
            shell level : 2
       user config file : /Users/guyer/.condarc
 populated config files : /Users/guyer/mambaforge-arm/.condarc
                          /Users/guyer/.condarc
          conda version : 23.7.4
    conda-build version : not installed
         python version : 3.10.10.final.0
       virtual packages : __archspec=1=arm64
                          __osx=13.6.2=0
                          __unix=0=0
       base environment : /Users/guyer/mambaforge-arm  (writable)
      conda av data dir : /Users/guyer/mambaforge-arm/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/guyer/mambaforge-arm/pkgs
                          /Users/guyer/.conda/pkgs
       envs directories : /Users/guyer/mambaforge-arm/envs
                          /Users/guyer/.conda/envs
               platform : osx-arm64
             user-agent : conda/23.7.4 requests/2.31.0 CPython/3.10.10 Darwin/22.6.0 OSX/13.6.2 solver/libmamba conda-libmamba-solver/23.3.0 libmambapy/1.5.1
                UID:GID : 17723:36677
             netrc file : None
           offline mode : False
guyer commented 9 months ago

Environment created with previous release is OK

conda create --channel conda-forge --name scotch_lt_7 "scotch<7.0" petsc4py

Installed packages

# packages in environment at /Users/guyer/mambaforge-arm/envs/scotch_lt_7:
#
# Name                    Version                   Build  Channel
bzip2                     1.0.8                h93a5062_5    conda-forge
c-ares                    1.23.0               h93a5062_0    conda-forge
ca-certificates           2023.11.17           hf0a4a13_0    conda-forge
fftw                      3.3.10          mpi_openmpi_haef8dc3_8    conda-forge
gmp                       6.3.0                h965bd2d_0    conda-forge
hdf5                      1.14.3          mpi_openmpi_h20f603a_0    conda-forge
hypre                     2.28.0          mpi_openmpi_haba3941_0    conda-forge
icu                       73.2                 hc8870d7_0    conda-forge
krb5                      1.21.2               h92f50d5_0    conda-forge
libaec                    1.1.2                h13dd4ca_1    conda-forge
libblas                   3.9.0           20_osxarm64_openblas    conda-forge
libcblas                  3.9.0           20_osxarm64_openblas    conda-forge
libcurl                   8.5.0                h2d989ff_0    conda-forge
libcxx                    16.0.6               h4653b0c_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libexpat                  2.5.0                hb7217d7_1    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0           13_2_0_hd922786_1    conda-forge
libgfortran5              13.2.0               hf226fd6_1    conda-forge
libhwloc                  2.9.3           default_h4394839_1009    conda-forge
libiconv                  1.17                 he4db4b2_0    conda-forge
liblapack                 3.9.0           20_osxarm64_openblas    conda-forge
libnghttp2                1.58.0               ha4dd798_1    conda-forge
libopenblas               0.3.25          openmp_h6c19121_0    conda-forge
libsqlite                 3.44.2               h091b4b1_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libxml2                   2.11.6               h0d0cfa8_0    conda-forge
libzlib                   1.2.13               h53f4e23_5    conda-forge
llvm-openmp               17.0.6               hcd81f8e_0    conda-forge
metis                     5.1.0             h13dd4ca_1007    conda-forge
mpfr                      4.2.1                h9546428_0    conda-forge
mpi                       1.0                     openmpi    conda-forge
mumps-include             5.2.1               hce30654_11    conda-forge
mumps-mpi                 5.2.1               h51c74a9_11    conda-forge
ncurses                   6.4                  h463b476_2    conda-forge
numpy                     1.26.2          py312h5d55045_0    conda-forge
openmpi                   4.1.6              h526c993_101    conda-forge
openssl                   3.2.0                h0d3ecfb_1    conda-forge
parmetis                  4.0.3             h6eb5794_1005    conda-forge
petsc                     3.20.2          real_hcab63ac_100    conda-forge
petsc4py                  3.20.1          real_h17b5e51_100    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
ptscotch                  6.0.9                h1aec651_2    conda-forge
python                    3.12.0          h47c9636_0_cpython    conda-forge
python_abi                3.12                    4_cp312    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
scalapack                 2.2.0                h515df86_1    conda-forge
scotch                    6.0.9                h7537618_2    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
suitesparse               5.10.1               h7cd81ec_1    conda-forge
superlu                   5.2.2                hc615359_0    conda-forge
superlu_dist              7.2.0                hfdb8677_0    conda-forge
tbb                       2021.11.0            h6aa02a4_0    conda-forge
tk                        8.6.13               h5083fa2_1    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
yaml                      0.2.5                h3422bc3_2    conda-forge
zlib                      1.2.13               h53f4e23_5    conda-forge
zstd                      1.5.5                h4f39d0f_0    conda-forge

Environment info


     active environment : scotch_lt_7
    active env location : /Users/guyer/mambaforge-arm/envs/scotch_lt_7
            shell level : 2
       user config file : /Users/guyer/.condarc
 populated config files : /Users/guyer/mambaforge-arm/.condarc
                          /Users/guyer/.condarc
          conda version : 23.7.4
    conda-build version : not installed
         python version : 3.10.10.final.0
       virtual packages : __archspec=1=arm64
                          __osx=13.6.2=0
                          __unix=0=0
       base environment : /Users/guyer/mambaforge-arm  (writable)
      conda av data dir : /Users/guyer/mambaforge-arm/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/guyer/mambaforge-arm/pkgs
                          /Users/guyer/.conda/pkgs
       envs directories : /Users/guyer/mambaforge-arm/envs
                          /Users/guyer/.conda/envs
               platform : osx-arm64
             user-agent : conda/23.7.4 requests/2.31.0 CPython/3.10.10 Darwin/22.6.0 OSX/13.6.2 solver/libmamba conda-libmamba-solver/23.3.0 libmambapy/1.5.1
                UID:GID : 17723:36677
             netrc file : None
           offline mode : False
dalcinl commented 9 months ago

Let see if conda-forge/petsc4py-feedstock#84 solves the issue.

dalcinl commented 9 months ago

Indeed, there seem to be issues with macOS, I can see the Symbol not found: _SCOTCH_errorPrint in the build logs. Unfortunately, I will not have time to figure out what's going on until end January. You will have to ask other petsc/petsc4py recipe maintainers for help.

guyer commented 9 months ago

For the time being, I'm fine with pinning the previous release. I just wanted to be sure you knew about it.

minrk commented 9 months ago

I believe this is an issue in scotch itself where libscotcherr wasn't linked from libscotch (possibly due to conda-forge -Wl,dead_strip_dylibs combined with something weird in how libscotch splits its functionality), but it should be fixed by the 7.0.4 migration, which should start today https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/4807

> otool -L ~/conda/envs/scotch702/lib/libscotch.dylib
/Users/minrk/conda/envs/scotch702/lib/libscotch.dylib:
    @rpath/libscotch.7.0.2.dylib (compatibility version 0.0.0, current version 7.0.2)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.0.0)
    @rpath/libz.1.dylib (compatibility version 1.0.0, current version 1.2.13)
    @rpath/libbz2.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/liblzma.5.dylib (compatibility version 8.0.0, current version 8.6.0)
> otool -L ~/conda/envs/scotch/lib/libscotch.dylib
/Users/minrk/conda/envs/scotch/lib/libscotch.dylib:
    @rpath/libscotch.7.0.4.dylib (compatibility version 0.0.0, current version 7.0.4)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.0.0)
    @rpath/libscotcherr.dylib (compatibility version 0.0.0, current version 0.0.0) <---- missing from 7.0.2
    @rpath/libz.1.dylib (compatibility version 1.0.0, current version 1.2.13)
    @rpath/libbz2.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/liblzma.5.dylib (compatibility version 8.0.0, current version 8.6.0)
minrk commented 9 months ago

7.0.4 builds should fix this when they arrive

guyer commented 9 months ago

7.0.4 build works for me, thanks