search

vtraag / leidenalg

Implementation of the Leiden algorithm for various quality functions to be used with igraph in Python.
GNU General Public License v3.0
575 stars 77 forks source link

leidenalg-0.8.8 failed to be updated in M1 Apple #88

Closed akhst7 closed 2 years ago

akhst7 commented 2 years ago

I have a leidenalg-0.8.7 installed and was trying to update to 0.8.8 but it failed to compile with a following error;

It looks like igraph is a culprit. 

running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-11-arm64-3.9
    creating build/lib.macosx-11-arm64-3.9/leidenalg
    copying src/leidenalg/functions.py -> build/lib.macosx-11-arm64-3.9/leidenalg
    copying src/leidenalg/Optimiser.py -> build/lib.macosx-11-arm64-3.9/leidenalg
    copying src/leidenalg/VertexPartition.py -> build/lib.macosx-11-arm64-3.9/leidenalg
    copying src/leidenalg/version.py -> build/lib.macosx-11-arm64-3.9/leidenalg
    copying src/leidenalg/__init__.py -> build/lib.macosx-11-arm64-3.9/leidenalg
    running build_ext
    running build_c_core
    We are going to build the C core of igraph.
      Source folder: vendor/source/igraph
      Build folder: vendor/build/igraph
      Install folder: vendor/install/igraph

    Configuring build...
    -- Version number: 0.9.1-12-gd829842c9
    --
    -- -----[ Build configuration ]----
    -- Version:     0.9.1-12-gd829842c9
    -- CMake build type:        Release
    -- Library type:             static
    --
    -- ----------[ Features ]----------
    -- GLPK for optimization:       yes
    -- Reading GraphML files:       yes
    -- Thread-local storage:         no
    -- Link-time optimization:       no
    --
    -- --------[ Dependencies ]--------
    -- ARPACK:                 vendored
    -- BISON:                       yes
    -- BLAS:                   vendored
    -- CXSparse:               vendored
    -- FLEX:                        yes
    -- GLPK:                   vendored
    -- GMP:                    vendored
    -- LAPACK:                 vendored
    -- LibXml2:                     yes
    --
    -- -----------[ Testing ]----------
    -- Diff tool:                  diff
    -- Sanitizers:                 none
    -- Code coverage:                no
    -- Verify 'finally' stack:       no
    --
    -- --------[ Documentation ]-------
    -- HTML:                         no
    -- PDF:                          no
    --
    -- igraph configured successfully.
    --
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /private/var/folders/sl/253njlzx5797z3x9nlhrs79m0000gn/T/pip-install-6ml452hp/leidenalg_806fa64602674ba1966e4361305233ae/vendor/build/igraph
    Running build...
    Consolidate compiler generated dependencies of target plfit
    [  2%] Built target plfit
    Consolidate compiler generated dependencies of target prpack
    [  4%] Built target prpack
    Consolidate compiler generated dependencies of target cliquer
    [  4%] Built target cliquer
    Consolidate compiler generated dependencies of target bliss
    [  4%] Built target bliss
    Consolidate compiler generated dependencies of target cxsparse_vendored
    [ 12%] Built target cxsparse_vendored
    Consolidate compiler generated dependencies of target arithchk
    [ 12%] Built target arithchk
    Consolidate compiler generated dependencies of target f2c_vendored
    [ 32%] Built target f2c_vendored
    Consolidate compiler generated dependencies of target glpk_vendored
    [ 48%] Built target glpk_vendored
    Consolidate compiler generated dependencies of target blas_vendored
    [ 53%] Built target blas_vendored
    Consolidate compiler generated dependencies of target lapack_vendored
    [ 67%] Built target lapack_vendored
    Consolidate compiler generated dependencies of target arpack_vendored
    [ 69%] Built target arpack_vendored
    Consolidate compiler generated dependencies of target gmp_vendored
    [ 69%] Built target gmp_vendored
    Consolidate compiler generated dependencies of target igraph
    [ 69%] Building CXX object src/CMakeFiles/igraph.dir/games/degree_sequence_vl/gengraph_graph_molloy_optimized.cpp.o
    /private/var/folders/sl/253njlzx5797z3x9nlhrs79m0000gn/T/pip-install-6ml452hp/leidenalg_806fa64602674ba1966e4361305233ae/vendor/source/igraph/src/games/degree_sequence_vl/gengraph_graph_molloy_optimized.cpp:1800:13: error: variable 'sum' set but not used [-Werror,-Wunused-but-set-variable]
            int sum = 0;
                ^
    1 error generated.
    make[2]: *** [src/CMakeFiles/igraph.dir/games/degree_sequence_vl/gengraph_graph_molloy_optimized.cpp.o] Error 1
    make[1]: *** [src/CMakeFiles/igraph.dir/all] Error 2
    make: *** [all] Error 2
    error: [Errno 2] No such file or directory: '/private/var/folders/sl/253njlzx5797z3x9nlhrs79m0000gn/T/pip-install-6ml452hp/leidenalg_806fa64602674ba1966e4361305233ae/vendor/install/igraph/build.cfg'

My Macmini setup ;

neofetch
                    'c.          akihikohoji@Akihikos-Mini
                 ,xNMM.          -------------------------
               .OMMMMo           OS: macOS 11.6 20G165 arm64
               OMMM0,            Host: Macmini9,1
     .;loddo:' loolloddol;.      Kernel: 20.6.0
   cKMMMMMMMMMMNWMMMMMMMMMM0:    Uptime: 10 days, 3 hours, 43 mins
 .KMMMMMMMMMMMMMMMMMMMMMMMWd.    Packages: 74 (brew)
 XMMMMMMMMMMMMMMMMMMMMMMMX.      Shell: zsh 5.8
;MMMMMMMMMMMMMMMMMMMMMMMM:       Resolution: 1920x1080
:MMMMMMMMMMMMMMMMMMMMMMMM:       DE: Aqua
.MMMMMMMMMMMMMMMMMMMMMMMMX.      WM: Quartz Compositor
 kMMMMMMMMMMMMMMMMMMMMMMMMWd.    WM Theme: Blue (Light)
 .XMMMMMMMMMMMMMMMMMMMMMMMMMMk   Terminal: iTerm2
  .XMMMMMMMMMMMMMMMMMMMMMMMMK.   Terminal Font: Courier 14
    kMMMMMMMMMMMMMMMMMMMMMMd     CPU: Apple M1
     ;KMMMMMMMWXXWMMMMMMMk.      GPU: Apple M1
       .cooc,.    .,coo:.        Memory: 2798MiB / 16384MiB

Compiler:

❯ clang -v
Homebrew clang version 13.0.0
Target: arm64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin

compiler flags

LDFLAGS=-L/opt/homebrew/lib
CPPFLAGS=-I/opt/homebrew/include
JAVA_HOME=/Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/home/jre
PKG_CONFIG_PATH=/opt/homebrew/Cellar/openblas/0.3.15_1/lib/pkgconfig /opt/homebrew/opt/lapack/lib/pkgconfig
OPENBLAS=/opt/homebrew/opt/openblas
CMAKE_CXX_COMPILER=/opt/homebrew/opt/llvm/bin/clang++
CXX=/opt/homebrew/opt/llvm/bin/clang++
vtraag commented 2 years ago

The igraph C core remained the same for versions 0.8.7 and 0.8.8. I would have already expected this error to surface in 0.8.7. Can you double check whether you are able to compile 0.8.7?

Compiling for ARM64 Mac is more difficult in the CI unfortunately, otherwise I would have provided pre-compiled binary wheels. There are ARM64 Mac builds available from Anaconda (conda-forge).

akhst7 commented 2 years ago

@vtraag

0.8.7 was complied on my M1 mini though PIP. So you did not change the code in igraph core. I will take a look at lines around gengraph_graph_molloy_optimized.cpp:1800:13:, which appears to throw errors. It could be a compiler flag issue since I use Clang over gcc. I will see if I could dig deeper.

Conda-forge builds for the aarch Macs are mess.

Thanks.

vtraag commented 2 years ago

Could you try to compile the latest version of igraph? There were some improvements related to supporting ARM64 Mac, I could pull in a new C core of igraph to correct those.

Conda-forge builds for the aarch Macs are mess.

Could you elaborate on this? Is there anything with (python-)igraphthat should be improved in conda-forge? Or with leidenalg?

akhst7 commented 2 years ago

@vtraag

python-igraph is already installed by pip. So the error must be from the code within C core of igraph that comes with leidenalg-0.8.8 ? Is it possible to compile leidenalg-0.8.8 without igraph ?

Regarding conda-forge Apple Arm64 binaries, I was talking in general, not specific for you packages. A majority of those packages are by no means stable. Some of them don't run at all in my M1 mini. As you can see below, many of python3.9 packages can natively be complied though pip as wheels and they are stable. I just need a few more packages such as llvmlite and number for what I need. 

❯ pip list
Package          Version
---------------- -------------------------
anndata          0.7.7.dev2+ge7a3a3e
blessings        1.7
cycler           0.10.0
Cython           0.29.24
gpustat          0.6.0
h5py             3.5.0
ipython-genutils 0.2.0
joblib           1.1.0
kiwisolver       1.3.2
leidenalg        0.8.7
matplotlib       3.4.3
natsort          7.1.1
numexpr          2.7.3
numpy            1.22.0.dev0+90.g21838b4b2
nvidia-ml-py3    7.352.0
packaging        21.0
pandas           1.3.4
patsy            0.5.2
Pillow           8.4.0
pip              21.3
protobuf         3.19.0
psutil           5.8.0
pyparsing        2.4.7
pyrsistent       0.18.0
python-dateutil  2.8.2
python-igraph    0.9.7
pytz             2021.3
scikit-learn     1.0
scipy            1.8.0.dev0+0.ebebeac
setuptools       58.2.0
setuptools-scm   6.3.2
six              1.16.0
statsmodels      0.13.0
tables           3.6.2.dev0
TBB              0.2
texttable        1.6.4
threadpoolctl    3.0.0
tomli            1.2.1
traitlets        5.1.0
wheel            0.37.0
xlrd             2.0.1
vtraag commented 2 years ago

python-igraph is already installed by pip. So the error must be from the code within C core of igraph that comes with leidenalg-0.8.8 ? Is it possible to compile leidenalg-0.8.8 without igraph ?

There are binary wheels for python-igraph provided also for ARM64 Mac (at least the most recent version), so it does not involve any compilation. You can force compilation by using --no-binary when pip installing python-igraph.

leidenalg and python-igraph both compile the C core of igraph. The version of the C core of leidenalg lags behind a bit, while the version included with python-igraph is more recent. Possibly, compilation of the more recent C core does not show the problems you encountered. I can also update the C core in leidenalg, so that you can test it, but I'm unable to do so now, and can only do that next week. If you can compile python-igraph, it is clear that that is the problem.

vtraag commented 2 years ago

I've updated the igraph C core of leidenalg to the latest version 0.9.5 that was just released. We made some corrections there for compiling with clang 13 which seemed to have caused your compilation problems as well. Could you perhaps test whether that is the case? You can install directly from github using:

pip install git+https://github.com/vtraag/leidenalg.git
akhst7 commented 2 years ago

@vtraag

It installed without a hitch !! Thanks. 

❯ pip install git+https://github.com/vtraag/leidenalg.git
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
Collecting git+https://github.com/vtraag/leidenalg.git
  Cloning https://github.com/vtraag/leidenalg.git to /private/var/folders/sl/253njlzx5797z3x9nlhrs79m0000gn/T/pip-req-build-gp2nbbwl
  Running command git clone --filter=blob:none -q https://github.com/vtraag/leidenalg.git /private/var/folders/sl/253njlzx5797z3x9nlhrs79m0000gn/T/pip-req-build-gp2nbbwl
  Resolved https://github.com/vtraag/leidenalg.git to commit b53366829360e10922a2dbf57eb405a516c23bc9
  Running command git submodule update --init --recursive -q
  Preparing metadata (setup.py) ... done
Requirement already satisfied: python-igraph>=0.9.0 in /opt/homebrew/lib/python3.9/site-packages (from leidenalg==0.8.9.dev1+gb533668) (0.9.7)
Requirement already satisfied: texttable>=1.6.2 in /opt/homebrew/lib/python3.9/site-packages (from python-igraph>=0.9.0->leidenalg==0.8.9.dev1+gb533668) (1.6.4)
Building wheels for collected packages: leidenalg
  Building wheel for leidenalg (setup.py) ... done
  Created wheel for leidenalg: filename=leidenalg-0.8.9.dev1+gb533668-cp39-cp39-macosx_11_0_arm64.whl size=204396 sha256=5b6d77ded58d005b49960921fb70d6f01094669e900f1e97b8836ec8f5463e20
  Stored in directory: /private/var/folders/sl/253njlzx5797z3x9nlhrs79m0000gn/T/pip-ephem-wheel-cache-62__ghrg/wheels/f8/85/e0/b609612aa57f8714297aacba413089790a069a94149a18bd17
Successfully built leidenalg
Installing collected packages: leidenalg
  Attempting uninstall: leidenalg
    Found existing installation: leidenalg 0.8.7
    Uninstalling leidenalg-0.8.7:
      Successfully uninstalled leidenalg-0.8.7
  DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
Successfully installed leidenalg-0.8.9.dev1+gb533668