Installation trouble: ModuleNotFoundError: No module named 'distutils.msvccompiler'

[Gilles86]

Dear pycortex team,

I'm trying to (re-)install pycortex on a M1 Macbook running Mac OS X 12.6 for 90 minutes now, but I just cannot pull it off. I stopped having fun.

I keep getting the following error when compiling (cloned the repo and pip install -e .).

File "/private/var/folders/d4/cpqhwlbn301clph1qk64k01nmjkty3/T/pip-build-env-yd2onq5x/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 557, in build_extension
          objects = self.compiler.compile(
        File "/private/var/folders/d4/cpqhwlbn301clph1qk64k01nmjkty3/T/pip-build-env-yd2onq5x/overlay/lib/python3.8/site-packages/numpy/distutils/ccompiler.py", line 89, in <lambda>
          m = lambda self, *args, **kw: func(self, *args, **kw)
        File "/private/var/folders/d4/cpqhwlbn301clph1qk64k01nmjkty3/T/pip-build-env-yd2onq5x/overlay/lib/python3.8/site-packages/numpy/distutils/ccompiler.py", line 272, in CCompiler_compile
          jobs = get_num_build_jobs()
        File "/private/var/folders/d4/cpqhwlbn301clph1qk64k01nmjkty3/T/pip-build-env-yd2onq5x/overlay/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 91, in get_num_build_jobs
          from numpy.distutils.core import get_distribution
        File "/private/var/folders/d4/cpqhwlbn301clph1qk64k01nmjkty3/T/pip-build-env-yd2onq5x/overlay/lib/python3.8/site-packages/numpy/distutils/core.py", line 24, in <module>
          from numpy.distutils.command import config, config_compiler, \
        File "/private/var/folders/d4/cpqhwlbn301clph1qk64k01nmjkty3/T/pip-build-env-yd2onq5x/overlay/lib/python3.8/site-packages/numpy/distutils/command/config.py", line 19, in <module>
          from numpy.distutils.mingw32ccompiler import generate_manifest
        File "/private/var/folders/d4/cpqhwlbn301clph1qk64k01nmjkty3/T/pip-build-env-yd2onq5x/overlay/lib/python3.8/site-packages/numpy/distutils/mingw32ccompiler.py", line 28, in <module>
          from distutils.msvccompiler import get_build_version as get_build_msvc_version
      ModuleNotFoundError: No module named 'distutils.msvccompiler'

I guess it's related to this issue https://numpy.org/doc/stable/reference/distutils_status_migration.html#distutils-status-migration

I tried many different strategies now, using completely empty conda environments for python=3.8/3.9/3.10 (It's very hard to get anything below 3.8 these days). I also tried downgrading setuptools to 59 or 64, as suggested on the internet, as well as downgrading numpy to 1.22.

If I do pip install -U git+https://github.com/gallantlab/pycortex.git --no-build-isolation I get

      [1/2] Cythonizing cortex/formats.pyx
      [2/2] Cythonizing cortex/openctm.pyx
      Traceback (most recent call last):
        File "/Users/gdehol/mambaforge/envs/prf_expect/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/gdehol/mambaforge/envs/prf_expect/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/Users/gdehol/mambaforge/envs/prf_expect/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/Users/gdehol/mambaforge/envs/prf_expect/lib/python3.9/site-packages/setuptools/build_meta.py", line 368, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/Users/gdehol/mambaforge/envs/prf_expect/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 131, in <module>
        File "/Users/gdehol/mambaforge/envs/prf_expect/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
          cythonize_one(*args)
        File "/Users/gdehol/mambaforge/envs/prf_expect/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: cortex/openctm.pyx
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

Do you have any insight here?

Cheers, Gilles

[NKUShaw]

same error

[mvdoc]

I can replicate the same error when installing pycortex from pypi. While we investigate that problem, I just pushed a commit that should fix the error when installing pycortex from the github repository. The following worked for me:

git clone https://github.com/gallantlab/pycortex.git
cd pycortex
conda create --name pycortex python=3.11
conda activate pycortex
pip install .

Let us know if this works for you too.

[mvdoc]

Version 1.2.9 of pycortex (just released on pypi) should also fix this issue. You can try with the following

conda create --name pycortexpip python=3.11
conda activate pycortexpip
pip install pycortex

[mvdoc]

With the fixes in #553, this issue should be resolved also in python 3.12. I'm closing this issue, but please let us know if you still encounter this error.

[Gilles86]

Now I get this

      x86_64-apple-darwin13.4.0-clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /Users/gdehol/mambaforge/envs/retinonumeral/include -arch arm64 -fPIC -O2 -isystem /Users/gdehol/mambaforge/envs/retinonumeral/include -arch arm64 -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fstack-protector-strong -O2 -pipe -isystem /Users/gdehol/mambaforge/envs/retinonumeral/include -D_FORTIFY_SOURCE=2 -isystem /Users/gdehol/mambaforge/envs/retinonumeral/include -DLZMA_PREFIX_CTM -DOPENCTM_BUILD -IOpenCTM-1.0.3/lib/ -IOpenCTM-1.0.3/lib/liblzma/ -I/Users/gdehol/mambaforge/envs/retinonumeral/lib/python3.12/site-packages/numpy/core/include -I/Users/gdehol/mambaforge/envs/retinonumeral/include/python3.12 -c OpenCTM-1.0.3/lib/compressMG1.c -o build/temp.macosx-11.0-arm64-cpython-312/OpenCTM-1.0.3/lib/compressMG1.o
      x86_64-apple-darwin13.4: error: unsupported argument 'core2' to option '-march='
      error: command '/Users/gdehol/mambaforge/envs/retinonumeral/bin/x86_64-apple-darwin13.4.0-clang' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pycortex
Failed to build pycortex
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (pycortex) 

[mvdoc]

That looks like a compiler error specific to mac and possibly unrelated to the distutils problem. However, I'm surprised because I can install pycortex on my mac m1 just fine.

Can you try installing pycortex in a completely new conda environment? If that still fails, maybe uninstalling and reinstalling the mac command line tools may help.