search

explosion / cython-blis

💥 Fast matrix-multiplication as a self-contained Python library – no system dependencies!
Other
218 stars 37 forks source link

Mac M1 Chip Support old versions of blis #64

Closed doomlab closed 2 years ago

doomlab commented 2 years ago

Hi guys,

I am trying to install spacy 2.2.3 because I use it to teach classes. I upgraded to a new M1 Max Mac, which is supported for newer versions of blis, but I am trying to install 0.4.0 due to spacy requirements. Any suggestions?

Defaulting to user installation because normal site-packages is not writeable
Collecting blis==0.4.0
  Using cached blis-0.4.0.tar.gz (1.8 MB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy>=1.15.0 in ./Library/Python/3.8/lib/python/site-packages (from blis==0.4.0) (1.21.5)
Building wheels for collected packages: blis
  Building wheel for blis (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Library/Developer/CommandLineTools/usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-install-z7uivl4e/blis_3824fe6c68df47c0b5d8286145a0069a/setup.py'"'"'; __file__='"'"'/private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-install-z7uivl4e/blis_3824fe6c68df47c0b5d8286145a0069a/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-wheel-v9hgnksk
       cwd: /private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-install-z7uivl4e/blis_3824fe6c68df47c0b5d8286145a0069a/
  Complete output (23 lines):
  BLIS_COMPILER? None
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.14-arm64-3.8
  creating build/lib.macosx-10.14-arm64-3.8/blis
  copying blis/benchmark.py -> build/lib.macosx-10.14-arm64-3.8/blis
  copying blis/__init__.py -> build/lib.macosx-10.14-arm64-3.8/blis
  copying blis/about.py -> build/lib.macosx-10.14-arm64-3.8/blis
  creating build/lib.macosx-10.14-arm64-3.8/blis/tests
  copying blis/tests/__init__.py -> build/lib.macosx-10.14-arm64-3.8/blis/tests
  copying blis/tests/test_dotv.py -> build/lib.macosx-10.14-arm64-3.8/blis/tests
  copying blis/tests/test_gemm.py -> build/lib.macosx-10.14-arm64-3.8/blis/tests
  copying blis/tests/common.py -> build/lib.macosx-10.14-arm64-3.8/blis/tests
  copying blis/cy.pyx -> build/lib.macosx-10.14-arm64-3.8/blis
  copying blis/py.pyx -> build/lib.macosx-10.14-arm64-3.8/blis
  copying blis/__init__.pxd -> build/lib.macosx-10.14-arm64-3.8/blis
  copying blis/cy.pxd -> build/lib.macosx-10.14-arm64-3.8/blis
  running build_ext
  unix
  py_compiler gcc
  error: [Errno 2] No such file or directory: '/private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-install-z7uivl4e/blis_3824fe6c68df47c0b5d8286145a0069a/blis/_src/make/darwin-generic.jsonl'
  ----------------------------------------
  ERROR: Failed building wheel for blis
  Running setup.py clean for blis
Failed to build blis
Installing collected packages: blis
    Running setup.py install for blis ... error
    ERROR: Command errored out with exit status 1:
     command: /Library/Developer/CommandLineTools/usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-install-z7uivl4e/blis_3824fe6c68df47c0b5d8286145a0069a/setup.py'"'"'; __file__='"'"'/private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-install-z7uivl4e/blis_3824fe6c68df47c0b5d8286145a0069a/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-record-msptrtm2/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /Users/erinbuchanan/Library/Python/3.8/include/python3.8/blis
         cwd: /private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-install-z7uivl4e/blis_3824fe6c68df47c0b5d8286145a0069a/
    Complete output (23 lines):
    BLIS_COMPILER? None
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.14-arm64-3.8
    creating build/lib.macosx-10.14-arm64-3.8/blis
    copying blis/benchmark.py -> build/lib.macosx-10.14-arm64-3.8/blis
    copying blis/__init__.py -> build/lib.macosx-10.14-arm64-3.8/blis
    copying blis/about.py -> build/lib.macosx-10.14-arm64-3.8/blis
    creating build/lib.macosx-10.14-arm64-3.8/blis/tests
    copying blis/tests/__init__.py -> build/lib.macosx-10.14-arm64-3.8/blis/tests
    copying blis/tests/test_dotv.py -> build/lib.macosx-10.14-arm64-3.8/blis/tests
    copying blis/tests/test_gemm.py -> build/lib.macosx-10.14-arm64-3.8/blis/tests
    copying blis/tests/common.py -> build/lib.macosx-10.14-arm64-3.8/blis/tests
    copying blis/cy.pyx -> build/lib.macosx-10.14-arm64-3.8/blis
    copying blis/py.pyx -> build/lib.macosx-10.14-arm64-3.8/blis
    copying blis/__init__.pxd -> build/lib.macosx-10.14-arm64-3.8/blis
    copying blis/cy.pxd -> build/lib.macosx-10.14-arm64-3.8/blis
    running build_ext
    unix
    py_compiler gcc
    error: [Errno 2] No such file or directory: '/private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-install-z7uivl4e/blis_3824fe6c68df47c0b5d8286145a0069a/blis/_src/make/darwin-generic.jsonl'
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Library/Developer/CommandLineTools/usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-install-z7uivl4e/blis_3824fe6c68df47c0b5d8286145a0069a/setup.py'"'"'; __file__='"'"'/private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-install-z7uivl4e/blis_3824fe6c68df47c0b5d8286145a0069a/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/2k/dx83q1d91gld71_lk76chpdw0000gn/T/pip-record-msptrtm2/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /Users/erinbuchanan/Library/Python/3.8/include/python3.8/blis Check the logs for full command output.
adrianeboyd commented 2 years ago

It should not cause any technical problems to run spacy v2.2.3 with the most recent version of blis, which should compile more easily, but it would involve installing the other packages with pip install --no-deps --no-build-isolation and installing dependencies by hand in order to get around some of the conflicts that pip reports. Basically, you would install all of these packages by hand with --no-deps --no-build-isolation up to "Optional dependencies" along with a newer version of blis:

https://github.com/explosion/spaCy/blob/v2.2.3/requirements.txt

The basic order to install should be: numpy, blis, murmurhash, cymem, preshed, srsly, thinc, spacy, plus all the remaining python-only packages. And I'm not 100% sure every single one of these packages can be installed easily from the source distributions with pip install. I overhauled all the packaging internals about a year ago, so any releases newer than Dec. 2020 should work more consistently.

Instead of doing this, I would strongly suggest upgrading your classes to the most recent v2.3.x (currently v2.3.7), where the install will be easier, also for students who have M1s. If you want to use conda, you can also install from binary packages from conda-forge instead of dealing with compiling from source. The miniforge osx arm64 conda installer works well on M1s and there are packages for spacy v2.3.5+.

And it probably isn't particularly relevant for teaching, but spacy v2 is going to be slow with any version of blis because the hardware isn't supported well. For v3 you can use thinc-apple-ops to speed it up.

adrianeboyd commented 2 years ago

Ah, and I forgot about a few bugs in older versions of spacy on arm64. Those bugs (mainly in the Matcher) are first fixed in v2.3.3.

doomlab commented 2 years ago

Thanks @adrianeboyd - I ended up testing the 2.3.7 version and my code works fine (for some reason, I thought we were version locked, but thankfully not). Appreciate your help/comments.