[BUG] Installation fails on macOS 12.6.7

[man00ka]

Describe the bug pip install contextualSpellCheck fails on macOS 12.6.7; M1 Pro MacBook Pro TL;DR:

editdistance/bycython.cpp:216:12: fatal error: 'longintrepr.h' file not found

I guess it's less of a bug than just the fact that it does not build for macOS 12.x yet?

To Reproduce

$ pip install contextualSpellCheck

Expected behavior Successful installation

Version (please complete the following information):

• contextualSpellCheck [0.4.3]
• Spacy: [3.6.0]
• transformers [4.30.2]
• Command Line Tools for Xcode []

Additional information

• I'm using miniforge3

• This is the complete install prompt:

  $ pip install contextualSpellCheck
  Collecting contextualSpellCheck
  Using cached contextualSpellCheck-0.4.3-py3-none-any.whl (128 kB)
  Collecting torch>=1.4 (from contextualSpellCheck)
  Using cached torch-2.0.1-cp311-none-macosx_11_0_arm64.whl (55.8 MB)
  Collecting editdistance==0.6.0 (from contextualSpellCheck)
  Using cached editdistance-0.6.0.tar.gz (29 kB)
  Preparing metadata (setup.py) ... done
  Collecting transformers>=4.0.0 (from contextualSpellCheck)
  Using cached transformers-4.30.2-py3-none-any.whl (7.2 MB)
  Requirement already satisfied: spacy>=3.0.0 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from contextualSpellCheck) (3.6.0)
  Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (3.0.12)
  Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (1.0.4)
  Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (1.0.9)
  Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (2.0.7)
  Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (3.0.8)
  Requirement already satisfied: thinc<8.2.0,>=8.1.8 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (8.1.10)
  Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (1.1.2)
  Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (2.4.6)
  Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (2.0.8)
  Requirement already satisfied: typer<0.10.0,>=0.3.0 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (0.9.0)
  Requirement already satisfied: pathy>=0.10.0 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (0.10.2)
  Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (5.2.1)
  Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (4.65.0)
  Requirement already satisfied: numpy>=1.15.0 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (1.25.1)
  Requirement already satisfied: requests<3.0.0,>=2.13.0 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (2.31.0)
  Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.11.0,>=1.7.4 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (1.10.11)
  Requirement already satisfied: jinja2 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (3.1.2)
  Requirement already satisfied: setuptools in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (68.0.0)
  Requirement already satisfied: packaging>=20.0 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (23.1)
  Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from spacy>=3.0.0->contextualSpellCheck) (3.3.0)
  Collecting filelock (from torch>=1.4->contextualSpellCheck)
  Using cached filelock-3.12.2-py3-none-any.whl (10 kB)
  Requirement already satisfied: typing-extensions in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from torch>=1.4->contextualSpellCheck) (4.7.1)
  Collecting sympy (from torch>=1.4->contextualSpellCheck)
  Using cached sympy-1.12-py3-none-any.whl (5.7 MB)
  Collecting networkx (from torch>=1.4->contextualSpellCheck)
  Using cached networkx-3.1-py3-none-any.whl (2.1 MB)
  Collecting huggingface-hub<1.0,>=0.14.1 (from transformers>=4.0.0->contextualSpellCheck)
  Using cached huggingface_hub-0.16.4-py3-none-any.whl (268 kB)
  Collecting pyyaml>=5.1 (from transformers>=4.0.0->contextualSpellCheck)
  Using cached PyYAML-6.0-cp311-cp311-macosx_11_0_arm64.whl (167 kB)
  Collecting regex!=2019.12.17 (from transformers>=4.0.0->contextualSpellCheck)
  Using cached regex-2023.6.3-cp311-cp311-macosx_11_0_arm64.whl (288 kB)
  Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers>=4.0.0->contextualSpellCheck)
  Using cached tokenizers-0.13.3-cp311-cp311-macosx_12_0_arm64.whl (3.9 MB)
  Collecting safetensors>=0.3.1 (from transformers>=4.0.0->contextualSpellCheck)
  Using cached safetensors-0.3.1-cp311-cp311-macosx_12_0_arm64.whl (401 kB)
  Collecting fsspec (from huggingface-hub<1.0,>=0.14.1->transformers>=4.0.0->contextualSpellCheck)
  Using cached fsspec-2023.6.0-py3-none-any.whl (163 kB)
  Requirement already satisfied: charset-normalizer<4,>=2 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy>=3.0.0->contextualSpellCheck) (3.2.0)
  Requirement already satisfied: idna<4,>=2.5 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy>=3.0.0->contextualSpellCheck) (3.4)
  Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy>=3.0.0->contextualSpellCheck) (2.0.3)
  Requirement already satisfied: certifi>=2017.4.17 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy>=3.0.0->contextualSpellCheck) (2023.5.7)
  Requirement already satisfied: blis<0.8.0,>=0.7.8 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from thinc<8.2.0,>=8.1.8->spacy>=3.0.0->contextualSpellCheck) (0.7.9)
  Requirement already satisfied: confection<1.0.0,>=0.0.1 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from thinc<8.2.0,>=8.1.8->spacy>=3.0.0->contextualSpellCheck) (0.1.0)
  Requirement already satisfied: click<9.0.0,>=7.1.1 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from typer<0.10.0,>=0.3.0->spacy>=3.0.0->contextualSpellCheck) (8.1.4)
  Requirement already satisfied: MarkupSafe>=2.0 in /Users/manu_neu/miniforge3/envs/spaCy/lib/python3.11/site-packages (from jinja2->spacy>=3.0.0->contextualSpellCheck) (2.1.3)
  Collecting mpmath>=0.19 (from sympy->torch>=1.4->contextualSpellCheck)
  Using cached mpmath-1.3.0-py3-none-any.whl (536 kB)
  Building wheels for collected packages: editdistance
  Building wheel for editdistance (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build/lib.macosx-11.0-arm64-cpython-311
    creating build/lib.macosx-11.0-arm64-cpython-311/editdistance
    copying editdistance/__init__.py -> build/lib.macosx-11.0-arm64-cpython-311/editdistance
    copying editdistance/_editdistance.h -> build/lib.macosx-11.0-arm64-cpython-311/editdistance
    copying editdistance/def.h -> build/lib.macosx-11.0-arm64-cpython-311/editdistance
    running build_ext
    building 'editdistance.bycython' extension
    creating build/temp.macosx-11.0-arm64-cpython-311
    creating build/temp.macosx-11.0-arm64-cpython-311/editdistance
    clang -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /Users/manu_neu/miniforge3/envs/spaCy/include -arch arm64 -fPIC -O2 -isystem /Users/manu_neu/miniforge3/envs/spaCy/include -arch arm64 -I./editdistance -I/Users/manu_neu/miniforge3/envs/spaCy/include/python3.11 -c editdistance/_editdistance.cpp -o build/temp.macosx-11.0-arm64-cpython-311/editdistance/_editdistance.o
    clang -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /Users/manu_neu/miniforge3/envs/spaCy/include -arch arm64 -fPIC -O2 -isystem /Users/manu_neu/miniforge3/envs/spaCy/include -arch arm64 -I./editdistance -I/Users/manu_neu/miniforge3/envs/spaCy/include/python3.11 -c editdistance/bycython.cpp -o build/temp.macosx-11.0-arm64-cpython-311/editdistance/bycython.o
    editdistance/bycython.cpp:216:12: fatal error: 'longintrepr.h' file not found
      #include "longintrepr.h"
               ^~~~~~~~~~~~~~~
    1 error generated.
    error: command '/usr/bin/clang' failed with exit code 1
    [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for editdistance
  Running setup.py clean for editdistance
  Failed to build editdistance
  ERROR: Could not build wheels for editdistance, which is required to install pyproject.toml-based projects

[R1j1t]

Thanks for reporting this issue. It looks like a dependency editdistance installation is failing and it is a known issue. I can try the new version to check if it fixes it?

Let me know if you wanna contribute this change? We need to change below to 0.6.2! https://github.com/R1j1t/contextualSpellCheck/blob/60e48564e9f3da1b3dd1cf2974b6e0c1be07abc6/setup.py#L34

[man00ka]

I tried 0.6.2 and everything works fine.

I forked your repo and changed the install requirements. How should I contribute this the proper way?

Am 25.07.2023 um 02:19 schrieb Rajat @.***>:

Thanks for reporting this issue. It looks like a dependency editdistance installation is failing and it is a known issue https://github.com/roy-ht/editdistance/issues/94. I can try the new version to check if it fixes it?

Let me know if you wanna contribute this change? We need to change below to 0.6.2! https://github.com/R1j1t/contextualSpellCheck/blob/60e48564e9f3da1b3dd1cf2974b6e0c1be07abc6/setup.py#L34 https://github.com/R1j1t/contextualSpellCheck/blob/60e48564e9f3da1b3dd1cf2974b6e0c1be07abc6/setup.py#L34 — Reply to this email directly, view it on GitHub https://github.com/R1j1t/contextualSpellCheck/issues/89#issuecomment-1648790240, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASVQVDDFSD2XW5EHZMWUOMDXR4GKXANCNFSM6AAAAAA2NJKBHA. You are receiving this because you authored the thread.

[R1j1t]

Sounds great! Could you please create a PR so that I can test and merge!

Thanks @adkiem for checking the new dependency version!

[albydeca]

Hey! Thanks for fixing this. Any chance it can be included in a new pyPI release?

[R1j1t]

I was waiting for another PR to merge as well, but I can release a patch by tomorrow.

[albydeca]

Up to you :)

[R1j1t]

Path released! Thanks @adkiem for your contribution. Feel free to add other features, I would be glad to incorporate them.

Cheers!