search

pypa / pipenv

Python Development Workflow for Humans.
https://pipenv.pypa.io
MIT License
24.83k stars 1.87k forks source link

failure to install spacy pipelines #5612

Closed ltrojan closed 1 year ago

ltrojan commented 1 year ago

Not sure this is a bug, a problem with the spacy packaging or with me doing something wrong so creating an issue instead of a bug to get some feedback...

as per the following instructions:

https://spacy.io/usage/models#download-pip

running the install command with pip leads to the successfull installation of the pakage:

/myproject$ pip install $(spacy info en_core_web_sm --url)
Collecting en-core-web-sm==3.5.0
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.5.0/en_core_web_sm-3.5.0-py3-none-any.whl (12.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.8/12.8 MB 2.0 MB/s eta 0:00:00
Requirement already satisfied: spacy<3.6.0,>=3.5.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from en-core-web-sm==3.5.0) (3.5.0)
Requirement already satisfied: jinja2 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (3.1.2)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (1.0.4)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (2.28.2)
Requirement already satisfied: packaging>=20.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (23.0)
Requirement already satisfied: typer<0.8.0,>=0.3.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (0.7.0)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (3.0.8)
Requirement already satisfied: thinc<8.2.0,>=8.1.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (8.1.7)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (2.0.8)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (3.3.0)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (1.0.9)
Requirement already satisfied: numpy>=1.15.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (1.24.2)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (2.4.5)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (6.3.0)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (4.64.1)
Requirement already satisfied: setuptools in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (67.3.1)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (1.1.1)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.11.0,>=1.7.4 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (1.10.4)
Requirement already satisfied: pathy>=0.10.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (0.10.1)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (3.0.12)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (2.0.7)
Requirement already satisfied: typing-extensions>=4.2.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from pydantic!=1.8,!=1.8.1,<1.11.0,>=1.7.4->spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (4.4.0)
Requirement already satisfied: idna<4,>=2.5 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (3.4)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (3.0.1)
Requirement already satisfied: certifi>=2017.4.17 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (2022.12.7)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (1.26.14)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from thinc<8.2.0,>=8.1.0->spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (0.7.9)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from thinc<8.2.0,>=8.1.0->spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (0.0.4)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from typer<0.8.0,>=0.3.0->spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (8.1.3)
Requirement already satisfied: MarkupSafe>=2.0 in /home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages (from jinja2->spacy<3.6.0,>=3.5.0->en-core-web-sm==3.5.0) (2.1.2)

when running the same with pipenv, the install fails with the following traceback:

myproject/$ pipenv install $(spacy info en_core_web_sm --url)
Installing https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.5.0/en_core_web_sm-3.5.0-py3-none-any.whl...
✔ Installation Succeeded
Pipfile.lock (a2172f) out of date, updating to (f082bd)...
Locking [dev-packages] dependencies...
Building requirements...
✘ Locking Failed!

Traceback (most recent call last):
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3038, in _dep_map
    return self.__dep_map
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2834, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 35, in __init__
    parsed = parse_requirement(requirement_string)
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 64, in parse_requirement
    return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 82, in _parse_requirement
    url, specifier, marker = _parse_requirement_details(tokenizer)
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 120, in _parse_requirement_details
    specifier = _parse_specifier(tokenizer)
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 209, in _parse_specifier
    tokenizer.consume("WS")
  File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_tokenizer.py", line 183, in enclosing_tokens
    self.raise_syntax_error(
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_tokenizer.py", line 163, in raise_syntax_error
    raise ParserSyntaxError(
pkg_resources.extern.packaging._tokenizer.ParserSyntaxError: Expected closing RIGHT_PARENTHESIS
    extract-msg (<=0.29.*)
                ~~~~~~~^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/home/ltrojan/.local/lib/python3.8/site-packages/pipenv/resolver.py", line 857, in <module>
    main()
  File "/home/ltrojan/.local/lib/python3.8/site-packages/pipenv/resolver.py", line 843, in main
    _main(
  File "/home/ltrojan/.local/lib/python3.8/site-packages/pipenv/resolver.py", line 822, in _main
    resolve_packages(
  File "/home/ltrojan/.local/lib/python3.8/site-packages/pipenv/resolver.py", line 783, in resolve_packages
    results = clean_results(results, resolver, project, dev)
  File "/home/ltrojan/.local/lib/python3.8/site-packages/pipenv/resolver.py", line 644, in clean_results
    reverse_deps = project.environment.reverse_dependencies()
  File "/home/ltrojan/.local/lib/python3.8/site-packages/pipenv/environment.py", line 785, in reverse_dependencies
    for req in self.get_package_requirements():
  File "/home/ltrojan/.local/lib/python3.8/site-packages/pipenv/environment.py", line 757, in get_package_requirements
    tree = PackageDAG.from_pkgs(packages).sort()
  File "/home/ltrojan/.local/lib/python3.8/site-packages/pipenv/vendor/pipdeptree.py", line 293, in from_pkgs
    m = {p: [ReqPackage(r, idx.get(r.key))
  File "/home/ltrojan/.local/lib/python3.8/site-packages/pipenv/vendor/pipdeptree.py", line 294, in <dictcomp>
    for r in p.requires()]
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2754, in requires
    dm = self._dep_map
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3040, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3050, in _compute_dependencies
    reqs.extend(parse_requirements(req))
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3103, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/home/ltrojan/.local/share/virtualenvs/document-processor-mqluLyq5/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 37, in __init__
    raise InvalidRequirement(str(e)) from e
pkg_resources.extern.packaging.requirements.InvalidRequirement: Expected closing RIGHT_PARENTHESIS
    extract-msg (<=0.29.*)
                ~~~~~~~^
matteius commented 1 year ago

@ltrojan I am not sure -- what is the resulting value of $(spacy info en_core_web_sm --url) and also what is pipenv --version??

ltrojan commented 1 year ago

thanks for the resonse @matteius

reagarding your firs Q, the command is simply outputting the latest version of the language model wheel (I'm not sure why spacy does things this way...):

$ # on my system
$ spacy info en_core_web_sm --url
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.5.0/en_core_web_sm-3.5.0-py3-none-any.whl

as for pipenv version:

$ pipenv --version
pipenv, version 2022.5.2
$ pipenv run python --version
Python 3.8.10

however, I just realized that the same error happens when installing just spacy by itself: 

$ pipenv install spacy
(...)
$ pipenv graph
Error running command:  $ /home/ltrojan/repo/emulator/services/document-processor/.venv/bin/python /home/ltrojan/.loca
l/lib/python3.8/site-packages/pipenv/vendor/pipdeptree.py -l
STDERR:  Traceback (most recent call last):

(... traceback)

  File "/home/ltrojan/repo/emulator/services/document-processor/.venv/lib/python3.8/site-packages/pkg_resources/_vendo
r/packaging/requirements.py", line 37, in __init__
    raise InvalidRequirement(str(e)) from e
pkg_resources.extern.packaging.requirements.InvalidRequirement: Expected end or semicolon (after version specifier)
    extract-msg<=0.29.*
matteius commented 1 year ago

@ltrojan To further debug, could you upgrade past 2022.5.2? -- that is kind of old at this point.

ltrojan commented 1 year ago

@matteius that worked!

once pipenv is upgraded to the 2023.02.18 the traceback disappears,

thanks you so much ;)

do I close the issue?

alexrudd2 commented 1 year ago

Also encountered on 2022.9.8. Upgrading to 2023.2.18 fixed.