search

explosion / spaCy

💫 Industrial-strength Natural Language Processing (NLP) in Python
https://spacy.io
MIT License
29.96k stars 4.39k forks source link

Linking en_sm as english model fails during pip install #1138

Closed tiberiuichim closed 7 years ago

tiberiuichim commented 7 years ago

In a new python3 virtualenv, I ran bin/pip install spacy-nightly, then bin/python -m spacy download en. The linking phase fails initially, but on the second tries it appear to succeed (but actually doesn't).

# bin/python -m spacy download en
Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0-alpha/en_core_web_sm-2.0.0-alpha.tar.gz
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0-alpha/en_core_web_sm-2.0.0-alpha.tar.gz (15.9MB)
    100% |████████████████████████████████| 15.9MB 514kB/s 
Requirement already satisfied: spacy-nightly<3.0.0,>=2.0.0a0 in ./lib/python3.5/site-packages (from en-core-web-sm==2.0.0a0)
Requirement already satisfied: thinc<6.8.0,>=6.7.3 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: msgpack-python in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: pip<10.0.0,>=9.0.0 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: regex==2017.4.5 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: msgpack-numpy in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: cymem<1.32,>=1.30 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: ujson>=1.35 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: numpy>=1.7 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: six in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: pathlib in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: preshed<2.0.0,>=1.0.0 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: dill<0.3,>=0.2 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: murmurhash<0.29,>=0.28 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: ftfy<5.0.0,>=4.4.2 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: chainer==1.24.0 in ./lib/python3.5/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in ./lib/python3.5/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: cytoolz<0.9,>=0.8 in ./lib/python3.5/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: termcolor in ./lib/python3.5/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: wrapt in ./lib/python3.5/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./lib/python3.5/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: certifi>=2017.4.17 in ./lib/python3.5/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in ./lib/python3.5/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: idna<2.6,>=2.5 in ./lib/python3.5/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: html5lib in ./lib/python3.5/site-packages (from ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: wcwidth in ./lib/python3.5/site-packages (from ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: protobuf>=2.6.0 in ./lib/python3.5/site-packages (from chainer==1.24.0->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: filelock in ./lib/python3.5/site-packages (from chainer==1.24.0->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: nose in ./lib/python3.5/site-packages (from chainer==1.24.0->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: toolz>=0.8.0 in ./lib/python3.5/site-packages (from cytoolz<0.9,>=0.8->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: setuptools>=18.5 in ./lib/python3.5/site-packages (from html5lib->ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: webencodings in ./lib/python3.5/site-packages (from html5lib->ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Installing collected packages: en-core-web-sm
  Running setup.py install for en-core-web-sm ... done
Successfully installed en-core-web-sm-2.0.0a0

    Can't locate model data
    The data should be located in en_core_web_sm

    Download successful
    Creating a shortcut link for 'en' didn't work (maybe you don't have
    admin permissions?), but you can still load the model via its full
    package name:

    nlp = spacy.load('en_core_web_sm')

# bin/python -m spacy download en                                                                                                            47.3s
Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0-alpha/en_core_web_sm-2.0.0-alpha.tar.gz
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0-alpha/en_core_web_sm-2.0.0-alpha.tar.gz (15.9MB)
    100% |████████████████████████████████| 15.9MB 518kB/s 
  Requirement already satisfied (use --upgrade to upgrade): en-core-web-sm==2.0.0a0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0-alpha/en_core_web_sm-2.0.0-alpha.tar.gz in ./lib/python3.5/site-packages
Requirement already satisfied: spacy-nightly<3.0.0,>=2.0.0a0 in ./lib/python3.5/site-packages (from en-core-web-sm==2.0.0a0)
Requirement already satisfied: murmurhash<0.29,>=0.28 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: pip<10.0.0,>=9.0.0 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: regex==2017.4.5 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: cymem<1.32,>=1.30 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: numpy>=1.7 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: six in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: ftfy<5.0.0,>=4.4.2 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: thinc<6.8.0,>=6.7.3 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: ujson>=1.35 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: preshed<2.0.0,>=1.0.0 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: msgpack-numpy in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: msgpack-python in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: dill<0.3,>=0.2 in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: pathlib in ./lib/python3.5/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in ./lib/python3.5/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: idna<2.6,>=2.5 in ./lib/python3.5/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: certifi>=2017.4.17 in ./lib/python3.5/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./lib/python3.5/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: wcwidth in ./lib/python3.5/site-packages (from ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: html5lib in ./lib/python3.5/site-packages (from ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: cytoolz<0.9,>=0.8 in ./lib/python3.5/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: wrapt in ./lib/python3.5/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: termcolor in ./lib/python3.5/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: chainer==1.24.0 in ./lib/python3.5/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in ./lib/python3.5/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: webencodings in ./lib/python3.5/site-packages (from html5lib->ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: setuptools>=18.5 in ./lib/python3.5/site-packages (from html5lib->ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: toolz>=0.8.0 in ./lib/python3.5/site-packages (from cytoolz<0.9,>=0.8->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: nose in ./lib/python3.5/site-packages (from chainer==1.24.0->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: protobuf>=2.6.0 in ./lib/python3.5/site-packages (from chainer==1.24.0->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: filelock in ./lib/python3.5/site-packages (from chainer==1.24.0->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)

    Linking successful
    /home/tibi/work/nlpwork/lib/python3.5/site-packages/en_core_web_sm -->
    /home/tibi/work/nlpwork/lib/python3.5/site-packages/spacy/data/en

    You can now load the model via spacy.load('en')

And now, in the python interpretor:

>>> nlp = spacy.load('en')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/tibi/work/nlpwork/lib/python3.5/site-packages/spacy/__init__.py", line 13, in load
    return util.load_model(name, **overrides)
  File "/home/tibi/work/nlpwork/lib/python3.5/site-packages/spacy/util.py", line 103, in load_model
    return load_model_from_link(name, **overrides)
  File "/home/tibi/work/nlpwork/lib/python3.5/site-packages/spacy/util.py", line 116, in load_model_from_link
    spec = importlib.util.spec_from_file_location(name, init_file)
  File "<frozen importlib._bootstrap_external>", line 549, in spec_from_file_location
AttributeError: 'PosixPath' object has no attribute 'endswith'
>>> nlp = spacy.load('en_core_web_sm')
>>>

Info about spaCy

ines commented 7 years ago

Thanks for the detailed report – this is interesting and likely a bug!

Just out of curiosity, what's in your site-packages/spacy/data/ directory now? Which symlinks did spaCy create, and what do they point to?

tiberiuichim commented 7 years ago

After the first try to install the language model, this is the contents:

bin/python -m spacy download en
 I  ~/work/spacy-work/lib/python3.6/site-packages/spacy/data  ls -la
total 12
drwxr-xr-x  3 tibi users 4096 Jun 23 11:27 ./
drwxr-xr-x 10 tibi users 4096 Jun 23 11:27 ../
-rw-r--r--  1 tibi users    0 Jun 23 11:26 __init__.py
drwxr-xr-x  2 tibi users 4096 Jun 23 11:27 __pycache__/

After the second run, there's now a link:

 I  ~/work/spacy-work/lib/python3.6/site-packages/spacy/data  ls -la
total 16
drwxr-xr-x  3 tibi users 4096 Jun 23 11:32 ./
drwxr-xr-x 10 tibi users 4096 Jun 23 11:27 ../
lrwxrwxrwx  1 tibi users   69 Jun 23 11:32 en -> /home/tibi/work/spacy-work/lib/python3.6/site-packages/en_core_web_sm/
-rw-r--r--  1 tibi users    0 Jun 23 11:26 __init__.py
drwxr-xr-x  2 tibi users 4096 Jun 23 11:27 __pycache__/
tibi@tibi-desktop:/home/tibi/work/spacy-work/lib/python3.6/site-packages/spacy/data$
tiberiuichim commented 7 years ago

On this computer that I have tried to reproduce the bug, after the second time the model is download, the model can be loaded succesfully with spacy.load('en').

oximer commented 7 years ago

I having the same issue. @tiberiuichim even that I got the message that is install corretly on the second time. I still can execute spacy.load('en'). Only spacy.load('en_core_web_sm') works.

 (.env) Urbano-MacBook-Pro-2:spaCy urbano$ python -m spacy download en
    Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0-alpha/en_core_web_sm-2.0.0-alpha.tar.gz
     Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0-alpha/en_core_web_sm-2.0.0-alpha.tar.gz (15.9MB)
    100% |████████████████████████████████| 15.9MB 16.8MB/s
Requirement already satisfied: spacy-nightly<3.0.0,>=2.0.0a0 in /Users/urbano/Documents/fontes/nlp/spaCy (from en-core-web-sm==2.0.0a0)
Requirement already satisfied: numpy>=1.7 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: murmurhash<0.29,>=0.28 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: cymem<1.32,>=1.30 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: preshed<2.0.0,>=1.0.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: thinc<6.8.0,>=6.7.3 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: pip<10.0.0,>=9.0.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: six in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: pathlib in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: ujson>=1.35 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: dill<0.3,>=0.2 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: regex==2017.4.5 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: ftfy<5.0.0,>=4.4.2 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: msgpack-python in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: msgpack-numpy in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: wrapt in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: chainer==1.24.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: termcolor in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: cytoolz<0.9,>=0.8 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: certifi>=2017.4.17 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: idna<2.6,>=2.5 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: html5lib in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: wcwidth in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: nose in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from chainer==1.24.0->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: filelock in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from chainer==1.24.0->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: protobuf>=2.6.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from chainer==1.24.0->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: toolz>=0.8.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from cytoolz<0.9,>=0.8->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: setuptools>=18.5 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from html5lib->ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: webencodings in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from html5lib->ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Installing collected packages: en-core-web-sm
  Running setup.py install for en-core-web-sm ... done
    Successfully installed en-core-web-sm-2.0.0a0

    Can't locate model data
    The data should be located in en_core_web_sm

    Download successful
    Creating a shortcut link for 'en' didn't work (maybe you don't have
    admin permissions?), but you can still load the model via its full
    package name:

    nlp = spacy.load('en_core_web_sm')
(.env) Urbano-MacBook-Pro-2:spaCy urbano$ python -m spacy download en
Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0-alpha/en_core_web_sm-2.0.0-alpha.tar.gz
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0-alpha/en_core_web_sm-2.0.0-alpha.tar.gz (15.9MB)
    100% |████████████████████████████████| 15.9MB 5.5MB/s
  Requirement already satisfied (use --upgrade to upgrade): en-core-web-sm==2.0.0a0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0-alpha/en_core_web_sm-2.0.0-alpha.tar.gz in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages
Requirement already satisfied: spacy-nightly<3.0.0,>=2.0.0a0 in /Users/urbano/Documents/fontes/nlp/spaCy (from en-core-web-sm==2.0.0a0)
Requirement already satisfied: numpy>=1.7 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: murmurhash<0.29,>=0.28 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: cymem<1.32,>=1.30 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: preshed<2.0.0,>=1.0.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: thinc<6.8.0,>=6.7.3 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: pip<10.0.0,>=9.0.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: six in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: pathlib in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: ujson>=1.35 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: dill<0.3,>=0.2 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: regex==2017.4.5 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: ftfy<5.0.0,>=4.4.2 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: msgpack-python in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: msgpack-numpy in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: wrapt in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: chainer==1.24.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: termcolor in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: cytoolz<0.9,>=0.8 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: certifi>=2017.4.17 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: idna<2.6,>=2.5 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: html5lib in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: wcwidth in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: nose in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from chainer==1.24.0->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: filelock in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from chainer==1.24.0->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
    Requirement already satisfied: protobuf>=2.6.0 in /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from chainer==1.24.0->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
    Requirement already satisfied: toolz>=0.8.0 in     /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from cytoolz<0.9,>=0.8->thinc<6.8.0,>=6.7.3->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
    Requirement already satisfied: setuptools>=18.5 in                     /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from html5lib->ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)
Requirement already satisfied: webencodings in 
    /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages (from html5lib->ftfy<5.0.0,>=4.4.2->spacy-nightly<3.0.0,>=2.0.0a0->en-core-web-sm==2.0.0a0)

    Linking successful
    /Users/urbano/Documents/fontes/nlp/.env/lib/python2.7/site-packages/en_core_web_sm
    --> spacy/data/en

    You can now load the model via spacy.load('en')

(.env) Urbano-MacBook-Pro-2:spaCy urbano$ python examples/mysample/english.py
Traceback (most recent call last):
  File "examples/mysample/english.py", line 2, in <module>
    en_nlp = spacy.load('en')
  File "/Users/urbano/Documents/fontes/nlp/spaCy/spacy/__init__.py", line 13, in load
    return util.load_model(name, **overrides)
  File "/Users/urbano/Documents/fontes/nlp/spaCy/spacy/util.py", line 103, in load_model
    return load_model_from_link(name, **overrides)
  File "/Users/urbano/Documents/fontes/nlp/spaCy/spacy/util.py", line 116, in load_model_from_link
    spec = importlib.util.spec_from_file_location(name, init_file)
AttributeError: 'module' object has no attribute 'util'
ines commented 7 years ago

Fixed and will be included in the next alpha release.

It looks like the problem was that the link command checks if a model name maps to an installed package by calling pip.get_installed_distributions(). If the package (i.e. model) was just installed via a subprocess in download, it's not yet included in the list. So instead, the download command now calls util.get_package_path() explicitly and passes the model path to link.

ines commented 7 years ago

Fixed in spacy-nightly-2.0.0a9!

bittlingmayer commented 7 years ago

I'm still getting this with pip install spacy-nightly==2.0.0a09.

(Rolled back to it to try to work around #1315.)

kevin91nl commented 6 years ago

I still encounter this issue with pip install spacy==2.0.9 (on Windows 10).

zhenzuo2 commented 6 years ago

In windows, you need to run Anaconda Prompt with administrator.

ptytb commented 6 years ago

@zhenzuo2 @kevin91nl

Instructions for Windows users:

mklink /j  %LOCALAPPDATA%\Programs\Python\Python36\lib\site-packages\spacy\data\en %LOCALAPPDATA%\Programs\Python\Python36\lib\site-packages\en_core_web_sm

No need for being Admin, no need for Anaconda.

kevin91nl commented 6 years ago

Thank you @ptytb, after pluging in the paths as pointed out by the python -m spacy download en command, it worked!

monajalal commented 6 years ago

I still have the issue:

[jalal@goku sentiment]$ sudo pip install spacy && python -m spacy download en
[sudo] password for jalal: 
Requirement already satisfied: spacy in /usr/lib64/python2.7/site-packages (2.0.9)
Requirement already satisfied: six in /usr/lib/python2.7/site-packages (from spacy) (1.11.0)
Requirement already satisfied: regex==2017.4.5 in /usr/lib64/python2.7/site-packages (from spacy) (2017.4.5)
Requirement already satisfied: pathlib in /usr/lib/python2.7/site-packages (from spacy) (1.0.1)
Requirement already satisfied: preshed<2.0.0,>=1.0.0 in /usr/lib64/python2.7/site-packages (from spacy) (1.0.0)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in /usr/lib/python2.7/site-packages (from spacy) (0.9.6)
Requirement already satisfied: numpy>=1.7 in /usr/lib64/python2.7/site-packages (from spacy) (1.13.3)
Requirement already satisfied: ftfy<5.0.0,>=4.4.2 in /usr/lib/python2.7/site-packages (from spacy) (4.4.3)
Requirement already satisfied: msgpack-python==0.5.4 in /usr/lib64/python2.7/site-packages (from spacy) (0.5.4)
Requirement already satisfied: cymem<1.32,>=1.30 in /usr/lib64/python2.7/site-packages (from spacy) (1.31.2)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/lib/python2.7/site-packages (from spacy) (2.18.4)
Requirement already satisfied: ujson>=1.35 in /usr/lib64/python2.7/site-packages (from spacy) (1.35)
Requirement already satisfied: msgpack-numpy==0.4.1 in /usr/lib/python2.7/site-packages (from spacy) (0.4.1)
Requirement already satisfied: html5lib==1.0b8 in /usr/lib/python2.7/site-packages (from spacy) (1.0b8)
Requirement already satisfied: murmurhash<0.29,>=0.28 in /usr/lib64/python2.7/site-packages (from spacy) (0.28.0)
Requirement already satisfied: thinc<6.11.0,>=6.10.1 in /usr/lib64/python2.7/site-packages (from spacy) (6.10.2)
Requirement already satisfied: dill<0.3,>=0.2 in /usr/lib/python2.7/site-packages (from spacy) (0.2.7.1)
Requirement already satisfied: wcwidth in /usr/lib/python2.7/site-packages (from ftfy<5.0.0,>=4.4.2->spacy) (0.1.7)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy) (1.22)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2018.1.18)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /usr/lib/python2.7/site-packages (from thinc<6.11.0,>=6.10.1->spacy) (4.19.6)
Requirement already satisfied: wrapt in /usr/lib64/python2.7/site-packages (from thinc<6.11.0,>=6.10.1->spacy) (1.10.11)
Requirement already satisfied: cytoolz<0.9,>=0.8 in /usr/lib64/python2.7/site-packages (from thinc<6.11.0,>=6.10.1->spacy) (0.8.2)
Requirement already satisfied: termcolor in /usr/lib/python2.7/site-packages (from thinc<6.11.0,>=6.10.1->spacy) (1.1.0)
Requirement already satisfied: toolz>=0.8.0 in /usr/lib/python2.7/site-packages (from cytoolz<0.9,>=0.8->thinc<6.11.0,>=6.10.1->spacy) (0.9.0)
rtslib-fb 2.1.63 has requirement pyudev>=0.16.1, but you'll have pyudev 0.15 which is incompatible.
matplotlib 2.2.2 has requirement python-dateutil>=2.1, but you'll have python-dateutil 1.5 which is incompatible.
bleach 2.1.3 has requirement html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,>=0.99999999pre, but you'll have html5lib 1.0b8 which is incompatible.
Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz (37.4MB)
    100% |████████████████████████████████| 37.4MB 11.5MB/s 
Requirement already satisfied (use --upgrade to upgrade): en-core-web-sm==2.0.0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz in /scratch/sjn/anaconda/lib/python3.6/site-packages

    Error: Couldn't link model to 'en'
    Creating a symlink in spacy/data failed. Make sure you have the required
    permissions and try re-running the command as admin, or use a
    virtualenv. You can still import the model as a module and call its
    load() method, or create the symlink manually.

    /scratch/sjn/anaconda/lib/python3.6/site-packages/en_core_web_sm -->
    /scratch/sjn/anaconda/lib/python3.6/site-packages/spacy/data/en

    Download successful but linking failed
    Creating a shortcut link for 'en' didn't work (maybe you don't have
    admin permissions?), but you can still load the model via its full
    package name:

    nlp = spacy.load('en_core_web_sm')

[jalal@goku sentiment]$ 
[jalal@goku sentiment]$ which python
/scratch/sjn/anaconda/bin/python
[jalal@goku sentiment]$ python -V
Python 3.6.3 :: Anaconda custom (64-bit)
[jalal@goku sentiment]$ lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 7.5.1804 (Core) 
Release:    7.5.1804
Codename:   Core
lock[bot] commented 6 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.