change spacy.load in data processor to spacy_en = spacy.load('en_core_web_sm')

[monajalal]

I got error from spacy loading en so I changed line 7 in data processor to

#spacy_en = spacy.load('en')
spacy_en = spacy.load('en_core_web_sm')

[jalal@goku document-e2e-sent]$ sudo pip install spacy && python -m spacy download en [sudo] password for jalal: Requirement already satisfied: spacy in /usr/lib64/python2.7/site-packages (2.0.9) Requirement already satisfied: preshed<2.0.0,>=1.0.0 in /usr/lib64/python2.7/site-packages (from spacy) (1.0.0) Requirement already satisfied: pathlib in /usr/lib/python2.7/site-packages (from spacy) (1.0.1) Requirement already satisfied: dill<0.3,>=0.2 in /usr/lib/python2.7/site-packages (from spacy) (0.2.7.1) Requirement already satisfied: ftfy<5.0.0,>=4.4.2 in /usr/lib/python2.7/site-packages (from spacy) (4.4.3) Requirement already satisfied: msgpack-numpy==0.4.1 in /usr/lib/python2.7/site-packages (from spacy) (0.4.1) Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/lib/python2.7/site-packages (from spacy) (2.18.4) Requirement already satisfied: numpy>=1.7 in /usr/lib64/python2.7/site-packages (from spacy) (1.13.3) Requirement already satisfied: murmurhash<0.29,>=0.28 in /usr/lib64/python2.7/site-packages (from spacy) (0.28.0) Requirement already satisfied: msgpack-python==0.5.4 in /usr/lib64/python2.7/site-packages (from spacy) (0.5.4) Requirement already satisfied: plac<1.0.0,>=0.9.6 in /usr/lib/python2.7/site-packages (from spacy) (0.9.6) Requirement already satisfied: ujson>=1.35 in /usr/lib64/python2.7/site-packages (from spacy) (1.35) Requirement already satisfied: regex==2017.4.5 in /usr/lib64/python2.7/site-packages (from spacy) (2017.4.5) Requirement already satisfied: six in /usr/lib/python2.7/site-packages (from spacy) (1.11.0) Requirement already satisfied: cymem<1.32,>=1.30 in /usr/lib64/python2.7/site-packages (from spacy) (1.31.2) Requirement already satisfied: thinc<6.11.0,>=6.10.1 in /usr/lib64/python2.7/site-packages (from spacy) (6.10.2) Collecting html5lib==1.0b8 (from spacy) Downloading https://files.pythonhosted.org/packages/3b/89/c32486cdca98f476252a9db2cc836b605de4807ac4602ca55b3e7b52b7a8/html5lib-1.0b8.tar.gz (889kB) 100% |████████████████████████████████| 890kB 10.5MB/s Requirement already satisfied: wcwidth in /usr/lib/python2.7/site-packages (from ftfy<5.0.0,>=4.4.2->spacy) (0.1.7) Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2018.1.18) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy) (3.0.4) Requirement already satisfied: idna<2.7,>=2.5 in /usr/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2.6) Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/lib/python2.7/site-packages (from requests<3.0.0,>=2.13.0->spacy) (1.22) Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /usr/lib/python2.7/site-packages (from thinc<6.11.0,>=6.10.1->spacy) (4.19.6) Requirement already satisfied: cytoolz<0.9,>=0.8 in /usr/lib64/python2.7/site-packages (from thinc<6.11.0,>=6.10.1->spacy) (0.8.2) Requirement already satisfied: wrapt in /usr/lib64/python2.7/site-packages (from thinc<6.11.0,>=6.10.1->spacy) (1.10.11) Requirement already satisfied: termcolor in /usr/lib/python2.7/site-packages (from thinc<6.11.0,>=6.10.1->spacy) (1.1.0) Requirement already satisfied: toolz>=0.8.0 in /usr/lib/python2.7/site-packages (from cytoolz<0.9,>=0.8->thinc<6.11.0,>=6.10.1->spacy) (0.9.0) Building wheels for collected packages: html5lib Running setup.py bdist_wheel for html5lib ... done Stored in directory: /root/.cache/pip/wheels/73/99/09/aa182bba1f0ac27cef2f0555a0d359b173657cbe1da6a41046 Successfully built html5lib rtslib-fb 2.1.63 has requirement pyudev>=0.16.1, but you'll have pyudev 0.15 which is incompatible. matplotlib 2.2.2 has requirement python-dateutil>=2.1, but you'll have python-dateutil 1.5 which is incompatible. bleach 2.1.3 has requirement html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,>=0.99999999pre, but you'll have html5lib 1.0b8 which is incompatible. Installing collected packages: html5lib Found existing installation: html5lib 1.0.1 Uninstalling html5lib-1.0.1: Successfully uninstalled html5lib-1.0.1 Successfully installed html5lib-1.0b8 Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz (37.4MB) 100% |████████████████████████████████| 37.4MB 11.3MB/s Requirement already satisfied (use --upgrade to upgrade): en-core-web-sm==2.0.0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz in /scratch/sjn-p2/anaconda/anaconda2/lib/python2.7/site-packages

Error: Couldn't link model to 'en'
Creating a symlink in spacy/data failed. Make sure you have the required
permissions and try re-running the command as admin, or use a
virtualenv. You can still import the model as a module and call its
load() method, or create the symlink manually.

/scratch/sjn-p2/anaconda/anaconda2/lib/python2.7/site-packages/en_core_web_sm
-->
/scratch/sjn-p2/anaconda/anaconda2/lib/python2.7/site-packages/spacy/data/en

Download successful but linking failed
Creating a shortcut link for 'en' didn't work (maybe you don't have
admin permissions?), but you can still load the model via its full
package name:

nlp = spacy.load('en_core_web_sm')

[jalal@goku document-e2e-sent]$ vi data_processor.py [jalal@goku document-e2e-sent]$ python data_processor.py