search

explosion / spaCy

💫 Industrial-strength Natural Language Processing (NLP) in Python
https://spacy.io
MIT License
29.7k stars 4.36k forks source link

Spacy model linking fails. #2873

Closed aanchan closed 5 years ago

aanchan commented 5 years ago

Summary

This is a bug during install. It is related to issue 1138. The issue does not seem to be resolved. The model is included in the requirements.txt, but linking fails. The linking only happens when trying to manually do a python -m spacy download en. Why is this not good? My CI/CD pipeline breaks.

Output during pip install:

pip install -r requirements.txt Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz#en_core_web_sm (from -r requirements.txt (line 88)) Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz (37.4MB) 100% |████████████████████████████████| 37.4MB 1.4MB/s Collecting absl-py==0.5.0 (from -r requirements.txt (line 1)) Collecting APScheduler==3.5.3 (from -r requirements.txt (line 2)) Using cached https://files.pythonhosted.org/packages/97/3a/fa3213cc325091b7729616594611fff31d72c2d4d590418c3efdf7424ae2/APScheduler-3.5.3-py2.py3-none-any.whl Collecting astor==0.7.1 (from -r requirements.txt (line 3)) Using cached https://files.pythonhosted.org/packages/35/6b/11530768cac581a12952a2aad00e1526b89d242d0b9f59534ef6e6a1752f/astor-0.7.1-py2.py3-none-any.whl Collecting attrs==18.2.0 (from -r requirements.txt (line 4)) Using cached https://files.pythonhosted.org/packages/3a/e1/5f9023cc983f1a628a8c2fd051ad19e76ff7b142a0faf329336f9a62a514/attrs-18.2.0-py2.py3-none-any.whl Collecting Automat==0.7.0 (from -r requirements.txt (line 5)) Using cached https://files.pythonhosted.org/packages/a3/86/14c16bb98a5a3542ed8fed5d74fb064a902de3bdd98d6584b34553353c45/Automat-0.7.0-py2.py3-none-any.whl Collecting bleach==1.5.0 (from -r requirements.txt (line 6)) Using cached https://files.pythonhosted.org/packages/33/70/86c5fec937ea4964184d4d6c4f0b9551564f821e1c3575907639036d9b90/bleach-1.5.0-py2.py3-none-any.whl Collecting boto3==1.9.8 (from -r requirements.txt (line 7)) Using cached https://files.pythonhosted.org/packages/38/aa/c100449bc8550c52b3f14e3b3831c2b5db6a4b7b752b7a11e76599508fc4/boto3-1.9.8-py2.py3-none-any.whl Collecting botocore==1.12.16 (from -r requirements.txt (line 8)) Downloading https://files.pythonhosted.org/packages/87/28/f24899d31a4c0da5e9c0c70a481503633f7b1e94c0671398546bc2241400/botocore-1.12.16-py2.py3-none-any.whl (4.7MB) 100% |████████████████████████████████| 4.7MB 7.0MB/s Collecting certifi==2018.8.24 (from -r requirements.txt (line 9)) Using cached https://files.pythonhosted.org/packages/df/f7/04fee6ac349e915b82171f8e23cee63644d83663b34c539f7a09aed18f9e/certifi-2018.8.24-py2.py3-none-any.whl Collecting chardet==3.0.4 (from -r requirements.txt (line 10)) Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl Collecting click==6.7 (from -r requirements.txt (line 11)) Using cached https://files.pythonhosted.org/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl Collecting cloudpickle==0.5.6 (from -r requirements.txt (line 12)) Using cached https://files.pythonhosted.org/packages/98/d6/a78a4589234cc6f47f29665c1225f30467db5fdaf4ca1fb52b0685bff108/cloudpickle-0.5.6-py2.py3-none-any.whl Collecting coloredlogs==10.0 (from -r requirements.txt (line 13)) Using cached https://files.pythonhosted.org/packages/08/0f/7877fc42fff0b9d70b6442df62d53b3868d3a6ad1b876bdb54335b30ff23/coloredlogs-10.0-py2.py3-none-any.whl Collecting colorhash==1.0.2 (from -r requirements.txt (line 14)) Using cached https://files.pythonhosted.org/packages/0e/e1/50dbc513aa74e99eca4c47f2a8206711f0bec436fdddd95eebaf7eaaa1aa/colorhash-1.0.2-py2.py3-none-any.whl Collecting ConfigArgParse==0.13.0 (from -r requirements.txt (line 15)) Collecting constantly==15.1.0 (from -r requirements.txt (line 16)) Using cached https://files.pythonhosted.org/packages/b9/65/48c1909d0c0aeae6c10213340ce682db01b48ea900a7d9fce7a7910ff318/constantly-15.1.0-py2.py3-none-any.whl Collecting cycler==0.10.0 (from -r requirements.txt (line 17)) Using cached https://files.pythonhosted.org/packages/f7/d2/e07d3ebb2bd7af696440ce7e754c59dd546ffe1bbe732c8ab68b9c834e61/cycler-0.10.0-py2.py3-none-any.whl Collecting cymem<2.1.0,>=2.0.2 (from -r requirements.txt (line 18)) Downloading https://files.pythonhosted.org/packages/6b/d5/c1583c90023608e71ee35b6943d2a5dc488d463b84ecd1c0fddbf23eed44/cymem-2.0.2-cp35-cp35m-manylinux1_x86_64.whl Collecting cytoolz==0.9.0.1 (from -r requirements.txt (line 19)) Collecting decorator==4.3.0 (from -r requirements.txt (line 20)) Using cached https://files.pythonhosted.org/packages/bc/bb/a24838832ba35baf52f32ab1a49b906b5f82fb7c76b2f6a7e35e140bac30/decorator-4.3.0-py2.py3-none-any.whl Collecting dill==0.2.8.2 (from -r requirements.txt (line 21)) Collecting docopt==0.6.2 (from -r requirements.txt (line 22)) Collecting docutils==0.14 (from -r requirements.txt (line 23)) Using cached https://files.pythonhosted.org/packages/36/fa/08e9e6e0e3cbd1d362c3bbee8d01d0aedb2155c4ac112b19ef3cae8eed8d/docutils-0.14-py3-none-any.whl Collecting fakeredis==0.10.3 (from -r requirements.txt (line 24)) Using cached https://files.pythonhosted.org/packages/64/bd/2756ddf350c4bb308e3255f9dcd6610f8b01344947bf74d5d166dc66b0a2/fakeredis-0.10.3-py2.py3-none-any.whl Collecting fbmessenger==5.3.0 (from -r requirements.txt (line 25)) Using cached https://files.pythonhosted.org/packages/05/37/c99c7148b311ad69f82ef5bde07f605568dcf1a78688914cadac53ba23d0/fbmessenger-5.3.0-py2.py3-none-any.whl Collecting Flask==1.0.2 (from -r requirements.txt (line 26)) Using cached https://files.pythonhosted.org/packages/7f/e7/08578774ed4536d3242b14dacb4696386634607af824ea997202cd0edb4b/Flask-1.0.2-py2.py3-none-any.whl Collecting Flask-Cors==3.0.6 (from -r requirements.txt (line 27)) Using cached https://files.pythonhosted.org/packages/d1/db/f3495569d5c3e2bdb9fb8a66c54503364abb6f35a9da2227cf5c9c50dc42/Flask_Cors-3.0.6-py2.py3-none-any.whl Collecting future==0.16.0 (from -r requirements.txt (line 28)) Collecting gast==0.2.0 (from -r requirements.txt (line 29)) Collecting gevent==1.3.6 (from -r requirements.txt (line 30)) Using cached https://files.pythonhosted.org/packages/a9/33/1fe2c8b3f9b6d0b667e42c8902beb6d944149e0330f564901d50fec76e41/gevent-1.3.6-cp35-cp35m-manylinux1_x86_64.whl Collecting graphviz==0.8.4 (from -r requirements.txt (line 31)) Using cached https://files.pythonhosted.org/packages/53/39/4ab213673844e0c004bed8a0781a0721a3f6bb23eb8854ee75c236428892/graphviz-0.8.4-py2.py3-none-any.whl Collecting greenlet==0.4.15 (from -r requirements.txt (line 32)) Using cached https://files.pythonhosted.org/packages/2e/65/27f35497cc0102a792390d056e793e064da95fc9eae45d75ae0ba49c0a0d/greenlet-0.4.15-cp35-cp35m-manylinux1_x86_64.whl Collecting grpcio==1.15.0 (from -r requirements.txt (line 33)) Using cached https://files.pythonhosted.org/packages/2d/2d/7d9cfd7b3fe8cb0281ea204f5f721749a4bfb048654f389887b82a295588/grpcio-1.15.0-cp35-cp35m-manylinux1_x86_64.whl Collecting h5py==2.8.0 (from -r requirements.txt (line 34)) Using cached https://files.pythonhosted.org/packages/d9/0a/f0dd6d533d6b5bd4c1ca186af2792186885a90b84df41f3e6867466761fc/h5py-2.8.0-cp35-cp35m-manylinux1_x86_64.whl Collecting html5lib==0.9999999 (from -r requirements.txt (line 35)) Collecting humanfriendly==4.16.1 (from -r requirements.txt (line 36)) Using cached https://files.pythonhosted.org/packages/fb/34/1e80e4a06020f7dfc7eee11387e400435236f992f80eaa6ed9b3d8252211/humanfriendly-4.16.1-py2.py3-none-any.whl Collecting hyperlink==18.0.0 (from -r requirements.txt (line 37)) Using cached https://files.pythonhosted.org/packages/a7/b6/84d0c863ff81e8e7de87cff3bd8fd8f1054c227ce09af1b679a8b17a9274/hyperlink-18.0.0-py2.py3-none-any.whl Collecting idna==2.7 (from -r requirements.txt (line 38)) Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl Collecting incremental==17.5.0 (from -r requirements.txt (line 39)) Using cached https://files.pythonhosted.org/packages/f5/1d/c98a587dc06e107115cf4a58b49de20b19222c83d75335a192052af4c4b7/incremental-17.5.0-py2.py3-none-any.whl Collecting itsdangerous==0.24 (from -r requirements.txt (line 40)) Collecting Jinja2==2.10 (from -r requirements.txt (line 41)) Using cached https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl Collecting jmespath==0.9.3 (from -r requirements.txt (line 42)) Using cached https://files.pythonhosted.org/packages/b7/31/05c8d001f7f87f0f07289a5fc0fc3832e9a57f2dbd4d3b0fee70e0d51365/jmespath-0.9.3-py2.py3-none-any.whl Collecting jsonpickle==0.9.6 (from -r requirements.txt (line 43)) Collecting jsonschema==2.6.0 (from -r requirements.txt (line 44)) Using cached https://files.pythonhosted.org/packages/77/de/47e35a97b2b05c2fadbec67d44cfcdcd09b8086951b331d82de90d2912da/jsonschema-2.6.0-py2.py3-none-any.whl Collecting Keras==2.2.2 (from -r requirements.txt (line 45)) Using cached https://files.pythonhosted.org/packages/34/7d/b1dedde8af99bd82f20ed7e9697aac0597de3049b1f786aa2aac3b9bd4da/Keras-2.2.2-py2.py3-none-any.whl Collecting Keras-Applications==1.0.4 (from -r requirements.txt (line 46)) Using cached https://files.pythonhosted.org/packages/54/90/8f327deaa37a71caddb59b7b4aaa9d4b3e90c0e76f8c2d1572005278ddc5/Keras_Applications-1.0.4-py2.py3-none-any.whl Collecting Keras-Preprocessing==1.0.2 (from -r requirements.txt (line 47)) Using cached https://files.pythonhosted.org/packages/71/26/1e778ebd737032749824d5cba7dbd3b0cf9234b87ab5ec79f5f0403ca7e9/Keras_Preprocessing-1.0.2-py2.py3-none-any.whl Collecting kiwisolver==1.0.1 (from -r requirements.txt (line 48)) Using cached https://files.pythonhosted.org/packages/7e/31/d6fedd4fb2c94755cd101191e581af30e1650ccce7a35bddb7930fed6574/kiwisolver-1.0.1-cp35-cp35m-manylinux1_x86_64.whl Collecting klein==17.10.0 (from -r requirements.txt (line 49)) Using cached https://files.pythonhosted.org/packages/8a/6b/adc97a7bb3fb781fdd9e49177ad873c1479f87b9745271cbeda81cbb9cc8/klein-17.10.0-py2.py3-none-any.whl Collecting Markdown==2.6.11 (from -r requirements.txt (line 50)) Using cached https://files.pythonhosted.org/packages/6d/7d/488b90f470b96531a3f5788cf12a93332f543dbab13c423a5e7ce96a0493/Markdown-2.6.11-py2.py3-none-any.whl Collecting MarkupSafe==1.0 (from -r requirements.txt (line 51)) Collecting matplotlib==2.2.3 (from -r requirements.txt (line 52)) Using cached https://files.pythonhosted.org/packages/de/af/6258db9b26313dd7ad70dba30a60bec62bf030a44208d4cb62966206666f/matplotlib-2.2.3-cp35-cp35m-manylinux1_x86_64.whl Collecting mattermostwrapper==2.1 (from -r requirements.txt (line 53)) Using cached https://files.pythonhosted.org/packages/93/70/203660597d12788e958dd691aa11c3c29caa075eadb2ce94d2eb53099d1b/mattermostwrapper-2.1-py2.py3-none-any.whl Collecting msgpack==0.5.6 (from -r requirements.txt (line 54)) Using cached https://files.pythonhosted.org/packages/08/72/5a01d2a6a894e7f6966b0038445c748d7a16754cceb0e988699269d8152a/msgpack-0.5.6-cp35-cp35m-manylinux1_x86_64.whl Collecting msgpack-numpy==0.4.3.2 (from -r requirements.txt (line 55)) Using cached https://files.pythonhosted.org/packages/ad/45/464be6da85b5ca893cfcbd5de3b31a6710f636ccb8521b17bd4110a08d94/msgpack_numpy-0.4.3.2-py2.py3-none-any.whl Collecting murmurhash==0.28.0 (from -r requirements.txt (line 56)) Downloading https://files.pythonhosted.org/packages/50/b1/b5356c5ec4b2a66ab41bd2689830615c8b8a983c005daf52bcf7d8a9d647/murmurhash-0.28.0-cp35-cp35m-manylinux1_x86_64.whl Collecting networkx==2.2 (from -r requirements.txt (line 57)) Collecting numpy==1.15.1 (from -r requirements.txt (line 58)) Using cached https://files.pythonhosted.org/packages/0a/fa/afc1dc818589c9fd36a53f78999f2b5bd88bd5b167eb7d87fb56b565c185/numpy-1.15.1-cp35-cp35m-manylinux1_x86_64.whl Collecting packaging==17.1 (from -r requirements.txt (line 59)) Using cached https://files.pythonhosted.org/packages/ad/c2/b500ea05d5f9f361a562f089fc91f77ed3b4783e13a08a3daf82069b1224/packaging-17.1-py2.py3-none-any.whl Collecting pathlib==1.0.1 (from -r requirements.txt (line 60)) Collecting plac==0.9.6 (from -r requirements.txt (line 61)) Using cached https://files.pythonhosted.org/packages/9e/9b/62c60d2f5bc135d2aa1d8c8a86aaf84edb719a59c7f11a4316259e61a298/plac-0.9.6-py2.py3-none-any.whl Collecting preshed<2.1.0,>=2.0.1 (from -r requirements.txt (line 62)) Downloading https://files.pythonhosted.org/packages/c5/e5/705d59b08fa3e5ba33d2371f6d52c1993539d5b3beb4130c40bb4e540706/preshed-2.0.1-cp35-cp35m-manylinux1_x86_64.whl (82kB) 100% |████████████████████████████████| 92kB 11.5MB/s Collecting protobuf==3.6.1 (from -r requirements.txt (line 63)) Using cached https://files.pythonhosted.org/packages/bf/d4/db7296a1407cad69f043537ba1e05afab3646451a066ead7a314d8714388/protobuf-3.6.1-cp35-cp35m-manylinux1_x86_64.whl Collecting PyHamcrest==1.9.0 (from -r requirements.txt (line 64)) Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl Collecting PyJWT==1.6.4 (from -r requirements.txt (line 65)) Using cached https://files.pythonhosted.org/packages/93/d1/3378cc8184a6524dc92993090ee8b4c03847c567e298305d6cf86987e005/PyJWT-1.6.4-py2.py3-none-any.whl Collecting pykwalify==1.6.0 (from -r requirements.txt (line 66)) Using cached https://files.pythonhosted.org/packages/6e/59/55c32d59b462a9c3fae2ab25f179f91d61617c0215e9f8ba3f4966b2b8b1/pykwalify-1.6.0-py2.py3-none-any.whl Collecting pyparsing==2.2.1 (from -r requirements.txt (line 67)) Using cached https://files.pythonhosted.org/packages/42/47/e6d51aef3d0393f7d343592d63a73beee2a8d3d69c22b053e252c6cfacd5/pyparsing-2.2.1-py2.py3-none-any.whl Collecting PySocks==1.6.8 (from -r requirements.txt (line 68)) Collecting python-crfsuite==0.9.6 (from -r requirements.txt (line 69)) Using cached https://files.pythonhosted.org/packages/4d/2c/274b89d009bb019038feaf96abb263d7cbff069a0771841013cf1832156e/python_crfsuite-0.9.6-cp35-cp35m-manylinux1_x86_64.whl Collecting python-dateutil==2.7.3 (from -r requirements.txt (line 70)) Using cached https://files.pythonhosted.org/packages/cf/f5/af2b09c957ace60dcfac112b669c45c8c97e32f94aa8b56da4c6d1682825/python_dateutil-2.7.3-py2.py3-none-any.whl Collecting python-telegram-bot==10.1.0 (from -r requirements.txt (line 71)) Using cached https://files.pythonhosted.org/packages/f1/51/d1bd383522c12b313eddd7b97b8e7d6cd2a8e3b44b8ff3c88e4a7b045cc8/python_telegram_bot-10.1.0-py2.py3-none-any.whl Collecting pytz==2018.5 (from -r requirements.txt (line 72)) Using cached https://files.pythonhosted.org/packages/30/4e/27c34b62430286c6d59177a0842ed90dc789ce5d1ed740887653b898779a/pytz-2018.5-py2.py3-none-any.whl Collecting PyYAML==3.13 (from -r requirements.txt (line 73)) Collecting rasa-core==0.10.3 (from -r requirements.txt (line 74)) Using cached https://files.pythonhosted.org/packages/7a/8e/0066253d669183f6cee3b5a09a6d5d5c3589d7c1c238747447b60e76e3cb/rasa_core-0.10.3-py2.py3-none-any.whl Collecting rasa-nlu==0.13.4 (from -r requirements.txt (line 75)) Using cached https://files.pythonhosted.org/packages/98/10/04b19b2d29efc047a894b6ac3de7227839cab77afa2d68970a2dcb34d284/rasa_nlu-0.13.4-py2.py3-none-any.whl Collecting redis==2.10.6 (from -r requirements.txt (line 76)) Using cached https://files.pythonhosted.org/packages/3b/f6/7a76333cf0b9251ecf49efff635015171843d9b977e4ffcf59f9c4428052/redis-2.10.6-py2.py3-none-any.whl Collecting regex==2018.01.10 (from -r requirements.txt (line 77)) Using cached https://files.pythonhosted.org/packages/76/f4/7146c3812f96fcaaf2d06ff6862582302626a59011ccb6f2833bb38d80f7/regex-2018.01.10.tar.gz Collecting requests==2.19.1 (from -r requirements.txt (line 78)) Using cached https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl Collecting ruamel.yaml==0.15.69 (from -r requirements.txt (line 79)) Using cached https://files.pythonhosted.org/packages/f8/62/231740df82cfcbe65deae6f5f3ae41f533ed50159be5783a1581e373a767/ruamel.yaml-0.15.69-cp35-cp35m-manylinux1_x86_64.whl Collecting s3transfer==0.1.13 (from -r requirements.txt (line 80)) Using cached https://files.pythonhosted.org/packages/d7/14/2a0004d487464d120c9fb85313a75cd3d71a7506955be458eebfe19a6b1d/s3transfer-0.1.13-py2.py3-none-any.whl Collecting scikit-learn==0.19.2 (from -r requirements.txt (line 81)) Using cached https://files.pythonhosted.org/packages/b6/e2/a1e254a4a4598588d4fe88b45ab88a226c289ecfd0f6c90474eb6a9ea6b3/scikit_learn-0.19.2-cp35-cp35m-manylinux1_x86_64.whl Collecting scipy==1.1.0 (from -r requirements.txt (line 82)) Using cached https://files.pythonhosted.org/packages/cd/32/5196b64476bd41d596a8aba43506e2403e019c90e1a3dfc21d51b83db5a6/scipy-1.1.0-cp35-cp35m-manylinux1_x86_64.whl Collecting simplejson==3.16.0 (from -r requirements.txt (line 83)) Collecting six==1.11.0 (from -r requirements.txt (line 84)) Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl Collecting sklearn-crfsuite==0.3.6 (from -r requirements.txt (line 85)) Using cached https://files.pythonhosted.org/packages/25/74/5b7befa513482e6dee1f3dd68171a6c9dfc14c0eaa00f885ffeba54fe9b0/sklearn_crfsuite-0.3.6-py2.py3-none-any.whl Collecting slackclient==1.3.0 (from -r requirements.txt (line 86)) Using cached https://files.pythonhosted.org/packages/0d/2f/1378e64a843a5a8a83d73caa59ac88c36c67e2b41ac0fab3422080ff13bd/slackclient-1.3.0-py2.py3-none-any.whl Collecting spacy<3.0.0,>=2.0.0 (from -r requirements.txt (line 87)) Downloading https://files.pythonhosted.org/packages/a6/71/9a127727e6130e10b84585d22f082b42f15cdb444d3604eb79c7bf249d39/spacy-2.0.16-cp35-cp35m-manylinux1_x86_64.whl (23.2MB) 100% |████████████████████████████████| 23.2MB 2.8MB/s Collecting tabulate==0.8.2 (from -r requirements.txt (line 89)) Collecting tensorboard==1.8.0 (from -r requirements.txt (line 90)) Using cached https://files.pythonhosted.org/packages/59/a6/0ae6092b7542cfedba6b2a1c9b8dceaf278238c39484f3ba03b03f07803c/tensorboard-1.8.0-py3-none-any.whl Collecting tensorflow==1.8.0 (from -r requirements.txt (line 91)) Using cached https://files.pythonhosted.org/packages/6d/dc/464f59597a5a8282585238e6e3a7bb3770c3c1f1dc8ee72bd5be257178ec/tensorflow-1.8.0-cp35-cp35m-manylinux1_x86_64.whl Collecting termcolor==1.1.0 (from -r requirements.txt (line 92)) Collecting thinc<6.13.0,>=6.12.0 (from -r requirements.txt (line 93)) Downloading https://files.pythonhosted.org/packages/e3/41/c072537c49900db582d0b4fab943d819b420db891d767c858a0aa36ee169/thinc-6.12.0-cp35-cp35m-manylinux1_x86_64.whl (1.9MB) 100% |████████████████████████████████| 1.9MB 14.5MB/s Collecting toolz==0.9.0 (from -r requirements.txt (line 94)) Collecting tqdm==4.26.0 (from -r requirements.txt (line 95)) Using cached https://files.pythonhosted.org/packages/79/43/19c9fee28110cd47f73e6bc596394337fe9f3e5825b4de402bbf30b3beb5/tqdm-4.26.0-py2.py3-none-any.whl Collecting twilio==6.17.0 (from -r requirements.txt (line 96)) Using cached https://files.pythonhosted.org/packages/78/e6/b4043da46b968103f75c7d4ebe45a84ffaa5a1185b0f751f685e2e71629a/twilio-6.17.0-py2.py3-none-any.whl Collecting Twisted==18.7.0 (from -r requirements.txt (line 97)) Collecting typing==3.6.6 (from -r requirements.txt (line 98)) Using cached https://files.pythonhosted.org/packages/4a/bd/eee1157fc2d8514970b345d69cb9975dcd1e42cd7e61146ed841f6e68309/typing-3.6.6-py3-none-any.whl Collecting tzlocal==1.5.1 (from -r requirements.txt (line 99)) Collecting ujson==1.35 (from -r requirements.txt (line 100)) Collecting urllib3==1.23 (from -r requirements.txt (line 101)) Using cached https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl Collecting websocket-client==0.53.0 (from -r requirements.txt (line 102)) Using cached https://files.pythonhosted.org/packages/14/d4/6a8cd4e7f67da465108c7cc0a307a1c5da7e2cdf497330b682069b1d4758/websocket_client-0.53.0-py2.py3-none-any.whl Collecting Werkzeug==0.14.1 (from -r requirements.txt (line 103)) Using cached https://files.pythonhosted.org/packages/20/c4/12e3e56473e52375aa29c4764e70d1b8f3efa6682bef8d0aae04fe335243/Werkzeug-0.14.1-py2.py3-none-any.whl Collecting wrapt==1.10.11 (from -r requirements.txt (line 104)) Collecting zope.interface==4.5.0 (from -r requirements.txt (line 105)) Downloading https://files.pythonhosted.org/packages/8c/e2/467655b0abe3ebc856d2feb9a1682adb37569b797890b2f295194b07453b/zope.interface-4.5.0-cp35-cp35m-manylinux1_x86_64.whl (166kB) 100% |████████████████████████████████| 174kB 16.7MB/s Requirement already satisfied: setuptools>=0.7 in ./venv/lib/python3.5/site-packages (from APScheduler==3.5.3->-r requirements.txt (line 2)) (40.4.3) Requirement already satisfied: wheel>=0.26; python_version >= "3" in ./venv/lib/python3.5/site-packages (from tensorboard==1.8.0->-r requirements.txt (line 90)) (0.32.2) Building wheels for collected packages: regex, en-core-web-sm Running setup.py bdist_wheel for regex ... done Stored in directory: /home/aanchan/.cache/pip/wheels/74/17/3f/c77bba99efd74ba1a19862c9dd97f4b6d735e2826721dc00ff Running setup.py bdist_wheel for en-core-web-sm ... done Stored in directory: /home/aanchan/.cache/pip/wheels/54/7c/d8/f86364af8fbba7258e14adae115f18dd2c91552406edc3fdaa Successfully built regex en-core-web-sm Installing collected packages: six, absl-py, pytz, tzlocal, APScheduler, astor, attrs, Automat, html5lib, bleach, python-dateutil, docutils, urllib3, jmespath, botocore, s3transfer, boto3, certifi, chardet, click, cloudpickle, humanfriendly, coloredlogs, colorhash, ConfigArgParse, constantly, cycler, cymem, toolz, cytoolz, decorator, dill, docopt, redis, fakeredis, idna, requests, fbmessenger, itsdangerous, MarkupSafe, Jinja2, Werkzeug, Flask, Flask-Cors, future, gast, greenlet, gevent, graphviz, grpcio, numpy, h5py, hyperlink, incremental, jsonpickle, jsonschema, scipy, Keras-Applications, PyYAML, Keras-Preprocessing, Keras, kiwisolver, zope.interface, PyHamcrest, Twisted, klein, Markdown, pyparsing, matplotlib, mattermostwrapper, msgpack, msgpack-numpy, murmurhash, networkx, packaging, pathlib, plac, preshed, protobuf, PyJWT, pykwalify, PySocks, python-crfsuite, python-telegram-bot, typing, simplejson, tqdm, rasa-nlu, twilio, tensorboard, termcolor, tensorflow, websocket-client, slackclient, scikit-learn, ruamel.yaml, rasa-core, regex, tabulate, sklearn-crfsuite, ujson, wrapt, thinc, spacy, en-core-web-sm Successfully installed APScheduler-3.5.3 Automat-0.7.0 ConfigArgParse-0.13.0 Flask-1.0.2 Flask-Cors-3.0.6 Jinja2-2.10 Keras-2.2.2 Keras-Applications-1.0.4 Keras-Preprocessing-1.0.2 Markdown-2.6.11 MarkupSafe-1.0 PyHamcrest-1.9.0 PyJWT-1.6.4 PySocks-1.6.8 PyYAML-3.13 Twisted-18.7.0 Werkzeug-0.14.1 absl-py-0.5.0 astor-0.7.1 attrs-18.2.0 bleach-1.5.0 boto3-1.9.8 botocore-1.12.16 certifi-2018.8.24 chardet-3.0.4 click-6.7 cloudpickle-0.5.6 coloredlogs-10.0 colorhash-1.0.2 constantly-15.1.0 cycler-0.10.0 cymem-2.0.2 cytoolz-0.9.0.1 decorator-4.3.0 dill-0.2.8.2 docopt-0.6.2 docutils-0.14 en-core-web-sm-2.0.0 fakeredis-0.10.3 fbmessenger-5.3.0 future-0.16.0 gast-0.2.0 gevent-1.3.6 graphviz-0.8.4 greenlet-0.4.15 grpcio-1.15.0 h5py-2.8.0 html5lib-0.9999999 humanfriendly-4.16.1 hyperlink-18.0.0 idna-2.7 incremental-17.5.0 itsdangerous-0.24 jmespath-0.9.3 jsonpickle-0.9.6 jsonschema-2.6.0 kiwisolver-1.0.1 klein-17.10.0 matplotlib-2.2.3 mattermostwrapper-2.1 msgpack-0.5.6 msgpack-numpy-0.4.3.2 murmurhash-0.28.0 networkx-2.2 numpy-1.15.1 packaging-17.1 pathlib-1.0.1 plac-0.9.6 preshed-2.0.1 protobuf-3.6.1 pykwalify-1.6.0 pyparsing-2.2.1 python-crfsuite-0.9.6 python-dateutil-2.7.3 python-telegram-bot-10.1.0 pytz-2018.5 rasa-core-0.10.3 rasa-nlu-0.13.4 redis-2.10.6 regex-2018.1.10 requests-2.19.1 ruamel.yaml-0.15.69 s3transfer-0.1.13 scikit-learn-0.19.2 scipy-1.1.0 simplejson-3.16.0 six-1.11.0 sklearn-crfsuite-0.3.6 slackclient-1.3.0 spacy-2.0.16 tabulate-0.8.2 tensorboard-1.8.0 tensorflow-1.8.0 termcolor-1.1.0 thinc-6.12.0 toolz-0.9.0 tqdm-4.26.0 twilio-6.17.0 typing-3.6.6 tzlocal-1.5.1 ujson-1.35 urllib3-1.23 websocket-client-0.53.0 wrapt-1.10.11 zope.interface-4.5.0

When trying to run script

python train_online.py Traceback (most recent call last): File "train_online.py", line 67, in run_twilio_bot() File "train_online.py", line 20, in run_twilio_bot interpreter = RasaNLUInterpreter('./models/current/nlu') File "/home/aanchan/tmp/synap-rasa-chatbot/venv/lib/python3.5/site-packages/rasa_core/interpreter.py", line 229, in init self._load_interpreter() File "/home/aanchan/tmp/synap-rasa-chatbot/venv/lib/python3.5/site-packages/rasa_core/interpreter.py", line 245, in _load_interpreter self.interpreter = Interpreter.load(self.model_directory) File "/home/aanchan/tmp/synap-rasa-chatbot/venv/lib/python3.5/site-packages/rasa_nlu/model.py", line 293, in load skip_validation) File "/home/aanchan/tmp/synap-rasa-chatbot/venv/lib/python3.5/site-packages/rasa_nlu/model.py", line 320, in create model_metadata, context) File "/home/aanchan/tmp/synap-rasa-chatbot/venv/lib/python3.5/site-packages/rasa_nlu/components.py", line 419, in load_component cached_component, context) File "/home/aanchan/tmp/synap-rasa-chatbot/venv/lib/python3.5/site-packages/rasa_nlu/registry.py", line 134, in load_component_by_name return component_clz.load(model_dir, metadata, cached_component, kwargs) File "/home/aanchan/tmp/synap-rasa-chatbot/venv/lib/python3.5/site-packages/rasa_nlu/utils/spacy_utils.py", line 126, in load nlp = spacy.load(model_name, parser=False) File "/home/aanchan/tmp/synap-rasa-chatbot/venv/lib/python3.5/site-packages/spacy/init.py", line 21, in load return util.load_model(name, overrides) File "/home/aanchan/tmp/synap-rasa-chatbot/venv/lib/python3.5/site-packages/spacy/util.py", line 119, in load_model raise IOError(Errors.E050.format(name=name)) OSError: [E050] Can't find model 'en'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.

After trying to download the model manually:

python -m spacy download en Requirement already satisfied: en_core_web_sm==2.0.0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz#egg=en_core_web_sm==2.0.0 in ./venv/lib/python3.5/site-packages (2.0.0)

Linking successful
/home/aanchan/tmp/synap-rasa-chatbot/venv/lib/python3.5/site-packages/en_core_web_sm
-->
/home/aanchan/tmp/synap-rasa-chatbot/venv/lib/python3.5/site-packages/spacy/data/en

You can now load the model via spacy.load('en')

Your Environment

ines commented 5 years ago

The issue you linked refers to a different problem where linking wouldn't actually work as expected internally. That should definitely be fixed.

Models are linked by creating a symlink in your spacy/data directory – that's a separate operation that can't be performed by pip when you install the model. Defining the model in your requirements.txt will only install it – if you also want to link it to a shortcut, this needs to be a separate step, e.g. you could run the command in your CI build after the dependencies were installed.

That said, if you're running automated pipelines, it's usually better to load and refer to models explicitly by their names instead of just shortcuts – e.g. en_core_web_sm instead of just en. This makes it immediately clear which model is loaded, and ensures that your application doesn't depend on symlink state.

lock[bot] commented 5 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.