How to package spacy 2.3.2 wih Pyinstaller?

[vaibhavgautam]

<I am unable to package my code written with the help of spacy 2.3.2 version python 3.8 with pyinstaller 4.1dev and 4.0 but my exe is giving issues ?

sometimes it is giving error as : python 3.7 package distribution error of sklearn ModuleNotFoundError: No module named 'srsly.msgpack.util' package distribution error scikit-learn not found

python 3.8.5 struct.error: 'i' format requires -2147483648 <= number <= 2147483647

please tell me the about all the hidden imports and data files with respect to any one environment?

This are my datas and hidden imports in my .spec file in pyinstaller(4.0):

import PyInstaller

datas = []

datas.extend(PyInstaller.utils.hooks.collect_data_files('spacy.lang', include_py_files = True)) datas.extend(PyInstaller.utils.hooks.collect_data_files('spacy_lookups_data'))

datas.extend(PyInstaller.utils.hooks.collect_data_files('thinc'))

datas.extend(PyInstaller.utils.hooks.collect_data_files('en_core_web_lg'))

hiddenimports=[

    'spacy.kb',

    'spacy.lexeme',

    'spacy.matcher._schemas',

    'spacy.morphology',

    'spacy.parts_of_speech',

    'spacy.syntax._beam_utils',

    'spacy.syntax._parser_model',

    'spacy.syntax.arc_eager',

    'spacy.syntax.ner',

    'spacy.syntax.nn_parser',

    'spacy.syntax.stateclass',

    'spacy.syntax.transition_system',

    'spacy.tokens._retokenize',

    'spacy.tokens.morphanalysis',

    'spacy.tokens.underscore',

    'spacy._align',

    'blis',

    'blis.py',

    'cymem',
    'cymem.cymem',

    'murmurhash',

    'murmurhash.mrmr',

    'preshed.maps',

    'srsly.msgpack.util',

    'thinc.extra.search',

    'thinc.linalg',

    'thinc.neural._aligned_alloc',

    'thinc.neural._custom_kernels',
   'textacy',

    'sklearn.utils._cython_blas', 
    'sklearn.neighbors',
    'sklearn.neighbors.typedefs',
    'sklearn.neighbors.quad_tree',
    'sklearn.tree._utils',
    'sklearn.utils.sparsetools._graph_validation',
    'sklearn.utils.sparsetools._graph_tools',
    'sklearn.utils.lgamma',
    'sklearn.utils.weight_vector',
    'sklearn.utils.fixes',
    'sklearn.utils.extmath',
    'sklearn.metrics.ranking',

     ],

Your Environment

<Spacy 2.3.2 Ubuntu 18.04 Pyinstaller 4.0 Environment

Name Version Build Channel

_libgcc_mutex 0.1 main
altgraph 0.17 pypi_0 pypi argon2-cffi 20.1.0 py38h1e0a361_1 conda-forge attrs 20.1.0 pyh9f0ad1d_0 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 py_2 conda-forge backports.functools_lru_cache 1.6.1 py_0 conda-forge blas 1.0 openblas
bleach 3.1.5 pyh9f0ad1d_0 conda-forge brotlipy 0.7.0 py38h1e0a361_1000 conda-forge ca-certificates 2020.7.22 0
cachetools 4.1.1 pypi_0 pypi catalogue 1.0.0 py_0 conda-forge certifi 2020.6.20 py38_0
cffi 1.14.2 py38he30daa8_0
chardet 3.0.4 py38h32f6830_1006 conda-forge cryptography 3.0 py38h766eaa4_0 conda-forge cymem 2.0.3 py38h950e882_2 conda-forge cython-blis 0.4.1 py38h1e0a361_1 conda-forge cytoolz 0.10.1 pypi_0 pypi dbus 1.13.16 hb2f20db_0
decorator 4.4.2 py_0 conda-forge defusedxml 0.6.0 py_0 conda-forge en-core-web-lg 2.3.1 pypi_0 pypi entrypoints 0.3 py38h32f6830_1001 conda-forge expat 2.2.9 he6710b0_2
fontconfig 2.13.0 h9420a91_0
freetype 2.10.2 h5ab3b9f_0
glib 2.65.0 h3eb4bd4_0
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb31296c_0
icu 58.2 he6710b0_3
idna 2.10 pyh9f0ad1d_0 conda-forge importlib-metadata 1.7.0 py38h32f6830_0 conda-forge importlib_metadata 1.7.0 0 conda-forge ipykernel 5.3.4 py38h23f93f0_0 conda-forge ipython 7.17.0 py38h1cdfbd6_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge ipywidgets 7.5.1 py_0
jedi 0.15.2 py38_0 conda-forge jellyfish 0.8.2 pypi_0 pypi jinja2 2.11.2 pyh9f0ad1d_0 conda-forge joblib 0.16.0 py_0
jpeg 9b h024ee3a_2
json5 0.9.4 pyh9f0ad1d_0 conda-forge jsonschema 3.2.0 py38h32f6830_1 conda-forge jupyter 1.0.0 py38_7
jupyter_client 6.1.7 py_0 conda-forge jupyter_console 6.1.0 py_0
jupyter_core 4.6.3 py38h32f6830_1 conda-forge jupyterlab 2.2.6 py_0 conda-forge jupyterlab_server 1.2.0 py_0 conda-forge ld_impl_linux-64 2.33.1 h53a641e_7
libblas 3.8.0 17_openblas conda-forge libcblas 3.8.0 17_openblas conda-forge libedit 3.1.20191231 h14c3975_1
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
libgfortran-ng 7.5.0 hdf63c60_16 conda-forge liblapack 3.8.0 17_openblas conda-forge libopenblas 0.3.10 pthreads_hb3c22a3_4 conda-forge libpng 1.6.37 hbc83047_0
libsodium 1.0.18 h516909a_0 conda-forge libstdcxx-ng 9.1.0 hdf63c60_0
libuuid 1.0.3 h1bed415_2
libxcb 1.14 h7b6447c_0
libxml2 2.9.10 he19cac6_1
markupsafe 1.1.1 py38h1e0a361_1 conda-forge mistune 0.8.4 py38h1e0a361_1001 conda-forge murmurhash 1.0.0 py38h950e882_0 conda-forge nbconvert 5.6.1 py38h32f6830_1 conda-forge nbformat 5.0.7 py_0 conda-forge ncurses 6.2 he6710b0_1
networkx 2.5 pypi_0 pypi notebook 6.1.1 py38_0
numpy 1.19.1 py38h30dfecb_0
numpy-base 1.19.1 py38h75fe3a5_0
openssl 1.1.1g h7b6447c_0
packaging 20.4 pyh9f0ad1d_0 conda-forge pandas 1.1.1 py38he6710b0_0
pandoc 2.10.1 h516909a_0 conda-forge pandocfilters 1.4.2 py_1 conda-forge parso 0.5.2 py_0
pcre 8.44 he6710b0_0
pexpect 4.8.0 py38h32f6830_1 conda-forge pickleshare 0.7.5 py38h32f6830_1001 conda-forge pip 20.2.2 py38_0
plac 1.0.0 py_0 conda-forge preshed 3.0.2 py38h950e882_3 conda-forge prometheus_client 0.8.0 pyh9f0ad1d_0 conda-forge prompt-toolkit 3.0.5 py_0
prompt_toolkit 3.0.5 0
ptyprocess 0.6.0 py_1001 conda-forge pycparser 2.20 pyh9f0ad1d_2 conda-forge pyemd 0.5.1 pypi_0 pypi pygments 2.6.1 py_0 conda-forge pyinstaller 4.0 pypi_0 pypi pyinstaller-hooks-contrib 2020.7 pypi_0 pypi pyopenssl 19.1.0 py_1 conda-forge pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge pypdf2 1.26.0 pypi_0 pypi pyphen 0.9.5 pypi_0 pypi pyqt 5.9.2 py38h05f1152_4
pyrsistent 0.16.0 py38h1e0a361_0 conda-forge pysocks 1.7.1 py38h32f6830_1 conda-forge python 3.8.5 hcff3b4d_0
python-dateutil 2.8.1 py_0
python_abi 3.8 1_cp38 conda-forge pytz 2020.1 py_0
pyzmq 19.0.2 py38ha71036d_0 conda-forge qt 5.9.7 h5867ecd_1
qtconsole 4.7.6 py_0
qtpy 1.9.0 py_0
readline 8.0 h7b6447c_0
requests 2.24.0 pyh9f0ad1d_0 conda-forge scikit-learn 0.23.2 py38h0573a6f_0
scipy 1.5.2 py38habc2bb6_0
send2trash 1.5.0 py_0 conda-forge setuptools 49.6.0 py38_0
sip 4.19.13 py38he6710b0_0
six 1.15.0 py_0
spacy 2.3.2 pypi_0 pypi spacy-lookups-data 0.3.2 pypi_0 pypi sqlite 3.33.0 h62c20be_0
srsly 1.0.2 py38h950e882_0 conda-forge terminado 0.8.3 py38h32f6830_1 conda-forge testpath 0.4.4 py_0 conda-forge textacy 0.10.0 pypi_0 pypi thinc 7.4.1 py38hbf85e49_0 conda-forge threadpoolctl 2.1.0 pyh5ca1d4c_0
tk 8.6.10 hbc83047_0
toolz 0.10.0 pypi_0 pypi tornado 6.0.4 py38h1e0a361_1 conda-forge tqdm 4.48.2 pyh9f0ad1d_0 conda-forge traitlets 4.3.3 py38h32f6830_1 conda-forge urllib3 1.25.10 py_0 conda-forge wasabi 0.7.1 pyh9f0ad1d_0 conda-forge wcwidth 0.2.5 pyh9f0ad1d_1 conda-forge webencodings 0.5.1 py_1 conda-forge wheel 0.35.1 py_0
widgetsnbextension 3.5.1 py38_0
xz 5.2.5 h7b6447c_0
zeromq 4.3.2 he1b5a44_3 conda-forge zipp 3.1.0 py_0 conda-forge zlib 1.2.11 h7b6447c_3

[vaibhavgautam]

Made my environment again updated above and getting this issue with scikit-learn: Traceback (most recent call last): File "text_isum_new.py", line 10, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "/home/vaibhav/anaconda3/envs/Text_AI/lib/python3.8/site-packages/PyInstaller/loader/pyimod03_importers.py", line 493, in exec_module exec(bytecode, module.dict) File "textacy/init.py", line 5, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "/home/vaibhav/anaconda3/envs/Text_AI/lib/python3.8/site-packages/PyInstaller/loader/pyimod03_importers.py", line 493, in exec_module exec(bytecode, module.dict) File "textacy/corpus.py", line 19, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "/home/vaibhav/anaconda3/envs/Text_AI/lib/python3.8/site-packages/PyInstaller/loader/pyimod03_importers.py", line 493, in exec_module exec(bytecode, module.dict) File "textacy/io/init.py", line 15, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "/home/vaibhav/anaconda3/envs/Text_AI/lib/python3.8/site-packages/PyInstaller/loader/pyimod03_importers.py", line 493, in exec_module exec(bytecode, module.dict) File "textacy/io/spacy.py", line 16, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "/home/vaibhav/anaconda3/envs/Text_AI/lib/python3.8/site-packages/PyInstaller/loader/pyimod03_importers.py", line 493, in exec_module exec(bytecode, module.dict) File "textacy/spacier/init.py", line 1, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "/home/vaibhav/anaconda3/envs/Text_AI/lib/python3.8/site-packages/PyInstaller/loader/pyimod03_importers.py", line 493, in exec_module exec(bytecode, module.dict) File "textacy/spacier/components.py", line 11, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "/home/vaibhav/anaconda3/envs/Text_AI/lib/python3.8/site-packages/PyInstaller/loader/pyimod03_importers.py", line 493, in exec_module exec(bytecode, module.dict) File "textacy/text_stats.py", line 16, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "/home/vaibhav/anaconda3/envs/Text_AI/lib/python3.8/site-packages/PyInstaller/loader/pyimod03_importers.py", line 493, in exec_module exec(bytecode, module.dict) File "textacy/extract.py", line 21, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "/home/vaibhav/anaconda3/envs/Text_AI/lib/python3.8/site-packages/PyInstaller/loader/pyimod03_importers.py", line 493, in exec_module exec(bytecode, module.dict) File "textacy/spacier/utils.py", line 17, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "/home/vaibhav/anaconda3/envs/Text_AI/lib/python3.8/site-packages/PyInstaller/loader/pyimod03_importers.py", line 493, in exec_module exec(bytecode, module.dict) File "textacy/spacier/core.py", line 16, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "/home/vaibhav/anaconda3/envs/Text_AI/lib/python3.8/site-packages/PyInstaller/loader/pyimod03_importers.py", line 493, in exec_module exec(bytecode, module.dict) File "textacy/lang_utils.py", line 221, in File "textacy/lang_utils.py", line 91, in init File "textacy/lang_utils.py", line 117, in _get_model_id File "pkg_resources/init.py", line 480, in get_distribution File "pkg_resources/init.py", line 356, in get_provider File "pkg_resources/init.py", line 899, in require File "pkg_resources/init.py", line 785, in resolve pkg_resources.DistributionNotFound: The 'scikit-learn' distribution was not found and is required by the application [32555] Failed to execute script text_isum_new

[svlandeg]

I'm not sure what is happening here, but are you sure this is related to spaCy? I don't think scikit-learn is a dependency in spacy or thinc - but it is in textacy (which is not published by us).

[no-response[bot]]

This issue has been automatically closed because there has been no response to a request for more information from the original author. With only the information that is currently in the issue, there's not enough information to take action. If you're the original author, feel free to reopen the issue if you have or find the answers needed to investigate further.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.