search

explosion / spaCy

💫 Industrial-strength Natural Language Processing (NLP) in Python
https://spacy.io
MIT License
29.58k stars 4.35k forks source link

Spacy errors out with PyFPE_jbuf when packaging in a PEX file and running inside a docker container #2887

Closed aritrachatterjee15 closed 5 years ago

aritrachatterjee15 commented 5 years ago

How to reproduce the problem

I have a requirements.txt file that looks like this: scikit-learn==0.19.2 lxml==4.2.5 numpy==1.15.1 scipy==1.1.0 xmltodict==0.11.0 tensorflow==1.8 chardet==3.0.4 beautifulsoup4==4.6.3 PyYAML==3.13 dpath==1.4.2 keras==2.2.4 spacy==2.0.16 dedupe==1.9.3 pandas==0.23.4 openpyxl==2.5.9 xlrd==1.1.0 PyInquirer==1.0.2 persist-queue==0.4.1 progress==1.4 tensorflow-hub==0.1.1 nltk==3.3 en-core-web-md==2.0.0

I am using Pantsbuild in order to gather the dependencies and then building a PEX file using this. When I execute this file in my local Ubuntu 16.04 VM, it works fine. However, when I copy this PEX file into a docker image '''python:3.5''' and execute the same command on the PEX file, I see the below error. Any clues what could be the cause?

    from spacy.lang.en import English
  File "/root/.pex/install/spacy-2.0.16-cp35-cp35m-manylinux1_x86_64.whl.e389cc5823f8f17d7b31453f151cc429b3fbebcd/spacy-2.0.16-cp35-cp35m-manylinux1_x86_64.whl/spacy/__init__.py", line 8, in <module>
    from thinc.neural.util import prefer_gpu, require_gpu
  File "/root/.pex/install/thinc-6.12.0-cp35-cp35m-manylinux1_x86_64.whl.a14889ae18f3e114a3a3ba2822b67f0d241fb2e9/thinc-6.12.0-cp35-cp35m-manylinux1_x86_64.whl/thinc/neural/__init__.py", line 1, in <module>
    from ._classes.model import Model
  File "/root/.pex/install/thinc-6.12.0-cp35-cp35m-manylinux1_x86_64.whl.a14889ae18f3e114a3a3ba2822b67f0d241fb2e9/thinc-6.12.0-cp35-cp35m-manylinux1_x86_64.whl/thinc/neural/_classes/model.py", line 12, in <module>
    from ..train import Trainer
  File "/root/.pex/install/thinc-6.12.0-cp35-cp35m-manylinux1_x86_64.whl.a14889ae18f3e114a3a3ba2822b67f0d241fb2e9/thinc-6.12.0-cp35-cp35m-manylinux1_x86_64.whl/thinc/neural/train.py", line 3, in <module>
    from .optimizers import Adam, SGD, linear_decay
  File "optimizers.pyx", line 13, in init thinc.neural.optimizers
  File "ops.pyx", line 18, in init thinc.neural.ops
  File "/root/.pex/install/cytoolz-0.9.0.1-cp35-cp35m-linux_x86_64.whl.53465d9b9c496dc8ee1ce1d75a845f5fda455ae6/cytoolz-0.9.0.1-cp35-cp35m-linux_x86_64.whl/cytoolz/__init__.py", line 3, in <module>
    from .functoolz import *
ImportError: /root/.pex/install/cytoolz-0.9.0.1-cp35-cp35m-linux_x86_64.whl.53465d9b9c496dc8ee1ce1d75a845f5fda455ae6/cytoolz-0.9.0.1-cp35-cp35m-linux_x86_64.whl/cytoolz/functoolz.cpython-35m-x86_64-linux-gnu.so: undefined symbol: PyFPE_jbuf

Your Environment

syllog1sm commented 5 years ago

This is a common problem, basically the way Python is built can affect its ABI. If you google around for that symbol you should find a lot of guidance on how to resolve this. You need to make sure the same Python is used at build and runtime.

lock[bot] commented 5 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.