PaddlePaddle / Paddle

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice (『飞桨』核心框架,深度学习&机器学习高性能单机、分布式训练和跨平台部署)
http://www.paddlepaddle.org/
Apache License 2.0
22.13k stars 5.56k forks source link

pyinstaller -D 打包paddlenlp 运行报错 #54490

Closed cqray1990 closed 1 year ago

cqray1990 commented 1 year ago

bug描述 Describe the Bug

Traceback (most recent call last): File "uie.py", line 2, in File "", line 1007, in _find_and_load File "", line 986, in _find_and_load_unlocked File "", line 680, in _load_unlocked File "PyInstaller/loader/pyimod02_importers.py", line 385, in exec_module File "paddlenlp/init.py", line 33, in File "", line 1007, in _find_and_load File "", line 986, in _find_and_load_unlocked File "", line 680, in _load_unlocked File "PyInstaller/loader/pyimod02_importers.py", line 385, in exec_module File "paddle/init.py", line 31, in File "", line 1007, in _find_and_load File "", line 986, in _find_and_load_unlocked File "", line 680, in _load_unlocked File "PyInstaller/loader/pyimod02_importers.py", line 385, in exec_module File "paddle/framework/init.py", line 17, in File "", line 1007, in _find_and_load File "", line 986, in _find_and_load_unlocked File "", line 680, in _load_unlocked File "PyInstaller/loader/pyimod02_importers.py", line 385, in exec_module File "paddle/framework/random.py", line 17, in File "", line 1007, in _find_and_load File "", line 986, in _find_and_load_unlocked File "", line 680, in _load_unlocked File "PyInstaller/loader/pyimod02_importers.py", line 385, in exec_module File "paddle/fluid/init.py", line 36, in File "", line 1007, in _find_and_load File "", line 986, in _find_and_load_unlocked File "", line 680, in _load_unlocked File "PyInstaller/loader/pyimod02_importers.py", line 385, in exec_module File "paddle/fluid/framework.py", line 35, in File "", line 1007, in _find_and_load File "", line 986, in _find_and_load_unlocked File "", line 680, in _load_unlocked File "PyInstaller/loader/pyimod02_importers.py", line 385, in exec_module File "paddle/fluid/core.py", line 393, in File "paddle/fluid/core.py", line 385, in set_paddle_lib_path TypeError: sequence item 0: expected str instance, NoneType found [14315] Failed to execute script 'uieextract' due to unhandled exception!

其他补充信息 Additional Supplementary Information

python3.9 cuda116 cudnn8.8

absl-py==1.4.0 addict==2.4.0 aiohttp==3.8.4 aiosignal==1.3.1 altgraph==0.17.3 amqp==5.1.1 anyio==3.6.2 appdirs==1.4.4 array-record==0.2.0 asgiref==3.6.0 astor==0.8.1 async-timeout==4.0.2 attr==0.3.1 attrs==23.1.0 auto-labeling-pipeline==0.1.21 azure-core==1.26.4 azure-storage-blob==12.16.0 Babel==2.12.1 bce-python-sdk==0.8.83 bcrypt==4.0.1 billiard==3.6.4.0 blinker==1.6.2 boto==2.49.0 boto3==1.16.63 botocore==1.19.63 boxing==0.1.4 Brotli==1.0.9 cachetools==5.3.0 celery==5.2.7 certifi==2023.5.7 cffi==1.15.1 chardet==4.0.0 charset-normalizer==2.0.12 click==8.1.3 click-didyoumean==0.3.0 click-plugins==1.1.1 click-repl==0.2.0 colorama==0.4.6 coloredlogs==15.0.1 colorlog==6.7.0 contourpy==1.0.7 coreapi==2.3.3 coreschema==0.0.4 cryptography==40.0.2 cssselect==1.2.0 cssutils==2.6.0 cycler==0.11.0 datasets==2.12.0 decorator==5.1.1 defusedxml==0.7.1 dill==0.3.4 dj-database-url==0.5.0 dj-rest-auth==2.2.8 Django==4.2.1 django-allauth==0.50.0 django-annoying==0.10.6 django-celery-results==2.4.0 django-cleanup==6.0.0 django-cors-headers==3.12.0 django-debug-toolbar==3.2.1 django-drf-filepond==0.4.1 django-extensions==3.1.0 django-filter==21.1 django-health-check==3.17.0 django-model-utils==4.1.1 django-polymorphic==3.1.0 django-ranged-fileresponse==0.1.2 django-rest-polymorphic==0.1.10 django-rest-swagger==2.2.0 django-rq==2.5.1 django-storages==1.13.2 django-user-agents==0.4.0 djangorestframework==3.13.1 djangorestframework-xml==2.0.0 dm-tree==0.1.8 doccano==1.8.3 drf-dynamic-fields==0.3.0 drf-flex-fields==0.9.5 drf-generators==0.3.0 drf-yasg==1.20.0 environs==9.5.0 et-xmlfile==1.1.0 etils==1.2.0 exceptiongroup==1.1.1 expiringdict==1.1.4 fastapi==0.95.1 fastdeploy-gpu-python==0.0.0 fastdeploy-tools==0.0.5 ffmpy==0.3.0 filelock==3.12.0 filetype==1.2.0 Flask==2.3.2 flask-babel==3.1.0 flatbuffers==23.5.9 flower==1.2.0 fonttools==4.39.4 frozenlist==1.3.3 fsspec==2023.5.0 furl==2.1.3 future==0.18.3 gensim==4.3.1 gevent==22.10.2 geventhttpclient==2.0.2 gitdb==4.0.10 GitPython==3.1.31 google-api-core==2.10.0 google-auth==2.19.0 google-auth-oauthlib==1.0.0 google-cloud-appengine-logging==1.1.0 google-cloud-audit-log==0.2.0 google-cloud-core==2.3.2 google-cloud-logging==2.7.2 google-cloud-storage==2.5.0 google-crc32c==1.5.0 google-resumable-media==2.3.3 googleapis-common-protos==1.56.4 gradio==3.11.0 greenlet==2.0.2 grpc-google-iam-v1==0.12.3 grpcio==1.55.0rc1 grpcio-status==1.48.2 gunicorn==20.1.0 h11==0.12.0 harvesttext==0.8.1.8 htmlmin==0.1.12 httpcore==0.15.0 httpx==0.24.0 huggingface-hub==0.14.1 humanfriendly==10.0 humanize==4.6.0 idna==3.4 imageio==2.29.0 imgaug==0.4.0 importlib-metadata==6.6.0 importlib-resources==5.12.0 inflection==0.5.1 iniconfig==2.0.0 install==1.3.5 isodate==0.6.1 itsdangerous==2.1.2 itypes==1.2.0 jieba==0.42.1 Jinja2==3.1.2 jmespath==0.10.0 joblib==1.2.0 jsonschema==3.2.0 kiwisolver==1.4.4 kombu==5.2.4 label-studio-converter==0.0.44 label-studio-tools==0.0.1 lanms==1.0.2 launchdarkly-server-sdk==7.3.0 lazy_loader==0.2 linkify-it-py==2.0.2 lmdb==1.4.1 lml==0.1.0 lockfile==0.12.2 loguru==0.7.0 lxml==4.9.2 Markdown==3.4.3 markdown-it-py==2.2.0 MarkupSafe==2.1.2 marshmallow==3.19.0 matplotlib==3.7.1 mdit-py-plugins==0.3.5 mdurl==0.1.2 mpmath==1.3.0 multidict==6.0.4 multiprocess==0.70.12.2 natsort==8.3.1 netron==6.9.2 networkx==3.1 nltk==3.6.7 numpy==1.24.3 oauthlib==3.2.2 onnx==1.14.0 onnxruntime-gpu==1.13.1 openapi-codec==1.3.2 opencc-python-reimplemented==0.1.7 opencv-python==4.7.0.72 openpyxl==3.1.2 opt-einsum==3.3.0 ordered-set==4.0.2 orderedmultidict==1.0.1 orjson==3.8.12 packaging==23.1 paddle-bfloat==0.1.7 paddle2onnx==1.0.6 paddlefsl==1.1.0 paddlenlp==2.5.2.post0 paddlepaddle-gpu==0.0.0.post116 pandas==1.4.2 paramiko==3.1.0 Pillow==9.0.1 pluggy==1.0.0 Polygon3==3.0.9.1 premailer==3.10.0 prometheus-client==0.16.0 promise==2.3 prompt-toolkit==3.0.38 proto-plus==1.22.2 protobuf==3.20.2 psutil==5.9.5 psycopg2-binary==2.9.1 pyarrow==12.0.0 pyasn1==0.5.0 pyasn1-modules==0.3.0 pyclipper==1.3.0.post4 pycparser==2.21 pycryptodome==3.17 pydantic==1.8.2 pydub==0.25.1 pyexcel==0.7.0 pyexcel-io==0.6.6 pyexcel-xlsx==0.6.0 Pygments==2.15.1 pyinstaller==5.12.0 pyinstaller-hooks-contrib==2023.3 PyJWT==2.7.0 PyNaCl==1.5.0 pyparsing==3.1.0b1 pypinyin==0.49.0 pyRFC3339==1.1 pyrsistent==0.19.3 pytest==7.3.1 python-dateutil==2.8.1 python-docx==0.8.11 python-dotenv==1.0.0 python-json-logger==2.0.4 python-louvain==0.16 python-multipart==0.0.6 python-rapidjson==1.10 python3-openid==3.2.0 pytz==2023.3 PyWavelets==1.4.1 PyYAML==6.0 rapidfuzz==3.0.0 rarfile==4.0 rdflib==6.3.2 redis==4.5.5 regex==2023.5.5 requests==2.27.1 requests-oauthlib==1.3.1 responses==0.18.0 rich==13.3.5 rq==1.10.1 rsa==4.9 ruamel.yaml==0.17.26 ruamel.yaml.clib==0.2.7 rules==2.2 s3transfer==0.3.7 scikit-image==0.20.0 scikit-learn==1.2.2 scipy==1.9.1 seaborn==0.12.2 semver==2.13.0 sentencepiece==0.1.99 sentry-sdk==1.22.2 seqeval==1.2.2 shapely==2.0.1 shortuuid==1.0.11 simplejson==3.19.1 six==1.16.0 smart-open==6.3.0 smmap==5.0.0 sniffio==1.3.0 some-package==0.1 SQLAlchemy==1.4.48 sqlparse==0.4.4 starlette==0.26.1 sympy==1.12 tensorboard==2.13.0 tensorboard-data-server==0.7.0 tensorflow-metadata==1.13.1 termcolor==2.3.0 texttable==1.6.7 threadpoolctl==3.1.0 tifffile==2023.4.12 toml==0.10.2 tomli==2.0.1 torch==1.13.1+cu116 torchaudio==0.13.1+cu116 torchvision==0.14.1+cu116 tornado==6.3.1 tqdm==4.65.0 tritonclient==2.33.0 typer==0.9.0 typing_extensions==4.5.0 tzdata==2023.3 ua-parser==0.16.1 uc-micro-py==1.0.2 ujson==5.7.0 ultralytics==8.0.112 uritemplate==4.1.1 urllib3==1.26.15 user-agents==2.2.0 uvicorn==0.16.0 vine==5.0.0 visualdl==2.5.2 w3lib==2.1.1 waitress==2.1.2 wcwidth==0.2.6 websockets==11.0.3 Werkzeug==2.3.4 whitenoise==6.4.0 wrapt==1.15.0 x2paddle==1.4.1 xmljson==0.2.0 xxhash==3.2.0 yarl==1.9.2 zipp==3.15.0 zope.event==4.6 zope.interface==6.1a2

kwdchol commented 1 year ago

hello 请问解决了吗 我pyinstaller 打包后 运行程序也报这个错误

kwdchol commented 1 year ago

可以参考一下https://github.com/PaddlePaddle/Paddle/issues/51212 修改core.py文件 然后 把路径修改为自己的路径


# set paddle lib path
def set_paddle_lib_path():
    site_dirs = (
        site.getsitepackages()
        if hasattr(site, 'getsitepackages')
        else [x for x in sys.path if 'site-packages' in x]
    )
    for site_dir in site_dirs:
        lib_dir = os.path.sep.join([site_dir, 'paddle', 'libs'])
        if os.path.exists(lib_dir):
            _set_paddle_lib_path(lib_dir)
            set_paddle_custom_device_lib_path(
                os.path.sep.join([lib_dir, '..', '..', 'paddle_custom_device'])
            )
            return
    if getattr(sys, 'frozen', False):
        basedir = sys._MEIPASS
    else:
        basedir = os.path.dirname(os.path.abspath(__file__))
    if hasattr(site, 'USER_SITE'):
        #lib_dir2 = os.path.sep.join([site.USER_SITE, 'paddle', 'libs'])
        lib_dir = os.path.sep.join([basedir, 'paddle', 'libs'])
        print('libdir:',lib_dir)
        #lib_dir = '/opt/homebrew/anaconda3/envs/ocr2udp/lib/python3.10/site-packages/paddle/libs'
        if os.path.exists(lib_dir):
            _set_paddle_lib_path(lib_dir)
            set_paddle_custom_device_lib_path(
                os.path.sep.join([lib_dir, '..', '..', 'paddle_custom_device'])
            )
`
其中 lib_dir = os.path.sep.join([basedir, 'paddle', 'libs']) 执行结果是 /Users/iqiyi/gitroot/ocr2udp/dist/xxx/paddle/libs
打包的时候添加参数--add-binary "/opt/homebrew/anaconda3/envs/ocr2udp/lib/python3.10/site-packages/paddle/libs:paddle/libs" \  
遇到tools  picodet_postprocess 等模块找不到 
使用-p 包含这个目录  
--hidden-import 添加这个模块
放一个我自己在mac m2上面完整打包的命令
```bash
pyinstaller   \
--add-data "/Users/iqiyi/gitroot/ocr2udp/models:models" \
--add-data "./resource/SIMFANG_GB2312.ttf:resource" \
--add-data "/opt/homebrew/anaconda3/envs/ocr2udp/lib/python3.10/site-packages/paddleocr/ppocr/utils/ppocr_keys_v1.txt:ppocr/utils/" \
--add-binary "/opt/homebrew/anaconda3/envs/ocr2udp/lib/python3.10/site-packages/paddle/libs:paddle/libs" \
-p /opt/homebrew/anaconda3/envs/ocr2udp/lib/python3.10/site-packages/paddleocr/  \
-p /opt/homebrew/anaconda3/envs/ocr2udp/lib/python3.10/site-packages/paddleocr/ppocr/utils/e2e_utils/  \
-p /opt/homebrew/anaconda3/envs/ocr2udp/lib/python3.10/site-packages/paddleocr/ppocr/postprocess/ \
--hidden-import tools \
--hidden-import extract_textpoint_slow \
--hidden-import picodet_postprocess \
-D  main.py`