Jetson TX2 NX利用PPOCRv3GPU推理找不到PPOCRv3模块

[ChihoJack]

环境

• 【FastDeploy版本】： fastdeploy-gpu-python==0.2.1
• 【编译命令】使用安装文档Jetson部署环境编译安装(编译安装成功)
• 【系统平台】: Linux x64(Ubuntu 18.04) Jetpack4.6
• 【硬件】： NVIDIA Pascalm architecture with 256 NVIDIAR CUDAR cores
• 【编译语言】： Python3.6.9

---

使用fastdeploy中的PPOCRv3的GPU推理找不到PPOCRv3的模块，但是使用CPU推理却又可以，推理代码如下： OPENBLAS_CORETYPE=ARMV8 python3 test.py --det_model ch_PP-OCRv3_det_infer --cls_model ch_ppocr_mobile_v2.0_cls_infer --rec_model ch_PP-OCRv3_rec_infer --rec_label_file ppocr_keys_v1.txt --image 12.jpg --device gpu

运行之后出现以下的情况： [WARNING] fastdeploy/backends/ort/ort_backend.cc(57)::BuildOption Compiled fastdeploy with onnxruntime doesn't support GPU, the available providers are CPUExecutionProvider, will fallback to CPUExecutionProvider. [INFO] fastdeploy/runtime.cc(283)::Init Runtime initialized with Backend::ORT in Device::GPU. [WARNING] fastdeploy/backends/ort/ort_backend.cc(57)::BuildOption Compiled fastdeploy with onnxruntime doesn't support GPU, the available providers are CPUExecutionProvider, will fallback to CPUExecutionProvider. [INFO] fastdeploy/runtime.cc(283)::Init Runtime initialized with Backend::ORT in Device::GPU. [WARNING] fastdeploy/backends/ort/ort_backend.cc(57)::BuildOption Compiled fastdeploy with onnxruntime doesn't support GPU, the available providers are CPUExecutionProvider, will fallback to CPUExecutionProvider. [INFO] fastdeploy/runtime.cc(283)::Init Runtime initialized with Backend::ORT in Device::GPU. Traceback (most recent call last): File "test.py", line 197, in ppocr_v3 = fd.vision.ocr.PPOCRv3( AttributeError: module 'fastdeploy.vision.ocr' has no attribute 'PPOCRv3' 请问该如何解决？

[wang-xinyu]

版本是fastdeploy-gpu-python==0.2.1 ? 试下最新代码编译，附上编译和安装的步骤

[ChihoJack]

版本是fastdeploy-gpu-python==0.2.1 ? 试下最新代码编译，附上编译和安装的步骤

编译代码： git clone https://github.com/PaddlePaddle/FastDeploy.git cd FastDeploy/python export BUILD_ON_JETSON=ON export ENABLE_VISION=ON

ENABLE_PADDLE_BACKEND & PADDLEINFERENCE_DIRECTORY为可选项

export ENABLE_PADDLE_BACKEND=ON export PADDLEINFERENCE_DIRECTORY=/Download/paddle_inference_jetson

python setup.py build python setup.py bdist_wheel 安装步骤： 编译完之后安装fastdeploy-gpu-python 安装代码如下： pip3 install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html 按照系统自动选择了fastdeploy-gpu-python==0.2.1

[wang-xinyu]

不是 pip3 install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html

而是pip install dist/xxx.whl，dist目录下生成的wheel文件

先uninstall，再install

[ChihoJack]

我安装了wheel的文件，但是还是一样，这个方法我试过了

[wang-xinyu]

which python看一下 which pip看一下 要确保pip和python是对应的 然后pip list看一下

[ChihoJack]

好，我现在试试看看

[ChihoJack]

lcfc@lcfc-desktop:~$ which python /usr/bin/python lcfc@lcfc-desktop:~$ which pip /home/lcfc/.local/bin/pip lcfc@lcfc-desktop:~$ which pip3 /home/lcfc/.local/bin/pip3

[ChihoJack]

不知道是不是版本的问题

[wang-xinyu]

看起来不对，建个conda环境 安装python和pip，然后编译和安装FastDeploy wheel吧

[ChihoJack]

jetson环境无法安装conda，我一开始也想装个conda，但是会出现这个错误代码：Illegal instruction (core dumped)

[ChihoJack]

所以我每次运行python3都只能加个OPENBLAS_CORETYPE=ARMV8，要不然import fastdeploy也会出现Illegal instruction (core dumped)

[wang-xinyu]

python3 -m pip list呢？

编译的时候用的是python还是python3命令

[ChihoJack]

python3的命令，如果直接用python命令的话只能使用python2.7的

[wang-xinyu]

python3 -m pip list看下

[ChihoJack]

lcfc@lcfc-desktop:~/Documents/FastDeploy-develop/examples/vision/ocr/PP-OCRv3/python$ python3 -m pip

Usage:
/usr/bin/python3 -m pip [options]

Commands: install Install packages. download Download packages. uninstall Uninstall packages. freeze Output installed packages in requirements format. list List installed packages. show Show information about installed packages. check Verify installed packages have compatible dependencies. config Manage local and global configuration. search Search PyPI for packages. cache Inspect and manage pip's wheel cache. index Inspect information available from package indexes. wheel Build wheels from your requirements. hash Compute hashes of package archives. completion A helper command used for command completion. debug Show information useful for debugging. help Show help for commands.

General Options: -h, --help Show help. --debug Let unhandled exceptions propagate outside the main subroutine, instead of logging them to stderr. --isolated Run pip in an isolated mode, ignoring environment variables and user configuration. -v, --verbose Give more output. Option is additive, and can be used up to 3 times. -V, --version Show version and exit. -q, --quiet Give less output. Option is additive, and can be used up to 3 times (corresponding to WARNING, ERROR, and CRITICAL logging levels). --log Path to a verbose appending log. --no-input Disable prompting for input. --proxy Specify a proxy in the form [user:passwd@]proxy.server:port. --retries Maximum number of retries each connection should attempt (default 5 times). --timeout Set the socket timeout (default 15 seconds). --exists-action Default action when a path already exists: (s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort. --trusted-host Mark this host or host:port pair as trusted, even though it does not have valid or any HTTPS. --cert Path to PEM-encoded CA certificate bundle. If provided, overrides the default. See 'SSL Certificate Verification' in pip documentation for more information. --client-cert Path to SSL client certificate, a single file containing the private key and the certificate in PEM format. --cache-dir

Store the cache data in . --no-cache-dir Disable the cache. --disable-pip-version-check Don't periodically check PyPI to determine whether a new version of pip is available for download. Implied with --no-index. --no-color Suppress colored output. --no-python-version-warning Silence deprecation warnings for upcoming unsupported Pythons. --use-feature Enable new functionality, that may be backward incompatible. --use-deprecated Enable deprecated functionality, that will be removed in the future.

[wang-xinyu]

list丢了

python3 -m pip list

[ChihoJack]

lcfc@lcfc-desktop:~/Documents/FastDeploy-develop/examples/vision/ocr/PP-OCRv3/python$ python3 -m pip list Package Version

---

anyio 3.6.2 apturl 0.5.2 asgiref 3.4.1 asn1crypto 0.24.0 astor 0.8.1 attrdict 2.0.1 attrs 17.4.0 Babel 2.11.0 bce-python-sdk 0.8.79 beautifulsoup4 4.6.0 blinker 1.4 Brlapi 0.6.6 cachetools 4.2.4 certifi 2018.1.18 chardet 3.0.4 charset-normalizer 2.0.12 click 8.0.4 contextlib2 21.6.0 contextvars 2.4 cryptography 2.1.4 cssselect 1.1.0 cssutils 2.3.1 cupshelpers 1.0 cycler 0.10.0 Cython 0.29.33 dataclasses 0.8 decorator 4.1.2 defer 1.0.6 dill 0.3.4 distro-info 0.18ubuntu0.18.04.1 Django 2.2.15 et-xmlfile 1.1.0 fastapi 0.83.0 fastdeploy-gpu-python 0.2.1 fastdeploy-tools 0.0.5 feedparser 5.2.1 Flask 2.0.3 Flask-Babel 2.0.0 future 0.18.3 graphsurgeon 0.4.5 h11 0.13.0 html5lib 0.999999999 httplib2 0.9.2 idna 3.4 immutables 0.19 importlib-metadata 4.8.3 importlib-resources 5.4.0 itsdangerous 2.0.1 Jetson.GPIO 2.0.17 Jinja2 3.0.3 joblib 0.11 keyring 10.6.0 keyrings.alt 3.0 language-selector 0.1 lanms-neo 1.0.2 launchpadlib 1.10.6 lazr.restfulclient 0.13.5 lazr.uri 1.0.3 lmdb 1.4.0 louis 3.5.0 lxml 4.2.1 macaroonbakery 1.1.3 Mako 1.0.7 MarkupSafe 2.0.1 matplotlib 2.1.1 multiprocess 0.70.12.2 nose 1.3.7 numpy 1.19.5 oauth 1.0.1 oauthlib 2.0.6 onboard 1.4.1 opencv-contrib-python 4.6.0.66 opencv-python 4.6.0.66 openpyxl 3.1.1 opt-einsum 3.3.0 packaging 21.3 paddle-bfloat 0.1.7 paddle2onnx 1.0.5 paddlepaddle-gpu 2.3.2 pandas 0.22.0 Pillow 8.4.0 pip 21.3.1 pluggy 0.6.0 Polygon3 3.0.9.1 premailer 3.10.0 protobuf 3.19.6 py 1.5.2 pycairo 1.16.2 pyclipper 1.3.0.post4 pycrypto 2.6.1 pycryptodome 3.17 pycups 1.9.73 pydantic 1.9.2 PyGObject 3.26.1 PyJWT 1.5.3 pymacaroons 0.13.0 PyNaCl 1.1.2 pyparsing 2.2.0 pyRFC3339 1.0 pytest 3.3.2 python-apt 1.6.5+ubuntu0.6 python-dateutil 2.6.1 python-debian 0.1.32 python-docx 0.8.10 pytz 2018.3 pyxattr 0.6.0 pyxdg 0.25 PyYAML 3.12 rapidfuzz 2.11.1 requests 2.27.1 requests-unixsocket 0.1.5 scikit-learn 0.19.1 scipy 0.19.1 SecretStorage 2.3.1 setuptools 49.2.1 Shapely 1.8.5.post1 simplejson 3.13.2 six 1.16.0 sniffio 1.2.0 sqlparse 0.3.1 ssh-import-id 5.7 starlette 0.19.1 system-service 0.3 systemd-python 234 tensorrt 8.0.1.6 tqdm 4.64.1 typing_extensions 4.1.1 ubuntu-drivers-common 0.0.0 uff 0.6.9 unity-scope-calculator 0.1 unity-scope-chromiumbookmarks 0.1 unity-scope-colourlovers 0.1 unity-scope-devhelp 0.1 unity-scope-firefoxbookmarks 0.1 unity-scope-manpages 0.1 unity-scope-openclipart 0.1 unity-scope-texdoc 0.1 unity-scope-tomboy 0.1 unity-scope-virtualbox 0.1 unity-scope-yelp 0.1 unity-scope-zotero 0.1 urllib3 1.22 urwid 2.0.1 uvicorn 0.16.0 virtualenv 15.1.0 visualdl 2.4.2 wadllib 1.3.2 webencodings 0.5 Werkzeug 2.0.3 wheel 0.30.0 xkit 0.0.0 youtube_dl 2018.3.14 zipp 3.6.0 zope.interface 4.3.2

[ChihoJack]

我的pip list有问题吗

[wang-xinyu]

这里的fastdeploy-gpu-python 还是0.2.1

python3 -m pip uninstall fastdeploy-gpu-python

再python3 -m pip install dist/xxx.whl

[ChihoJack]

好，我试试

[ChihoJack]

用了这些指令： python3 -m pip uninstall fastdeploy-gpu-python

再python3 -m pip install dist/xxx.whl 之后去PPOCRv3推理结果是这样子的： lcfc@lcfc-desktop:~/Documents/FastDeploy-develop/examples/vision/ocr/PP-OCRv3/python$ OPENBLAS_CORETYPE=ARMV8 python3 test.py --det_model ch_PP-OCRv3_det_infer --cls_model ch_ppocr_mobile_v2.0_cls_infer --rec_model ch_PP-OCRv3_rec_infer --rec_label_file ppocr_keys_v1.txt --image 12.jpg --device gpu WARNING:root:The installed fastdeploy-python package is not built with GPU, will force to use CPU. To use GPU, following the commands to install fastdeploy-gpu-python. WARNING:root: ================= Install GPU FastDeploy=============== WARNING:root: python -m pip uninstall fastdeploy-python WARNING:root: python -m pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html WARNING:root:The installed fastdeploy-python package is not built with GPU, will force to use CPU. To use GPU, following the commands to install fastdeploy-gpu-python. WARNING:root: ================= Install GPU FastDeploy=============== WARNING:root: python -m pip uninstall fastdeploy-python WARNING:root: python -m pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html WARNING:root:The installed fastdeploy-python package is not built with GPU, will force to use CPU. To use GPU, following the commands to install fastdeploy-gpu-python. WARNING:root: ================= Install GPU FastDeploy=============== WARNING:root: python -m pip uninstall fastdeploy-python WARNING:root: python -m pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html /usr/lib/python3/dist-packages/apport/report.py:13: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import fnmatch, glob, traceback, errno, sys, atexit, locale, imp, stat Traceback (most recent call last): File "test.py", line 188, in det_model_file, det_params_file, runtime_option=det_option) File "/home/lcfc/.local/lib/python3.6/site-packages/fastdeploy/vision/ocr/ppocr/init.py", line 218, in init self._model = C.vision.ocr.DBDetector( AttributeError: module 'fastdeploy.c_lib_wrap' has no attribute 'vision'

[wang-xinyu]

再python3 -m pip list看下呢

[ChihoJack]

lcfc@lcfc-desktop:~/Documents/FastDeploy-develop/examples/vision/ocr/PP-OCRv3/python$ python3 -m pip list Package Version

---

anyio 3.6.2 apturl 0.5.2 asgiref 3.4.1 asn1crypto 0.24.0 astor 0.8.1 attrdict 2.0.1 attrs 17.4.0 Babel 2.11.0 bce-python-sdk 0.8.79 beautifulsoup4 4.6.0 blinker 1.4 Brlapi 0.6.6 cachetools 4.2.4 certifi 2018.1.18 chardet 3.0.4 charset-normalizer 2.0.12 click 8.0.4 contextlib2 21.6.0 contextvars 2.4 cryptography 2.1.4 cssselect 1.1.0 cssutils 2.3.1 cupshelpers 1.0 cycler 0.10.0 Cython 0.29.33 dataclasses 0.8 decorator 4.1.2 defer 1.0.6 dill 0.3.4 distro-info 0.18ubuntu0.18.04.1 Django 2.2.15 et-xmlfile 1.1.0 fastapi 0.83.0 fastdeploy-python 0.0.0 fastdeploy-tools 0.0.5 feedparser 5.2.1 Flask 2.0.3 Flask-Babel 2.0.0 future 0.18.3 graphsurgeon 0.4.5 h11 0.13.0 html5lib 0.999999999 httplib2 0.9.2 idna 3.4 immutables 0.19 importlib-metadata 4.8.3 importlib-resources 5.4.0 itsdangerous 2.0.1 Jetson.GPIO 2.0.17 Jinja2 3.0.3 joblib 0.11 keyring 10.6.0 keyrings.alt 3.0 language-selector 0.1 lanms-neo 1.0.2 launchpadlib 1.10.6 lazr.restfulclient 0.13.5 lazr.uri 1.0.3 lmdb 1.4.0 louis 3.5.0 lxml 4.2.1 macaroonbakery 1.1.3 Mako 1.0.7 MarkupSafe 2.0.1 matplotlib 2.1.1 multiprocess 0.70.12.2 nose 1.3.7 numpy 1.19.5 oauth 1.0.1 oauthlib 2.0.6 onboard 1.4.1 opencv-contrib-python 4.6.0.66 opencv-python 4.6.0.66 openpyxl 3.1.1 opt-einsum 3.3.0 packaging 21.3 paddle-bfloat 0.1.7 paddle2onnx 1.0.5 paddlepaddle-gpu 2.3.2 pandas 0.22.0 Pillow 8.4.0 pip 21.3.1 pluggy 0.6.0 Polygon3 3.0.9.1 premailer 3.10.0 protobuf 3.19.6 py 1.5.2 pycairo 1.16.2 pyclipper 1.3.0.post4 pycrypto 2.6.1 pycryptodome 3.17 pycups 1.9.73 pydantic 1.9.2 PyGObject 3.26.1 PyJWT 1.5.3 pymacaroons 0.13.0 PyNaCl 1.1.2 pyparsing 2.2.0 pyRFC3339 1.0 pytest 3.3.2 python-apt 1.6.5+ubuntu0.6 python-dateutil 2.6.1 python-debian 0.1.32 python-docx 0.8.10 pytz 2018.3 pyxattr 0.6.0 pyxdg 0.25 PyYAML 3.12 rapidfuzz 2.11.1 requests 2.27.1 requests-unixsocket 0.1.5 scikit-learn 0.19.1 scipy 0.19.1 SecretStorage 2.3.1 setuptools 49.2.1 Shapely 1.8.5.post1 simplejson 3.13.2 six 1.16.0 sniffio 1.2.0 sqlparse 0.3.1 ssh-import-id 5.7 starlette 0.19.1 system-service 0.3 systemd-python 234 tensorrt 8.0.1.6 tqdm 4.64.1 typing_extensions 4.1.1 ubuntu-drivers-common 0.0.0 uff 0.6.9 unity-scope-calculator 0.1 unity-scope-chromiumbookmarks 0.1 unity-scope-colourlovers 0.1 unity-scope-devhelp 0.1 unity-scope-firefoxbookmarks 0.1 unity-scope-manpages 0.1 unity-scope-openclipart 0.1 unity-scope-texdoc 0.1 unity-scope-tomboy 0.1 unity-scope-virtualbox 0.1 unity-scope-yelp 0.1 unity-scope-zotero 0.1 urllib3 1.22 urwid 2.0.1 uvicorn 0.16.0 virtualenv 15.1.0 visualdl 2.4.2 wadllib 1.3.2 webencodings 0.5 Werkzeug 2.0.3 wheel 0.30.0 xkit 0.0.0 youtube_dl 2018.3.14 zipp 3.6.0 zope.interface 4.3.2

[wang-xinyu]

fastdeploy-python是CPU的，

这个命令具体执行的命令发一下python3 -m pip install dist/xxx.whl

另外看一下dist/目录

[ChihoJack]

dist目录下只有这个文件： fastdeploy_python-0.0.0-cp36-cp36m-linux_aarch64.whl 执行命令为： python3 -m pip instal fastdeploy_python-0.0.0-cp36-cp36m-linux_aarch64.whl

[ChihoJack]

dist目录下只有这个文件： fastdeploy_python-0.0.0-cp36-cp36m-linux_aarch64.whl 执行命令为： python3 -m pip install fastdeploy_python-0.0.0-cp36-cp36m-linux_aarch64.whl

[wang-xinyu]

编译的包不对，如果是gpu的应该是fastdeploy_gpu_python-0.0.0-cp36-cp36m-linux_aarch64.whl

删除FastDeploy/python目录下的build和.setuptools-cmake-build两个子目录后再重新编译试试吧，重新export那几个环境变量

[ChihoJack]

好

[ChihoJack]

按照你做完给我的方法，我重新编译了，但是还是没有生成fastdeploy_gpu_python-0.0.0-cp36-cp36m-linux_aarch64.whl，还是生成了fastdeploy_python-0.0.0-cp36-cp36m-linux_aarch64.whl，之后运行还是一样。

[ChihoJack]

按照你昨晚给我的方法，我重新编译了，但是还是没有生成fastdeploy_gpu_python-0.0.0-cp36-cp36m-linux_aarch64.whl，还是生成了fastdeploy_python-0.0.0-cp36-cp36m-linux_aarch64.whl，之后运行还是一样。

[wang-xinyu]

我刚编译了一下，是生成的fastdeploy_gpu_python，把完整的编译log通过附件发一下吧

[ChihoJack]

就是我编译一次，然后把log输出出来发出来是吧？

[wang-xinyu]

对

[ChihoJack]

好的，稍等

[ChihoJack]

log.txt 这次编译到一半失败了，可以先给你看看这部分log

[ChihoJack]

log.txt log2.txt 这是编译成功后的log，你看看还有什么问题不，编译之后生成的还是fastdeploy_python-0.0.0-cp36-cp36m-linux_aarch64.whl

[wang-xinyu]

看起来这两个环境变量的设置，都没有生效， export BUILD_ON_JETSON=ON export ENABLE_VISION=ON

[ChihoJack]

那该如何解决呢，我已经按照文档说的将这两个环境变量设置了

[wang-xinyu]

echo $BUILD_ON_JETSON 看一下

[ChihoJack]

还是一样，生成的还是fastdeploy_python-0.0.0-cp36-cp36m-linux_aarch64.whl

[jiangjiajun]

@ChihoJack 建议不用安装wheel文件了，推测你的python环境可能设置上有些不同，你按如下方式来使用吧。

• 1. 正常进入FastDeploy/python目录编译，包括执行python setup.py build和python setup.py bdist_wheel两个命令
• 2. 在FastDeploy/python目录下，执行命令export PYTHONPATH=${PWD}，接下来再尝试（注意export变量仅在当前打开的命令终端生效）

[ChihoJack]

好的，我现在去试试

[ChihoJack]

刚刚试了，生成的还是fastdeploy_python-0.0.0-cp36-cp36m-linux_aarch64.whl

[jiangjiajun]

无需安装任何whl文件，按照如下步骤执行即可

• 1. 正常进入FastDeploy/python目录编译，包括执行python setup.py build和python setup.py bdist_wheel两个命令
• 2. 无需安装whl，而是在在FastDeploy/python目录下，执行命令export PYTHONPATH=${PWD}，接下来再尝试（注意export变量仅在当前打开的命令终端生效）使用（例如打开python，执行import fastdeploy，或者是python my_script.py，注意是需要在同一个命令终端下）

[ChihoJack]

好，我再试试

[ChihoJack]

我用了你这个方法，确实可以import fastdeploy，但是我之后查看了pip list，发现还是自动安装上了fastdeploy-python而不是fastdeploy-gpu-python

[jiangjiajun]

export PYTHONPATH之后，pip是否有安装已经不再重要。 因为python会优先使用PYTHONPATH下的fastdeploy

[ChihoJack]

我就是用pythonpath的，然后还是发现不是gpu版本的

[ChihoJack]

还有其他什么解决办法吗？