ARM-software / armnn

Arm NN ML Software. The code here is a read-only mirror of https://review.mlplatform.org/admin/repos/ml/armnn
https://developer.arm.com/products/processors/machine-learning/arm-nn
MIT License
1.14k stars 307 forks source link

CMake Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) #690

Closed matheusfalcaopinto closed 1 year ago

matheusfalcaopinto commented 1 year ago

Hi! I'm building ArmNN and pyArmNN with the build-tool scripts. When I run ./build-armnn.sh --target-arch=aarch64 --all --neon-backend --cl-backend --armnn-cmake-args='-DBUILD_PYTHON_SRC=1 -DBUILD_PYTHON_WHL=1' everything works but when it starts Building Arm NN for aarch64 it throws me this error:

CMake Error at /usr/local/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message): Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) (Required is at least version "3") Call Stack (most recent call first): /usr/local/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE) /usr/local/share/cmake-3.16/Modules/FindPythonLibs.cmake:310 (FIND_PACKAGE_HANDLE_STANDARD_ARGS) cmake/GlobalConfig.cmake:407 (find_package) CMakeLists.txt:20 (include)

I already tried to set these two variables manully but it doesn't work. Any helpers? Thanks!

james-conroy-arm commented 1 year ago

Hi @matheusfalcaopinto ,

Many thanks for raising this issue. The Arm NN Build Tool was released without PyArmNN support and we recommend Python developers to use the TF Lite Delegate Python API to ensure the widest support of ML operators in Arm NN. There is a Python-based TF Lite Delegate quick start guide here: https://github.com/ARM-software/armnn/blob/HEAD/delegate/DelegateQuickStartGuide.md

The Build Tool can still be used to build PyArmNN with some modifications. The install-packages.sh script supplied with the build-tool does not install the PyArmNN specific packages which are documented here: https://github.com/ARM-software/armnn/tree/branches/armnn_22_08/python/pyarmnn#setup-development-environment

In particular, from the error message it looks like your system does not have Python 3.X installed (at least 3.6 is required). Please refer to the above link for more information.

Could you also answer these questions which will help us with your query:

  1. Host machine arch (x86_64 / aarch64) and OS
  2. Using build-tool with or without Docker?
  3. Are you using a virtual environment (venv) to handle Python dependencies (refer to second link above)

Thanks, James

matheusfalcaopinto commented 1 year ago

Hi! Thanks for the reply!

My machine is aarch64, I'm building without docker and not using any virtual env.

I have python3.9 installed and I tried setting the variables manually and passing as parameters in the cmake command with no success. I've been able to build without the pyarmnn

PYTHON_LIBRARY='/usr/lib/python3.9/config-3.9-aarch64-linux-gnu/libpython3.9.so' PYTHON_INCLUDE_DIR='/usr/lib/python3.9'

james-conroy-arm commented 1 year ago

Hi,

Do you have the python3.9-dev apt package installed (python3.6-dev is recommended in the PyArmNN guide).

James

matheusfalcaopinto commented 1 year ago

Running apt install python3.9-dev give me E: Couldn't find any package by glob 'python3.9-dev'. I read today that the -dev packages now have been merged with the python package. I don't know.

Now I'm trying to install pyarmnn following instructions for "Standalone build" from: https://git.mlplatform.org/ml/armnn.git/tree/python/pyarmnn/README.md

But now I can't install SWAG because it needs PCRE2 and again the apt don't have the packages anymore (i think)

james-conroy-arm commented 1 year ago

Please refer to this StackOverflow issue which may help with your original approach: https://stackoverflow.com/questions/24174394/cmake-is-not-able-to-find-python-libraries

Which OS distribution are you using?

matheusfalcaopinto commented 1 year ago

I tried everything in this post already.

I'm using an Orange pi 4 LTS with Debian GNU/Linux 11 (bullseye).

ArmRyan commented 1 year ago

Hey, I think PYTHON_INCLUDE_DIR is looking for something like /usr/include/python3.9/

libpcre2 should be available on bullseye with this package libpcre2-dev (10.36-2) New Perl Compatible Regular Expression Library - development files

As for swig, Swig 4.0 might be already included in the debian build, otherwise it should be available already: swig (4.0.2-1) Generate scripting interfaces to C/C++ code

Can you check /usr/bin/ for swig executable and pcre2-config?

Thanks, Ryan

matheusfalcaopinto commented 1 year ago

hi @ArmRyan! I just figure out that my debian had trouble looking for some apt pakages (missing some sources). I manage to install the debian bullseye source pakage for armnn and pyarmnn with aptitude install command. After that I manage do run the example for object detection using pyarmnn and mobilenetv1 but I had a few problems, the system was missing some openCL libraries first and after installing these libraries now I have Segmentation fault when trying to delegate to GPU (mali-T860). It runs perfectly on CpuAcc and CpuRef (the detections are garbage and the labels are wrong but it's a start!)

Part of the trouble that I had was this missing sources for apt. Now i manage to install pcre2 and swag.

Thanks!

`

FrancisMurtagh-arm commented 1 year ago

Hi @matheusfalcaopinto,

What were your commands for "installing the debian bullseye source pakage for armnn and pyarmnn with aptitude"? Out of interest.

Were you using the Debian packages from the archive? https://packages.debian.org/source/bullseye/armnn

These packages are quite old fwiw.

Thanks, Francis.

matheusfalcaopinto commented 1 year ago

@FrancisMurtagh-arm Hi! Yes, I istalled these packges form https://packages.debian.org/source/bullseye/armnn. I just installed all of them:

aptitude install libarmnn22 libarmnn22:armhf python3-pyarmnn python3-pyarmnn:armhf libarmnntfliteparser22 libarmnntfliteparser22:armhf libarmnntfliteparser-dev libarmnntfliteparser-dev:armhf libarmnnaclcommon22 libarmnnaclcommon22:armhf libarmnn-dev libarmnn-dev:armhf libarmnn-cpuref-backend22 libarmnn-cpuref-backend22:armhf libarmnn-gpuacc-backend22 libarmnn-gpuacc-backend22:armhf libarmnn-cpuacc-backend22 libarmnn-cpuacc-backend22:armhf

and

aptitude install python3-pyarmnn

I've been trying to build armnn for the past 2 weeks now and every path that I follow lead me to errors that I cannot find the solution. My best attempt so far was using the build-tool scripts (could not build with pyarmnn) and now these packages (works more or less). Now with the help with this post I will try to build again from the build-tool scripts.

FrancisMurtagh-arm commented 1 year ago

@matheusfalcaopinto,

If you are able to use Ubuntu instead we have a PPA (Personal Package Archive) which will let you download the newest version of ArmNN by following this guide:

https://github.com/ARM-software/armnn/blob/branches/armnn_22_08/InstallationViaAptRepository.md

sudo apt install software-properties-common sudo add-apt-repository ppa:armnn/ppa sudo apt update sudo apt-get install -y python3-pyarmnn armnn-latest-all

Although I understand it Ubuntu might not suit your needs.

The build tool on the other hand does have support for the TfLite Delegate (which packages don't) this is our preferred route rather than PyArmNN.

Regards, Francis.

matheusfalcaopinto commented 1 year ago

@FrancisMurtagh-arm I might as well try Ubuntu, yes. My first choice was PyarmNN but I got the same seg fault when delegating to GpuAcc as in Tflite Delegate.