search

larq / compute-engine

Highly optimized inference engine for Binarized Neural Networks
https://docs.larq.dev/compute-engine
Apache License 2.0
243 stars 35 forks source link

Building aarch64 compatible pip package #653

Closed utexasnew closed 3 years ago

utexasnew commented 3 years ago

I have followed the instructions located in the docs: https://docs.larq.dev/compute-engine/build/docker/ to build a manylinux2010 compatible pip package.

My question is now how to build the pip package for aarch64 systems? I have tried a simple addition of --config=aarch64 to the bazel build command but that was unsuccessful. Any guidance will surely be appreciated.

Tombana commented 3 years ago

Hi @utexasnew ,

We currently don't support aarch64 builds of our converter package. In theory, adding --config=aarch64 to bazel build build_pip_pkg should do the trick, but when I try to do this locally then I get compile errors for some llvm-related files that are part of the tensorflow converter. This is not something we can easily fix because these errors are not caused by 'our' files.

If you are only looking to compile the python interpreter tool (and not the converter) then you might be able to get it to work by removing these lines:

https://github.com/larq/compute-engine/blob/48bb86bf9ed66d7459f4488d9f44ca236e40091f/larq_compute_engine/BUILD#L24

https://github.com/larq/compute-engine/blob/48bb86bf9ed66d7459f4488d9f44ca236e40091f/larq_compute_engine/__init__.py#L1

and remove "convert_saved_model" from this line: https://github.com/larq/compute-engine/blob/48bb86bf9ed66d7459f4488d9f44ca236e40091f/larq_compute_engine/__init__.py#L12

Finally, remove the ${TMPDIR}/larq_compute_engine/mlir/*.so part of both of these lines (but leave the ${TMPDIR}/larq_compute_engine/tflite/python/*.so part):

https://github.com/larq/compute-engine/blob/48bb86bf9ed66d7459f4488d9f44ca236e40091f/build_pip_pkg.sh#L80-L81

This way you should get a package that only contains the python interpreter without the converter.

When running the build_pip_pkg script (after bazel has completed) you have to add --plat=linux_aarch64 as an argument:

bazel-bin/build_pip_pkg artifacts --plat=linux_aarch64

You might have to update the pip packages setuptools and/or wheel to the latest version to get support for this --plat flag.

GhaziXX commented 3 years ago

Hi @Tombana

I tried to cross compile the python interpreter tool to work on raspberrypi 4 running on ubuntu 20.04 (aarch64) but i am always getting errors.

os used for compilation : ubuntu 20.04.2 LTS (x86_64) target os : raspberrypi 4 running ubuntu 20.04.2 (aarch64) python version: 3.8.10 type of error : fatal error: aarch64-linux-gnu/python3.8/pyconfig.h: No such file or directory

Btw i did all the steps that you have mentioned and always the same error. I will be grateful if you or anyone else can help me solve this or maybe just share the wheel file for aarch64. I just need it for inference.

CNugteren commented 3 years ago

You can try to install a package on your system that provides that missing Python header file. You can use apt-file for that (see https://askubuntu.com/a/1912 for instructions) and then install it (e.g. `apt install libpython3.8-dev), e.g.:

$ apt-file search pyconfig.h
libpython2.7-dbg: /usr/include/aarch64-linux-gnu/python2.7_d/pyconfig.h
libpython2.7-dbg: /usr/include/python2.7_d/pyconfig.h
libpython2.7-dev: /usr/include/aarch64-linux-gnu/python2.7/pyconfig.h
libpython2.7-dev: /usr/include/python2.7/pyconfig.h
libpython3.7-dbg: /usr/include/aarch64-linux-gnu/python3.7dm/pyconfig.h
libpython3.7-dbg: /usr/include/python3.7dm/pyconfig.h
libpython3.7-dev: /usr/include/aarch64-linux-gnu/python3.7m/pyconfig.h
libpython3.7-dev: /usr/include/python3.7m/pyconfig.h
(...)

However it seems on my test system there is no such Python 3.8 package available. If you can't manage to install the required package, you can try to manually download the file (e.g. from here) and place it in the right location, but I'm not sure if you'll run into other errors later on then.

GhaziXX commented 3 years ago

Thank you @CNugteren, I ended up installing the same python version on the raspberry pi and then I copied the aarch64-linux-gnu folder to the Linux computer. By doing that, I was able to build successfully. Unfortunately, when I tried to build the pip package then, I came across an error, I really can't identify it's origins. I don't know if it is something related to the build or something else. Here is the log: Build log:

$ bazel build :build_pip_pkg --config=aarch64
Target //:build_pip_pkg up-to-date:
bazel-bin/build_pip_pkg
INFO: Elapsed time: 630.677s, Critical Path: 122.83s
INFO: 499 processes: 179 internal, 320 local.
INFO: Build completed successfully, 499 total actions

pip Package build:

$ bazel-bin/build_pip_pkg artifacts
++ uname -s
++ tr A-Z a-z
+ PLATFORM=linux
+ is_windows
+ [[ linux =~ msys_nt*|mingw*|cygwin*|uwin* ]]
+ PIP_FILE_PREFIX=bazel-bin/build_pip_pkg.runfiles/larq_compute_engine/
+ main artifacts
+ DEST=artifacts
+ BUILD_FLAG=
+ [[ -z artifacts ]]
+ mkdir -p artifacts
++ abspath artifacts
+++ dirname artifacts
++ cd .
+++ basename artifacts
++ echo /home/gtounsi/Desktop/compute-engine/artifacts
++ cd /home/gtounsi/Desktop/compute-engine
+ DEST=/home/gtounsi/Desktop/compute-engine/artifacts
+ echo '=== destination directory: /home/gtounsi/Desktop/compute-engine/artifacts'
=== destination directory: /home/gtounsi/Desktop/compute-engine/artifacts
++ mktemp -d -t tmp.XXXXXXXXXX
+ TMPDIR=/tmp/tmp.UyTI0WPySF
++ date
+ echo Wed 18 Aug 2021 09:17:22 AM CET : '=== Using tmpdir: /tmp/tmp.UyTI0WPySF'
Wed 18 Aug 2021 09:17:22 AM CET : === Using tmpdir: /tmp/tmp.UyTI0WPySF
+ echo '=== Copy Larq Compute Engine files'
=== Copy Larq Compute Engine files
+ cp bazel-bin/build_pip_pkg.runfiles/larq_compute_engine/setup.py /tmp/tmp.UyTI0WPySF
+ cp bazel-bin/build_pip_pkg.runfiles/larq_compute_engine/MANIFEST.in /tmp/tmp.UyTI0WPySF
+ cp bazel-bin/build_pip_pkg.runfiles/larq_compute_engine/README.md /tmp/tmp.UyTI0WPySF
+ cp bazel-bin/build_pip_pkg.runfiles/larq_compute_engine/LICENSE /tmp/tmp.UyTI0WPySF
+ is_linux
+ [[ linux == \l\i\n\u\x ]]
+ touch /tmp/tmp.UyTI0WPySF/stub.cc
+ is_windows
+ [[ linux =~ msys_nt*|mingw*|cygwin*|uwin* ]]
+ rsync -avm -L '--exclude=*_test.py' bazel-bin/build_pip_pkg.runfiles/larq_compute_engine/larq_compute_engine 
/tmp/tmp.UyTI0WPySF
building file list ... done
larq_compute_engine/
larq_compute_engine/__init__.py
larq_compute_engine/mlir/
larq_compute_engine/mlir/__init__.py
larq_compute_engine/mlir/python/
larq_compute_engine/mlir/python/__init__.py
larq_compute_engine/tflite/
larq_compute_engine/tflite/__init__.py
larq_compute_engine/tflite/python/
larq_compute_engine/tflite/python/__init__.py
larq_compute_engine/tflite/python/interpreter.py
larq_compute_engine/tflite/python/interpreter_base.py
larq_compute_engine/tflite/python/interpreter_wrapper_lite.so

sent 2,520,971 bytes  received 183 bytes  5,042,308.00 bytes/sec
total size is 2,519,573  speedup is 1.00
+ pushd /tmp/tmp.UyTI0WPySF
/tmp/tmp.UyTI0WPySF ~/Desktop/compute-engine
+ is_windows
+ [[ linux =~ msys_nt*|mingw*|cygwin*|uwin* ]]
+ echo '=== Stripping symbols'
=== Stripping symbols
+ chmod +w '/tmp/tmp.UyTI0WPySF/larq_compute_engine/mlir/*.so' 
/tmp/tmp.UyTI0WPySF/larq_compute_engine/tflite/python/interpreter_wrapper_lite.so
chmod: cannot access '/tmp/tmp.UyTI0WPySF/larq_compute_engine/mlir/*.so': No such file or directory
Tombana commented 3 years ago

Here is the log:

pip Package build:

$ bazel-bin/build_pip_pkg artifacts
...
chmod: cannot access '/tmp/tmp.UyTI0WPySF/larq_compute_engine/mlir/*.so': No such file or directory

On line 80 and 81 of build_pip_pkg.sh (in the root of the repository), there is this:

    chmod +w ${TMPDIR}/larq_compute_engine/mlir/*.so ${TMPDIR}/larq_compute_engine/tflite/python/*.so
    strip -x ${TMPDIR}/larq_compute_engine/mlir/*.so ${TMPDIR}/larq_compute_engine/tflite/python/*.so

Could you try to remove the mlir/*.so parts:

    chmod +w ${TMPDIR}/larq_compute_engine/tflite/python/*.so
    strip -x ${TMPDIR}/larq_compute_engine/tflite/python/*.so

I'll update the original answer with these instructions.

GhaziXX commented 3 years ago

Here is the log: pip Package build:

$ bazel-bin/build_pip_pkg artifacts
...
chmod: cannot access '/tmp/tmp.UyTI0WPySF/larq_compute_engine/mlir/*.so': No such file or directory

On line 80 and 81 of build_pip_pkg.sh (in the root of the repository), there is this:

    chmod +w ${TMPDIR}/larq_compute_engine/mlir/*.so ${TMPDIR}/larq_compute_engine/tflite/python/*.so
    strip -x ${TMPDIR}/larq_compute_engine/mlir/*.so ${TMPDIR}/larq_compute_engine/tflite/python/*.so

Could you try to remove the mlir/*.so parts:

    chmod +w ${TMPDIR}/larq_compute_engine/tflite/python/*.so
    strip -x ${TMPDIR}/larq_compute_engine/tflite/python/*.so

I'll update the original answer with these instructions.

That finally worked, I am able to build the .whl file now. But another problem happened (I don't why it going on with me really :'( ) when I tried to install the file on the raspberry pi I got this error: ERROR: larq_compute_engine-0.6.1-cp38-cp38-linux_x86_64.whl is not a supported wheel on this platform. and I am sure that I used the --config=aarch64 when running Bazel build build_pip_package. Thank you very much @Tombana

Tombana commented 3 years ago

ERROR: larq_compute_engine-0.6.1-cp38-cp38-linux_x86_64.whl is not a supported wheel on this platform. and I am sure that I used the --config=aarch64 when running Bazel build build_pip_package. Thank you very much @Tombana

Can you try to add the following --plat flag to build_pip_pkg?

bazel-bin/build_pip_pkg artifacts --plat=linux_aarch64

I'll also add this to the instructions in the original answer.

GhaziXX commented 3 years ago

ERROR: larq_compute_engine-0.6.1-cp38-cp38-linux_x86_64.whl is not a supported wheel on this platform. and I am sure that I used the --config=aarch64 when running Bazel build build_pip_package. Thank you very much @Tombana

Can you try to add the following --plat flag to build_pip_pkg?

bazel-bin/build_pip_pkg artifacts --plat=linux_aarch64

I'll also add this to the instructions in the original answer.

That finally worked!! Thank you very much for your help and time

chartain02 commented 3 months ago

I am getting this issue even after removinbg .mlir lines from build_pip_pkg.sh file. can someone help me with this?

Tombana commented 3 months ago

I think it's using the strip tool from your host system instead of the crosscompiler toolchain. Can you try removing that strip -x line from build_pip_pkg.sh?

chartain02 commented 3 months ago

I think it's using the strip tool from your host system instead of the crosscompiler toolchain. Can you try removing that strip -x line from build_pip_pkg.sh?

yeah thats working now but as per my observation from other issue....cant we perform inference using this pip package in Raspbian OS? do we have to install other os?

chartain02 commented 3 months ago

Hello, can someone tell me if there is any way to benchmark the lce model on int8? when we use provided executable then is it for float 32 or int8? and is its float32, which i assume looking at the padding being multiples of 32 then how can we benchmark for float32? Thank in advance @Tombana

Tombana commented 3 months ago

The benchmark executable should support both float and int8 layers.