Compiled libqi-python for Jetson TX2

[robertanto]

Dear maintainers,

I read in a ArXiv paper from SoftBank Robotics Europe that the library has been already compiled for Jetson TX2.

May you share a link to download the compiled library?

Thanks in advance.

SOLUTION:

I have created a repository containing a faster way to use the library: https://github.com/robertanto/libqi-python-nvidia-jetson

It also works with the latest JetPack (4.4.1).

I hope that this repository can help researcher in their activities.

[mbusy]

Hi, if you want to reproduce the setup described in the paper, you will need to compile both libqi and libqi-python for the TX2. To do so, please follow this tutorial:

libqi & libqi-python installation instructions

Clone the libqi repository and the libqi-python repositories on the Jetson (in a qi_ws folder for instance, or wherever you want basically).

# From the Jetson:

# OPTIONAL: If you want to use qibuild
sudo pip install qibuild

# We will use a qi_ws folder in this tutorial, but you can clone the repositories wherever you want
mkdir qi_ws && cd qi_ws

# Clone the repos
git clone git@gitlab.aldebaran.lan:qi/libqi.git
git clone git@gitlab.aldebaran.lan:qi/libqi-python.git

# checkout the release-2.5 branch
cd libqi && git checkout release-2.5
cd ../libqi-python && git checkout release-2.5

cd ..

Then, download and copy the patches libqi-python-2.5_jetson.patch and libqi-2.5_jetson.patch on the Jetson. Respectively apply the patches to the repositories.

# From your computer:
scp libqi-2.5_jetson.patch nvidia@tegra-ubuntu.local:
scp libqi-python-2.5_jetson.patch nvidia@tegra-ubuntu.local:

# From the Jetson:
cd qi_ws/libqi && git apply path/to/patch/libqi-2.5_jetson.patch
cd ../libqi-python && git apply path/to/patch/libqi-python-2.5_jetson.patch

Once the patches are applied, build and install the projects.

# From the Jetson:
cd qi_ws/libqi
./compile.sh
sudo ./install.sh

cd ../libqi-python
./compile.sh
sudo ./install.sh

[robertanto]

Sorry for the late response. First of all, thank you @mbusy.

I cannot install it on my Jetson TX2 . I have installed the JetPack 4.4 (Ubuntu 18.04.5 LTS), is this setup compatible?

[mbusy]

@robertanto you're welcome. The setup isn't compatible, we used JetPack 3.3 (Ubuntu 16.04). The Boost versions (Boost used by Ubuntu 16.04 vs 18.04) being different, the patch that I provided won't work for your setup

[robertanto]

@mbusy thank you again!

I will install the same jetpack version and I will let you know.

[robertanto]

I compiled libqi.

Trying to compiling libqi-python, cmake asks me for QIPYTHON_DIR variable.

[mbusy]

Strange... Could you paste the full error log ?

[robertanto]

I solved adding to the cmake command -DWITH_BOOST_PYTHON. Now I have compiled both libqi and libqi-python for arm but when I try to import qi this is the message:

File` "/usr/lib/python2.7/dist-packages/qi/init.py", line 88, in from _qi import Application as _Application ImportError: dynamic module does not define init function (init_qi)

[robertanto]

I see that this is an open issue. Refer to #2 .

[nyibbang]

Following the discussion on #2, it seems you're using the qi module installed from pip, although you said you couldn't install it for your target platform (arm). Are you using the Python interpreter from a x86 host ?

[robertanto]

The qi package has been moved in the dist-packages folder due to the install.sh script contained in the patch.

[robertanto]

Are you using the Python interpreter from a x86 host ?

I am running the docker container directly on the Jetson.

[nyibbang]

Okay, so your _qi.so file should be located at /usr/lib/python2.7/dist-packages/_qi.so right ? is it the one you ran readelf on ? if not, can you check that one ?

Can you also check that libqi.so was installed in /usr/lib/, and check that all the dependencies of _qi.so are found by running ldd _qi.so ?

[robertanto]

It seems all ok.

root@c0598afa3794:/usr/lib/python2.7/dist-packages# readelf -s _qi.so | grep init
    60: 0000000000001a98     0 FUNC    GLOBAL DEFAULT    9 _init
    53: 0000000000014a48     0 OBJECT  LOCAL  DEFAULT   17 __frame_dummy_init_array_
    81: 0000000000003138     1 OBJECT  LOCAL  DEFAULT   13 _ZN5boostL17initialized_v
   111: 0000000000015158     1 OBJECT  LOCAL  DEFAULT   25 _ZStL8__ioinit
   130: 00000000000032d8    16 OBJECT  LOCAL  DEFAULT   13 _ZZ15init_module__qivE12_
   166: 0000000000002038   416 FUNC    LOCAL  DEFAULT   11 _Z41__static_initializati
   207: 0000000000001e48   496 FUNC    LOCAL  DEFAULT   11 _Z15init_module__qiv
   275: 0000000000001e20    40 FUNC    LOCAL  DEFAULT   11 init_qi
   329: 0000000000001a98     0 FUNC    GLOBAL DEFAULT    9 _init

root@c0598afa3794:/usr/lib/python2.7/dist-packages# ldd _qi.so 
 linux-vdso.so.1 =>  (0x0000007faf84c000)
 libpython2.7.so.1.0 => /usr/lib/aarch64-linux-gnu/libpython2.7.so.1.0 (0x0000007faf4b5000)
 libqipython.so => /usr/lib/libqipython.so (0x0000007faf242000)
 libboost_python.so.1.64.0 => /usr/local/lib/libboost_python.so.1.64.0 (0x0000007faf1de000)
 libqi.so => /usr/lib/libqi.so (0x0000007faded4000)
 libboost_system.so.1.64.0 => /usr/local/lib/libboost_system.so.1.64.0 (0x0000007fadec0000)
 libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007fadd31000)
 libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007fadd10000)
 libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007fadbca000)
 /lib/ld-linux-aarch64.so.1 (0x0000007faf821000)
 libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007fadb9e000)
 libz.so.1 => /lib/aarch64-linux-gnu/libz.so.1 (0x0000007fadb77000)
 libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007fadb64000)
 libutil.so.1 => /lib/aarch64-linux-gnu/libutil.so.1 (0x0000007fadb51000)
 libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007fadaa4000)
 libboost_thread.so.1.64.0 => /usr/local/lib/libboost_thread.so.1.64.0 (0x0000007fada6c000)
 libboost_filesystem.so.1.64.0 => /usr/local/lib/libboost_filesystem.so.1.64.0 (0x0000007fada42000)
 libboost_locale.so.1.64.0 => /usr/local/lib/libboost_locale.so.1.64.0 (0x0000007fad990000)
 libboost_chrono.so.1.64.0 => /usr/local/lib/libboost_chrono.so.1.64.0 (0x0000007fad978000)
 libboost_regex.so.1.64.0 => /usr/local/lib/libboost_regex.so.1.64.0 (0x0000007fad87a000)
 libboost_program_options.so.1.64.0 => /usr/local/lib/libboost_program_options.so.1.64.0 (0x0000007fad7f0000)
 libssl.so.1.0.0 => /lib/aarch64-linux-gnu/libssl.so.1.0.0 (0x0000007fad787000)
 libcrypto.so.1.0.0 => /lib/aarch64-linux-gnu/libcrypto.so.1.0.0 (0x0000007fad5e3000)

[nyibbang]

Can you try to run strace -e trace=file python -c "import qi" 2>strace.log and give us the content of grep _qi.so strace.log ?

[robertanto]

root@c0598afa3794:/# strace -e trace=file python -c "import qi" 2>strace.log
root@c0598afa3794:/# grep _qi.so strace.log 
openat(AT_FDCWD, "_qi.so", O_RDONLY)    = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/python2.7/_qi.so", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/python2.7/plat-aarch64-linux-gnu/_qi.so", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/python2.7/lib-tk/_qi.so", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/python2.7/lib-old/_qi.so", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/python2.7/lib-dynload/_qi.so", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/local/lib/python2.7/dist-packages/_qi.so", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/python2.7/dist-packages/_qi.so", O_RDONLY) = 4
openat(AT_FDCWD, "/usr/lib/python2.7/dist-packages/qi/_qi.so", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "_qi.so", O_RDONLY)    = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/python2.7/_qi.so", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/python2.7/plat-aarch64-linux-gnu/_qi.so", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/python2.7/lib-tk/_qi.so", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/python2.7/lib-dynload/_qi.so", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/local/lib/python2.7/dist-packages/_qi.so", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/python2.7/dist-packages/_qi.so", O_RDONLY) = 4
openat(AT_FDCWD, "/usr/lib/python2.7/dist-packages/_qi.so", O_RDONLY|O_CLOEXEC) = 5

[nyibbang]

Everything seems good, I'm sorry, I'm out of ideas at the moment. I'll come back to you with more things to check to investigate this issue.

[robertanto]

@nyibbang thank you for your time! I will try by using directly the Jetpack 3.3 as done by @mbusy.

A solution may be to provide your official container image for the Jetson Platform. In this way, developers can directly pull that image from the Docker Hub even on recent JetPacks avoiding problems!

[robertanto]

@mbusy have you compiled boost by yourself?

[mbusy]

@robertanto

I will try by using directly the Jetpack 3.3 as done by @mbusy.

Good idea, I'm not sure to understand how using a docker container would cause that problem, but that's the only difference between your setup and ours

A solution may be to provide your official container image for the Jetson Platform. In this way, developers can directly pull that image from the Docker Hub even on recent JetPacks avoiding problems!

Indeed, the idea is interesting and we might upload that on docker hub

@mbusy have you compiled boost by yourself?

Not at all, the instructions and patches that I gave should be sufficient for you to mirror our setup (provided that you use JetPack 3.3 (Ubuntu 16.04))

[robertanto]

Thank you again to both!

I will let you know!

[robertanto]

@mbusy @nyibbang

It worked with the Jet pack 3.3.3.

I installed boost 1.58.0 with:

sudo apt-get install libboost-all-dev

Then I followed the guide.

I will try to do the same in a container.

Thank you again!

[robertanto]

Now (using boost from apt) it works also on Docker.

Probably the problem was in the way in which I compiled boost or in the version itself.

[robertanto]

I have created a repository containing a faster way to use the library: https://github.com/robertanto/libqi-python-nvidia-jetson

It also works with the latest JetPack (4.4.1).

I hope that this repository can help researchers in their activities.