Support for ARMV6 Architecture

[ergouser]

Using the develop-SNAPSHOT from Maven:

The Linux-ARM binary is ARMV7 on the Pi Zero this give you:

siginfo: si_signo: 4 (SIGILL), si_code: 1 (ILL_ILLOPC), si_addr: 0xa75b61f0

(oddly in one of the JNA stackframes) as the library loads.

Pi Zero (and the old Pi 1's) are ARMV6, not ARMV7 (all the later Pi's).

Pi ZERO: $ cat /proc/cpuinfo processor : 0 model name : ARMv6-compatible processor rev 7 (v6l) BogoMIPS : 697.95 Features : half thumb fastmult vfp edsp java tls CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x0 CPU part : 0xb76 CPU revision : 7

Hardware : BCM2835 Revision : 9000c1 Serial : 0000000003492989 Model : Raspberry Pi Zero W Rev 1.1

The maven build for JNA is V6

$ arm-linux-gnueabihf-readelf -A com/sun/jna/linux-arm/libjnidispatch.so Attribute Section: aeabi File Attributes Tag_CPU_name: "6" Tag_CPU_arch: v6 Tag_ARM_ISA_use: Yes Tag_THUMB_ISA_use: Thumb-1 Tag_FP_arch: VFPv2 Tag_ABI_PCS_wchar_t: 4 Tag_ABI_FP_denormal: Needed Tag_ABI_FP_exceptions: Needed Tag_ABI_FP_number_model: IEEE 754 Tag_ABI_align_needed: 8-byte Tag_ABI_enum_size: int Tag_ABI_HardFP_use: Deprecated Tag_ABI_VFP_args: VFP registers Tag_CPU_unaligned_access: v6

I would suggest that the HID build could be the same, it's currently V7 $ arm-linux-gnueabihf-readelf -A libhidapi.so Attribute Section: aeabi File Attributes Tag_CPU_name: "7-A" Tag_CPU_arch: v7 Tag_CPU_arch_profile: Application Tag_ARM_ISA_use: Yes Tag_THUMB_ISA_use: Thumb-2 Tag_FP_arch: VFPv3-D16 Tag_ABI_PCS_wchar_t: 4 Tag_ABI_FP_denormal: Needed Tag_ABI_FP_exceptions: Needed Tag_ABI_FP_number_model: IEEE 754 Tag_ABI_align_needed: 8-byte Tag_ABI_align_preserved: 8-byte, except leaf SP Tag_ABI_enum_size: int Tag_ABI_VFP_args: VFP registers Tag_CPU_unaligned_access: v6

[ergouser]

I would add that rebuilding the .so's for ARMv6 solved the ILL_ILOPC issue. I built them native on the Pi Zero from the libusb/hidapi repository so don't have suggestions for cross-compiling.

[tresf]

Gary uses dockcross for his builds.

What's a bit confusing to me is that the linux-armel profile claims to builds for V6 where as the linux-arm builds for V7, both seem to use hardware-floating-point tools to do so despite the armel being described as a "software-floating-point architecture"? Anyway, I'm not very familiar with the differences, but wanted to provide some insight into the build system Gary uses. Below are the relevant lines. I'm working through this build process for a snapshot I need for a project. I'm sure your fix is just a matter of tweaking one of the lines below, then you can send a pull request with the required change. :)

linux-armel

https://github.com/gary-rowe/hid4java/blob/b1c3a6dcd7e33ee818ebcdb827ab92d4ca28de53/build-hidapi.sh#L224-L227

linux-arm

https://github.com/gary-rowe/hid4java/blob/b1c3a6dcd7e33ee818ebcdb827ab92d4ca28de53/build-hidapi.sh#L244-L247

[tresf]

So it turns out dockcross uses the hardware-floating-point utility for compiling V6 targets... quoting:

# The CROSS_TRIPLE is a configured alias of the "armv6-unknown-linux-gnueabihf" target.
ENV CROSS_TRIPLE armv6-unknown-linux-gnueabihf
ENV CROSS_ROOT ${XCC_PREFIX}/${CROSS_TRIPLE}
ENV AS=${CROSS_ROOT}/bin/${CROSS_TRIPLE}-as \
    AR=${CROSS_ROOT}/bin/${CROSS_TRIPLE}-ar \
    CC=${CROSS_ROOT}/bin/${CROSS_TRIPLE}-gcc \
    CPP=${CROSS_ROOT}/bin/${CROSS_TRIPLE}-cpp \
    CXX=${CROSS_ROOT}/bin/${CROSS_TRIPLE}-g++ \
    LD=${CROSS_ROOT}/bin/${CROSS_TRIPLE}-ld \
    FC=${CROSS_ROOT}/bin/${CROSS_TRIPLE}-gfortran

There's a chance this compiler allows forcing back to software-floating-point, but from what I can tell, the builds are simply broken in the scripts current form and needs to be fixed. I'll see if I can patch it to work on my local.

[ergouser]

I don't claim to be an expert on the different Pi's (I was blind-sided by Pi3/4 and Pi Zero difference yesterday). However, it is my understanding the that ArmV6 in the Pi Zero does have hardware floating point support.

[ergouser]

I'll see if I can get the cross compiling to work, last time I tried Manjaro complained about something that seemed as though it was going to be a lengthy problem to resolve.

In the Pi Zero, "out of the box" applications use the linux-arm library, I don't know what platforms might choose the armel by default. (Is there a runtime switch to choose the architecture?).

The solution to getting a v6 build would seems to be just to change line 247

dockcross-linux-armv7 ----> dockcross-linux-armv6

Whether, philosophically, that's the right approach I don't feel qualified to judge, but it does seem to be a common approach for Pi binaries - JNA provides V6. Eclipse UPM compiled on the Pi4 also generates V6 .so's.

I assume that there's a performance (and other?) penalty but, for the HID code I would also not expect it to be significant. Software floating point might be a bottleneck, except that the amount of floating point in the HID code is likely pretty limited.

[tresf]

it is my understanding the that ArmV6 in the Pi Zero does have hardware floating point support.

Ok, so I'm focusing on the wrong binary. :). Well, the armel is definitely broken, it can't find <libudev.h> for some reason.

To your issue, the hard-float library (a.k.a. linux-arm, that seems to just be a number away from being the armv6.

- echo -e "${green}Building ARMv7 hard float${plain}"
+ echo -e "${green}Building ARMv6 hard float${plain}"
-    dockcross-linux-armv7 bash -c 'sudo dpkg --add-architecture armhf && sudo rm -Rf /var/lib/apt/lists && sudo apt-get update && sudo apt-get --yes install libudev-dev:armhf libusb-1.0-0-dev:armhf gcc-arm-linux-gnueabihf && sudo make clean && sudo ./bootstrap && sudo ./configure --host=arm-linux-gnueabihf CC=arm-linux-gnueabihf-gcc && sudo make'
+    dockcross-linux-armv6 bash -c 'sudo dpkg --add-architecture armhf && sudo rm -Rf /var/lib/apt/lists && sudo apt-get update && sudo apt-get --yes install libudev-dev:armhf libusb-1.0-0-dev:armhf gcc-arm-linux-gnueabihf && sudo make clean && sudo ./bootstrap && sudo ./configure --host=arm-linux-gnueabihf CC=arm-linux-gnueabihf-gcc && sudo make'

[ergouser]

That's what I would expect and I would then expect that binary to work on all Pi's (0-4). I don't see any downside to this change although, admittedly my view may be a little biased...

[tresf]

Note, I tested the line above. The build says this:

linux-arm
ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=ff49c8f1c9270a9393e7b7cca94ba2d769ef8528, with debug_info, not stripped
ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=a7a785c285f9bfc911b25566497ffccc385a980f, with debug_info, not stripped

Is EABI5 the same as armv6?

[ergouser]

Yes. The only command I know is :

arm-linux-gnueabihf-readelf -A libhidapi.so

which will identify the architecture.

[tresf]

my view may be a little biased...

If it's what JNA targets, it's what this library should target since it depends on JNA, it should have identical levels of compatibility.

Yes. The only command I know is : arm-linux-gnueabihf-readelf -A libhidapi.so

It took me a while to get it working, but it gives me this, which I don't think is what we want:

Attribute Section: aeabi
File Attributes
  Tag_CPU_name: "7-A"
  Tag_CPU_arch: v7
  Tag_CPU_arch_profile: Application
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-2
  Tag_FP_arch: VFPv3-D16
  Tag_ABI_PCS_wchar_t: 4
  Tag_ABI_FP_denormal: Needed
  Tag_ABI_FP_exceptions: Needed
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_align_preserved: 8-byte, except leaf SP
  Tag_ABI_enum_size: int
  Tag_ABI_VFP_args: VFP registers
  Tag_CPU_unaligned_access: v6

[ergouser]

I would say not...

$ arm-linux-gnueabihf-readelf -A libhidapi.so Attribute Section: aeabi File Attributes Tag_CPU_name: "6" Tag_CPU_arch: v6 Tag_ARM_ISA_use: Yes Tag_THUMB_ISA_use: Thumb-1 Tag_FP_arch: VFPv2 Tag_ABI_PCS_wchar_t: 4 Tag_ABI_FP_denormal: Needed Tag_ABI_FP_exceptions: Needed Tag_ABI_FP_number_model: IEEE 754 Tag_ABI_align_needed: 8-byte Tag_ABI_align_preserved: 8-byte, except leaf SP Tag_ABI_enum_size: int Tag_ABI_VFP_args: VFP registers Tag_CPU_unaligned_access: v6

[tresf]

Yeah, I don't know why it's building for the wrong architecture, the dockcross container clearly states it should build for ARMv6 with the one we're using... https://github.com/dockcross/dockcross/blob/aa4bcf4a34692fb4d0dc61d6dad8f6737c0b6f83/linux-armv6/Dockerfile.in#L10.

I'll see if I can figure out what I'm doing wrong. It should be pretty straight forward.

[ergouser]

There's an issue that I don't really understand related to linking, gcc and ARMv7

https://stackoverflow.com/questions/55465118/gcc-8-cross-compiler-outputs-armv7-executable-instead-of-armv6

[tresf]

There's an issue that I don't really understand related to linking, gcc and ARMv7

https://stackoverflow.com/questions/55465118/gcc-8-cross-compiler-outputs-armv7-executable-instead-of-armv6

Good catch. Check this out: https://stackoverflow.com/a/65186762/3196753

Quoting:

By default, newer GCC versions do not create correct binaries for ARMv6. Even though you pass the correct -mcpu= flag to gcc, it will create startup code for the newer ARMv7 architecture. Running them on your RasPI Zero will cause an "Illegal Instruction" exception.

this info is mentioned here https://github.com/Pro/raspi-toolchain

Ouch!

[tresf]

Also (essentially the same observation): https://stackoverflow.com/questions/55465118/gcc-8-cross-compiler-outputs-armv7-executable-instead-of-armv6

Yeah, I tried the linux-armv6-lts Docker file as well as the linux-armv6-musl. The first still produced v7 binaries, the latter gets hung up on libudev.h: No such file or directory, just like the soft-float compilation I was working on earlier.

Most of the forums state to avoid Ubuntu's linker so I'm not sure if its possible, but I was hoping that perhaps the musl version lacked such a linker defect. :/

[tresf]

for the HID code I would also not expect it to be significant. Software floating point might be a bottleneck

From my experience, you can't mix hf and sf binaries. Logic would suggest sf would be backwards compatible, but in my experience it's not, which is why projects like JNA explicitly offer both (in addition to any performance gains, of course)

[tresf]

I'm not sure if its possible

Note, everything's possible. If you do get dockcross setup (please work from develop branch, it has some fixes) take a look at how we shimmed a custom cross-compiler into the environment here: #115

Specifically this part, which downloads, extracts and configures a custom compiler:

https://github.com/gary-rowe/hid4java/blob/971456d0120c23104a5dd75285869ce617688865/build-hidapi.sh#L162-L165

1. Fetch and extract a custom toolchain (https://github.com/mstorsjo/llvm-mingw/releases/download/20201020/llvm-mingw-20201020-msvcrt-ubuntu-18.04.tar.xz)
2. Add the toolchain's path to our path
3. Unset all compiler variables like CC CPP, etc.
4. Install clang (that particular one uses clang under the covers)
5. Configure using --host=<toochain>

[ergouser]

IIRC the native build got stuck on that. I installed apt install libudev-dev to resolve it.

BUT that seems to already be in the script.

[tresf]

BUT that seems to already be in the script.

Right, same with linux-armel. I don't know why this fails. Some tutorials state that it's because pkg-config can't find it, I don't believe that although it may be that the include line is version specific and it's really in lib-1.0.0/libudev.h or something, but re-running the container each time is quite a bit of trial and error. If you can get it working I'd be happy to help with the PR.

[ergouser]

Once solution would be to pull in the build tools from, say, here: https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Raspberry%20Pi%20GCC%20Native-Compiler%20Toolchains/Buster/GCC%209.3.0/

If I understand the dockcross correctly (seems somewhat unlikely) it's using qemu to emulate the processor and so would use the "native" files, which I know work.

Problem is, I can't get the existing build to work, which discourages me from attempting modifications. Currently problem is, I assume, related to /usr/local/bin/sudo not being found so something to do with install-gosu-binary-wrapper.sh and so common.dotcross not being called. Which seems to happen from the Makefile and I can't figure out what/why a make should be in progress...

Building ARMv7 hard float /usr/local/bin/sudo: 5: 3: not found /usr/local/bin/sudo: 5: 3: not found /usr/local/bin/sudo: 5: 2: not found Get:1 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB] Get:2 http://deb.debian.org/debian bullseye InRelease [154 kB] Get:3 http://security.debian.org/debian-security bullseye-security/main armhf Packages [808 B] Get:4 http://security.debian.org/debian-security bullseye-security/main amd64 Packages [812 B] Get:5 http://deb.debian.org/debian bullseye-updates InRelease [40.1 kB] Get:6 http://deb.debian.org/debian bullseye/main amd64 Packages [8178 kB] Get:7 http://deb.debian.org/debian bullseye/main armhf Packages [7940 kB] Fetched 16.4 MB in 5s (3478 kB/s)
Reading package lists... Done /usr/local/bin/sudo: 5: 6: not found Reading package lists... Done Building dependency tree... Done Reading state information... Done The following additional packages will be installed: binutils-arm-linux-gnueabihf cpp-10-arm-linux-gnueabihf cpp-arm-linux-gnueabihf gcc-10-arm-linux-gnueabihf gcc-10-arm-linux-gnueabihf-base gcc-10-base:armhf gcc-10-cross-base krb5-locales libasan6-armhf-cross libatomic1-armhf-cross libc6:armhf libc6-armhf-cross libc6-dev-armhf-cross libcom-err2:armhf libcrypt1:armhf libgcc-10-dev-armhf-cross libgcc-s1:armhf libgcc-s1-armhf-cross libgomp1-armhf-cross libgssapi-krb5-2 libgssapi-krb5-2:armhf libidn2-0:armhf libk5crypto3 libk5crypto3:armhf libkeyutils1:armhf libkrb5-3 libkrb5-3:armhf libkrb5support0 libkrb5support0:armhf libnsl2:armhf libnss-nis:armhf libnss-nisplus:armhf [...lines of apt-like text....] Setting up libnsl2:armhf (1.3.0-2) ... Setting up libnss-nisplus:armhf (1.3-4) ... Setting up libnss-nis:armhf (3.1-4) ... Processing triggers for libc-bin (2.31-12) ... /usr/local/bin/sudo: 5: 2: not found make: *** No rule to make target 'clean'. Stop. Failed - Removing damaged targets rm: cannot remove '../../Java/Personal/hid4java/src/main/resources/linux-arm/libhidapi.so': No such file or directory

[tresf]

related to /usr/local/bin/sudo not being found so something to do with

From my experience, this message is benign.

so would use the "native" files, which I know work.

Correct.

make: *** No rule to make target 'clean'. Stop.

There's your problem! Gary expected there to be a build beforehand and there's not (he re-uses the build directory across containers, something I think we'll be changing soon). Anyway, just change make clean && to make clean ; so it doesn't bomb out your script. 🍻

[ergouser]

This pulls in the cross compiler from https://github.com/abhiTronix/raspberry-pi-cross-compilers .

It gets stuck on configure

_checking for clockgettime in -lrt... no

It could likely be made to succeed without too much effort, but it's not a particularly good solution even if completed. It'll only run on a Linux host and the use of a third party cross compiler seem cluncky.

dockcross-linux-armv7 bash -c 'cat /usr/local/bin/sudo && echo which gosu && sudo dpkg --add-architecture armhf && echo "First complete" && sudo rm -Rf /var/lib/apt/lists && sudo apt-get update && sudo apt-get --yes install libudev-dev:armhf libusb-1.0-0-dev:armhf gcc-arm-linux-gnueabihf binutils-arm-linux-gnueabihf && ls -altr /usr/include/ && if [ ! -f cross-gcc-9.3.0-pi_0-1.tar.gz ]; then wget https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Raspberry%20Pi%20GCC%20Cross-Compiler%20Toolchains/Buster/GCC%209.3.0/Raspberry%20Pi%201%2C%20Zero/cross-gcc-9.3.0-pi_0-1.tar.gz ; fi && tar -xzf cross-gcc-9.3.0-pi_0-1.tar.gz && ./cross-pi-gcc-9.3.0-0/bin/arm-linux-gnueabihf-gcc -v && AR=arm-linux-gnueabihf-ar && export CC=arm-linux-gnueabihf-gcc && export CXX=arm-linux-gnueabihf-g++ && export CPP=arm-linux-gnueabihf-cpp && export RANLIB=arm-linux-gnueabihf-gcc-ranlib && export LD=$CXX && sudo ln -sf /usr/include/arm-linux-gnueabihf/asm/ /usr/include/asm/ ; sudo ln -sf /usr/include/arm-linux-gnueabihf/gnu/ /usr/include/gnu/ ; sudo ln -sf /usr/include/arm-linux-gnueabihf/bits/ /usr/include/bits/ ; sudo ln -sf /usr/include/arm-linux-gnueabihf/sys/ /usr/include/sys/ ; sudo ln -sf /usr/include/arm-linux-gnueabihf/openssl/ /usr/include/openssl/ ; sudo ln -sf /usr/lib/arm-linux-gnueabihf/crtn.o /usr/lib/crtn.o ; sudo ln -sf /usr/lib/arm-linux-gnueabihf/crt1.o /usr/lib/crt1.o ; sudo ln -sf /usr/lib/arm-linux-gnueabihf/crti.o /usr/lib/crti.o ; export PATH=pwd/cross-pi-gcc-9.3.0-0/bin:$PATH && export LD_LIBRARY_PATH=pwd/cross-pi-gcc-9.3.0-0/lib:$LD_LIBRARY_PATH && pwd && arm-linux-gnueabihf-gcc -v && sudo ./bootstrap && sudo ./configure --host=arm-linux-gnueabihf && sudo make'

[ergouser]

Where do the dockcross compilers come from/how are they built? And should this issue be pushed upstream?

Fundamentally, the issue, as we know is that the arm-linux-gnueabihf-gcc is built without the flag:

--with-arch=armv6

and so applying:

-march=armv6

fails. If I understand correctly this means that dockcross cannot be used for any project for the Pi zero. Which suggests any fix/workaround should be at the dockcross level (or higher) not here.

Target: arm-linux-gnueabihf Configured with: ../src/configure -v --with-pkgversion='Debian 10.2.1-6' --with-bugurl=file:///usr/share/doc/gcc-10/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-10 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libitm --disable-libquadmath --disable-libquadmath-support --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --without-target-system-zlib --enable-multiarch --disable-sjlj-exceptions --with-arch=armv7-a --with-fpu=vfpv3-d16 --with-float=hard --with-mode=thumb --disable-werror --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=arm-linux-gnueabihf --program-prefix=arm-linux-gnueabihf- --includedir=/usr/arm-linux-gnueabihf/include --with-build-config=bootstrap-lto-lean --enable-link-mutex Thread model: posix Supported LTO compression algorithms: zlib gcc version 10.2.1 20210110 (Debian 10.2.1-6)

[ergouser]

HIDAPI compiles native on the Pi Zero with the default gcc. A workaround for anyone hitting this issue before it's resolved.

Add libudev: sudo apt install libudev-dev

Download the hidapi repository (libusb/hidapi) and build in the usual way (./bootstrap ; ./configure ; make )

You may get errors with some additional missing packages, all should be solvable with apt

If you get a .so called libhidapi-hidraw.so rename it to libhidapi.so You should also get a libhidapi-libusb.so

Check that you have a v6 version (see comments above if you're not sure how to interpret the output).

arm-linux-gnueabihf-readelf -A libhidapi.so

Replace the .so's with the same name in the linux-arm folder of the distribution jar.

[tresf]

it's not a particularly good solution even if completed. It'll only run on a Linux host

This statement is true for the entire dock-cross environment actually, they're currently all Linux-only hosts, so that's not particularly a problem. Dockcross seems to use Debian Jessie as the baseline for all containers.

and the use of a third party cross compiler seem cluncky.

This is the strategy used in #115, which successfully switches a Dockcross container to a 3rd party llvm-mingw compiler (one which simply is a wrapper around the system installed clang and which is also used by many projects). I do not this this is a hinderance or deterrent. :)

Fundamentally, the issue, as we know is that the arm-linux-gnueabihf-gcc is built without the flag: --with-arch=armv6 -march=armv6 fails. If I understand correctly this means that dockcross cannot be used for any project for the Pi zero. Which suggests any fix/workaround should be at the dockcross level (or higher) not here.

It depends how fast you want the issue resolved. Dockcross isn't a requirement, just a convenience for Gary's build toolchain to speed up releases. Opening an upstream bug report is absolutely warranted, but at this point, it seems the ARMv6 bugs with Ubuntu are fairly well-known by the Pi community, so in my opinion, the "upstream" dockcross project just need to rename the container to more properly represent what it builds (e.g. ARMv7) or remove it altogether. Adding the Pi toolchain would be a better approach, but I'm just not sure how difficult that is versus shimming our own. See https://github.com/dockcross/dockcross/issues/463#issuecomment-881089728 Notice, upvotes, not comments. 🤷‍♂️

Note, from what I've read ARMv6 binaries are absolutely attainable using Ubuntu, but the linker doesn't support ARMv6 and is what breaks compatibility.

If you don't mind sharing your script, I'd be happy to test it out and try to get it building. My project (and I'm sure many others) have a few Raspberry Pi customers and I'd love to add HID support for them!

[tresf]

If you don't mind sharing your script, I'd be happy to test it out and try to get it building. My project (and I'm sure many others) have a few Raspberry Pi customers and I'd love to add HID support for them!

Whoops, I found it. I'll take a look, thanks!

Click to expand script

```bash dockcross-linux-armv7 bash -c 'cat /usr/local/bin/sudo && echo which gosu && sudo dpkg --add-architecture armhf && echo "First complete" && sudo rm -Rf /var/lib/apt/lists && sudo apt-get update && sudo apt-get --yes install libudev-dev:armhf libusb-1.0-0-dev:armhf gcc-arm-linux-gnueabihf binutils-arm-linux-gnueabihf && ls -altr /usr/include/ && if [ ! -f cross-gcc-9.3.0-pi_0-1.tar.gz ]; then wget https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Raspberry%20Pi%20GCC%20Cross-Compiler%20Toolchains/Buster/GCC%209.3.0/Raspberry%20Pi%201%2C%20Zero/cross-gcc-9.3.0-pi_0-1.tar.gz ; fi && tar -xzf cross-gcc-9.3.0-pi_0-1.tar.gz && ./cross-pi-gcc-9.3.0-0/bin/arm-linux-gnueabihf-gcc -v && AR=arm-linux-gnueabihf-ar && export CC=arm-linux-gnueabihf-gcc && export CXX=arm-linux-gnueabihf-g++ && export CPP=arm-linux-gnueabihf-cpp && export RANLIB=arm-linux-gnueabihf-gcc-ranlib && export LD=$CXX && sudo ln -sf /usr/include/arm-linux-gnueabihf/asm/ /usr/include/asm/ ; sudo ln -sf /usr/include/arm-linux-gnueabihf/gnu/ /usr/include/gnu/ ; sudo ln -sf /usr/include/arm-linux-gnueabihf/bits/ /usr/include/bits/ ; sudo ln -sf /usr/include/arm-linux-gnueabihf/sys/ /usr/include/sys/ ; sudo ln -sf /usr/include/arm-linux-gnueabihf/openssl/ /usr/include/openssl/ ; sudo ln -sf /usr/lib/arm-linux-gnueabihf/crtn.o /usr/lib/crtn.o ; sudo ln -sf /usr/lib/arm-linux-gnueabihf/crt1.o /usr/lib/crt1.o ; sudo ln -sf /usr/lib/arm-linux-gnueabihf/crti.o /usr/lib/crti.o ; export PATH=pwd/cross-pi-gcc-9.3.0-0/bin:$PATH && export LD_LIBRARY_PATH=pwd/cross-pi-gcc-9.3.0-0/lib:$LD_LIBRARY_PATH && pwd && arm-linux-gnueabihf-gcc -v && sudo ./bootstrap && sudo ./configure --host=arm-linux-gnueabihf && sudo make' ```

[ergouser]

There's some obvious debugging in there (some echo's and ls's) the linking (ln -sf) of includes fails because the directories don't exist. I'm not sure why or the consequences. It may not be necessary to apt gcc, but, except for the time, I didn't feel it would be harmful and may pull something useful (like includes).

[tresf]

There's some obvious debugging in there (some echo's and ls's) the linking (ln -sf) of includes fails because the directories don't exist. I'm not sure why or the consequences. It may not be necessary to apt gcc, but, except for the time, I didn't feel it would be harmful and may pull something useful (like includes).

I'm working up a small version of it now on my local. Unfortunately I had some issues and had to blow away my containers and I'm currently on a metered connection so it's running slowly.

I'll share with you my initial config I'm working from while I await the environment to finish.

Click to expand

```bash cross_gcc_pi="https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Raspberry%20Pi%20GCC%20Cross-Compiler%20Toolchains/Buster/GCC%209.3.0/Raspberry%20Pi%201%2C%20Zero/cross-gcc-9.3.0-pi_0-1.tar.gz" download_extract='sudo mkdir -p /opt/cross-pi && wget -qO- '$cross_gcc_pi' | sudo tar xJvf - --strip 1 -C /opt/cross-pi/ > /dev/null && export PATH=/opt/cross-pi/bin:$PATH && export LD_LIBRARY_PATH=/opt/cross-pi/lib:$LD_LIBRARY_PATH' unsets='unset CC CPP CXX LD FC' dockcross-linux-x64 bash -c "$unsets && $download_extract"' && sudo apt-get install --yes libudev-dev:armhf libusb-1.0-0-dev:armhf && sudo make clean; sudo ./bootstrap && sudo ./configure --host=arm-linux-gnueabihf && sudo make' ```

Note, I've intentionally chosen to re-use the dockcross-linux-x64 container because I know for sure it's not snagging the broken Ubuntu armhf toolchain. :)

[tresf]

There's some obvious debugging in there (some echo's and ls's) the linking (ln -sf) of includes fails because the directories don't exist.

Yeah, the issue I rant into with #115 is that the Docker sets up the compiler variables and I had to unset them to get it to take a 3rd-party one. In regards to finding headers, libs and stuff that should all be pretty straight-forward since toolchain builds are actually quite common. In the past, I've just placed the relevant files earlier in the search path. This should allow the environment to fallback on native files when needed (such as the headers)

It may not be necessary to apt gcc, but, except for the time, I didn't feel it would be harmful and may pull something useful (like includes).

Ideally, Ubuntu will grab gcc as a prerequisite for any cross-compiler you specify. Since these VMs were all created for cross-compilation, I assume most come with either gcc or in some cases clang out of the box. :)

[tresf]

So I've gotten a bit further...

It gets further, but I receive the following build error:

- Library librt was not found on this system.
- Please install it and re-run ./configure

Digging deeper, I can see that librt.so.1 is located in a subdirectory of the Raspberry Pi Cross compiler, specifically in arm-linux-gnueabihf/libc/lib/librt.so.1 however I can't see to get the compiler to look there. I've tried export LD_LIBRARY_PATH to no avail as well as LD_FLAGS. Perhaps I'm missing a simple build flag? I feel like we're pretty close.

Perhaps if we start fresh from a clean Debian install and the same toolchain?

cross_gcc_pi="https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Raspberry%20Pi%20GCC%20Cross-Compiler%20Toolchains/Buster/GCC%209.3.0/Raspberry%20Pi%201%2C%20Zero/cross-gcc-9.3.0-pi_0-1.tar.gz"
download_extract='sudo mkdir -p /opt/cross-pi && wget -qO- '$cross_gcc_pi' | sudo tar xzvf - --strip 1 -C /opt/cross-pi/ > /dev/null && export PATH=/opt/cross-pi/bin:$PATH'
# Keep for metered connection, we can use "~/Workspaces/Cpp/hidapi/opt" instead :)
# download_extract='sudo mkdir -p opt/cross-pi && export PATH=opt/cross-pi/bin:$PATH'
unsets='unset AS AR CC CPP CXX LD FC'
dockcross-linux-x64 bash -c "$unsets && $download_extract"' && sudo dpkg --add-architecture armhf && sudo rm -Rf /var/lib/apt/lists && sudo apt-get update && sudo apt-get install --yes libudev-dev:armhf libusb-1.0-0-dev:armhf && sudo make clean ; sudo ./bootstrap && sudo ./configure --host=arm-linux-gnueabihf && sudo make'

[ergouser]

ldconfig might be worth a try, but I'm not convinced it would help.

[tresf]

ldconfig might be worth a try, but I'm not convinced it would help

Thanks, yeah I too tried this and didn't have a whole lot of luck.

[tresf]

@ergouser Just an update, dockcross replied to the request to add the new Windows architecture, so if we figure out how to get the prefix for the Pi toolchain working, I'm sure they'd be happy to do the same. :)

[tresf]

I took one more swing at this and failed. What I tried was to switch to another toolchain, but this time it complains:

checking for libudev... yes
checking for clock_gettime in -lrt... no

- Library librt was not found on this system.
- Please install it and re-run ./configure

Failed - Removing damaged targets

if ! dockcross-linux-armv7 bash -c 'export PATH="./cross-pi-gcc-8.3.0-0/bin:$PATH" && unset CC CPP CXX LD FC AR && sudo dpkg --add-architecture armhf && sudo rm -Rf /var/lib/apt/lists && sudo apt-get update && sudo apt-get --yes install libc6 libudev-dev:armhf libusb-1.0-0-dev:armhf && make clean && ./bootstrap && ./configure --host=arm-linux-gnueabihf && make CFLAGS="-march=armv6 -mfloat-abi=hard -mfpu=vfp"';

But first, I downloaded and extracted this toolchain (1.3 GB) to ~/Workspaces/Cpp/hidapi/cross-pi-gcc-8.3.0-0/ to avoid downloading each attempt.

Also, I used the following recommended optimization flags: https://github.com/abhiTronix/raspberry-pi-cross-compilers#optimization-flags-involved

I've tried to adjust LDFLAGS to no avail. librt.so seems to be properly located in the toolchain directory so I'm at a loss. Perhaps there's a compatibility issue with the container and this toolchain.

Next, I discovered that the toolchain was built for Debian "Buster", which dockcross seems to offer as dockcrosslinux-armv6-lts but when I attempt to use this, it grabs "Bullseye" instead, mucks up the ./configure step and somehow attempts to build with an ARMv7 toolchain again.

Hopefully the above information is helpful, but I've exhausted my patience trying to build for ARMv6. 🍻

[gary-rowe]

Thank you for update, and the massive effort you're putting in here. I think we should get a pair programming session organised to tackle this. I'll get in touch on Telegram and see what we can do.

[ergouser]

Was the issue ever posted upstream? They may have seen the -lrt issue before...

[tresf]

Was the issue ever posted upstream? They may have seen the -lrt issue before...

No, and they're the real culprit since they advertise ARMv6 but don't offer any suitable way (that I can find) to actually build for it. I think this has happened over time because only the Pi Zero and Pi 1 have this requirement of ARMv6 but also hardfloat.

I was hoping to get a proof of concept up, then we could open a bug report, but the toolchain is just not obvious how to get working. I really hate to point the finger at volunteers, I'd rather point them to a solution, hence my hesitation to escalate. 🙁

[ergouser]

The Pi Zero seems to be popular and not likely to go away (or get updated to an ARM7) any time soon. It would be nice to have a solution to offer, but we seem to be stuck and if they've suffered with this before they may have some insight and suggestions.

Mostly about getting their help. Can you craft a summary and request and post it upstream?

[tresf]

The Pi Zero seems to be popular and not likely to go away

Agreed.

Can you craft a summary and request and post it upstream?

I'm not particularly vested in fixing this and I don't particularly care for the design choice to use dockcross (it could be argued that the time saved with the convenience of dockcross -- e.g. versus simply compiling this on a Zero -- is grossly overshadowed by the rube-goldberg process and never gained back. 🤷‍♂️

I think the best way to handle this problem is to:

1. Locally match the Debian OS level of a x86_64 toolchain of choice
2. Locally configure the toolchain
3. Locally build the project
4. Adapt to dockcross

Next is lobbying for dockcross to actually build for the architecture they promise. They'll probably be receptive to this, but there seems to be quite a few toolchain options out there for compiling for ARMv6. There's also the decision of which optimization to add to the build.

When I combine these future efforts with past efforts, I hope you understand why I choose to provide assistance and not take lead on this. The investment of time is too large. 🍻

[ergouser]

I don't have time to go back and check it, but dockcross may have fixed the issue: https://github.com/gotson/komga/releases/tag/v0.115.1

[tresf]

I don't have time to go back and check it, but dockcross may have fixed the issue: https://github.com/gotson/komga/releases/tag/v0.115.1

Perhaps, but this bug report and all of our testing came after the July merge which claims to have fixed ARMv6 support, so perhaps we're just doing something wrong?

[gary-rowe]

There should now be an ARMv6 linux-arm for both hidraw and libusb in the develop branch for hidapi 0.12.0. This was compiled on a RPi and copied over as I couldn't get the library to compile reliably using Dockcross.

[gary-rowe]

Given that this library has been verified as working on an RPi with ARMv6 I'll close this issue.