Support Raspberry Pi

[ThomasJClark]

It would be cool if GRIP ran on a Raspberry Pi. To do this, we need:

• [ ] A Pi build of libntcore
• [ ] A Pi build of JavaCV
• [x] A more flexible deploy too
  • Probably with defaults suited to the roborio, but have the username/password/directories customizable

GRIP would likely only work in headless mode (I don't think JavaFX is supported on arm/linux)

[bradamiller]

I have a pi set aside to bring in tomorrow anticipating this discussion. I wonder if it would work as a test if we created the directories on the pi so it looks like a roboRIO?

Brad

_____________________________

From: Thomas Clark notifications@github.com Sent: Thursday, January 14, 2016 9:14 PM Subject: [GRIP] Support Raspberry Pi (#366) To: WPIRoboticsProjects/GRIP grip@noreply.github.com

It would be cool if GRIP ran on a Raspberry Pi. To do this, we need: A Pi build of libntcore A Pi build of JavaCV A more flexible deploy tool Probably with defaults suited to the roborio, but have the username/password/directories customizable

GRIP would likely only work in headless mode (I don't think JavaFX is supported on arm/linux)

— Reply to this email directly or view it on GitHub.

[ThadHouse]

Pi 1 will not work for sure, It's Arm v6 whereas the RoboRIO is Arm v7. I did try RoboRIO ntcore builds on a BeagleBoneBlack, and they did not work either. I'm guessing because of the VFP vs HardFloat differences. I'm actually setting up a build server to build ntcore for Armv7 hf and android. The builds I'm building can be found here http://198.199.94.5/ntcore/ if you want to try them out. The arm-linux-gnueabihf should be the builds that work on the Pi 2. They will not work on a Pi 1, but I will look at trying to find a Pi 1 cross compiler, and then can add it really easily.

[JLLeitschuh]

The deploy script is already incredibly flexible. The UI components are extensible. I could probably get this working in ~1.5 hours. (Getting the UI to look right will probably take the most time)

[bradamiller]

I wouldn't spend much time on a pi 1 build where the the performance is so much less and a pi 2 is only $40. Maybe I'll have time to play with it over the weekend, but I'm bring a few pi 2 boards if the grip guys want to try.

[ThadHouse]

I just a bit of testing. Running my build on a BeagleBone Black started to load correctly, however then gave an error saying libstdc++.so.6 wasn't right. Spent a while trying to debug that, but couldn't. Don't know enough linux to debug that more. The cross compiler was GCC 4.9.3, but my BBB only had 4.9.2, and I couldn't figure out how to upgrade it. Hopefully it works on a Pi 2, or you can get GCC 4.9.3 onto it.

I also retested the RoboRIO image on the BBB. It gave a bad format error when trying to load. So that build will almost certainly not work on a Pi.

[ThadHouse]

I added a zip of the lib folder used for cross compiling to the ntcore folder. That should have all the libraries needed if any more are needed.

[ThadHouse]

Ok. So by preloading the libstdc++.so.6 in the zip folder, the cross compiled version started working on my BeagleBone Black. So the builds should work on Pi 2 either without issues, or require preloading a library before running, which shouldn't be too bad.

EDIT: Tested on my brand new CHIP as well, and the builds worked there too, as long as you had the specific libstdc++.so.6. So I put that file alone in the ntcore directory. And all you should need to do is when you run the program set LD_LIBRARY_PATH to whatever folder you put that library in.

[dcodeh]

Could you elaborate on the process you used to get this working on the BBB/Pi 2?

We have been attempting to re-build with some of the suggestions in this thread to no avail.

[ThadHouse]

I didn't test getting the full GRIP suite working. I was only doing testing to see if I could get NetworkTables working properly.

[multiplemonomials]

I also got an error message about how the JVM couldn't load the libraries when I tried GRIP on my Beaglebone Black (running Arch Linux). However, ThadHouse's ntcore.so and libstdc++ didn't fix it for me. Then, I tried rebuilding ntcore on the BBB, and using that .so file fixed the error message about ntcore.so. I then rebuilt all of OpenCV and the JavaCPP bindings (javacpp-presets) using arm-linux-gnueabihf-gcc from the AUR in an Arch VM, and then substituted them into the GRIP jar file. It WORKED! I guess the libjniopencv_foo libraries in opencv-3.0.0-1.1-linux-frc.jar have the wrong ABI for the BBB (and others?).

If you want to try it, just scp the this file and a GRIP save file onto your coprocessor, and run with

java -jar GRIP-archlinuxarm-deployable.jar test.grip

[ThadHouse]

You should have been able to use the jar in the Java folder, but maybe it doesn't support Arch. I've only ever tested it on Debian based OS's, and only the actual library, not the jar. I'll try the jar tonight on my BBB and Pi 2.

And it's not surprising that the FRC jar does not work. The RIO is is Soft Float, whereas Pi's and BBBs are Hard Float.

[multiplemonomials]

It was a journey, but I completed the first two items on your list of what was necessary for Raspberry Pi support. I built NTCore, opencv, and the OpenCV JavaCPP Preset for linux-arm. Not entirely sure about the ABI, these were compiled with GCC 5.3.0 arm-linux-gnueabihf on Arch Linux. I also added the prebuilt natives (which existed for every platform except linux-arm), and edited the build.gradle file to add all of this stuff into the jar. I changed the core project dependencies block to this:

dependencies {
        compile name: 'javacpp-opencv-preset'

        compile name: 'NetworkTables-3.0.0-natives-arm'    
        compile name: 'opencv-natives-linux-arm-hardfloat'
        compile name: 'opencv-natives-linux-x86'
        compile name: 'opencv-natives-linux-x86_64'
        compile name: 'opencv-natives-macosx-x86_64'
        compile name: 'opencv-natives-windows-x86'
        compile name: 'opencv-natives-windows-x86_64'

        compile group: 'org.bytedeco', name: 'javacpp', version: '1.1'
        compile group: 'org.bytedeco', name: 'javacv', version: '1.1'
        compile group: 'org.python', name: 'jython', version: '2.7.0'
        compile group: 'com.thoughtworks.xstream', name: 'xstream', version: '1.4.8'
        compile group: 'org.apache.commons', name: 'commons-lang3', version: '3.4'
        compile group: 'com.google.guava', name: 'guava', version: '18.0'
        // We use the no_aop version of Guice because the aop isn't avaiable in arm java
        // http://stackoverflow.com/a/15235190/3708426
        // https://github.com/google/guice/wiki/OptionalAOP
        compile group: 'com.google.inject', name: 'guice', version: '4.0', classifier: 'no_aop'
        compile group: 'com.google.inject.extensions', name: 'guice-assistedinject', version: '4.0'
        compile group: 'edu.wpi.first.wpilib.networktables.java', name: 'NetworkTables', version: '3.0.0-SNAPSHOT', classifier: 'desktop'
}

JavaCPP doesn't distinguish between hard and soft float ARM (I assume that's the crucial difference between the RPi and the RoboRIO), and there's no easy way to make it do so. Therefore, natives for the RoboRIO and other Linux boards cannot coexist in the same jar. I think it would make sense to have two jars that are deployable: one with some or all of the above natives, and one with just

        compile name: 'opencv-linux-arm-roborio'
        compile name: 'NetworkTables-3.0.0-natives-roborio'

The user could select whether to deploy to a "generic" coprocessor, which would use the multi-native file, or a RoboRIO specifically, which would use the other jar with just those natives.

Also, arm-linux does actually have OpenJFX, so using these libraries would allow GRIP's GUI to run on embedded Linux boards.

I hope this is useful to you, and I hope we can get this working. You can download the libs here.

Things I built: javacpp-opencv-preset.jar: The Java code which interfaces directly with the native OpenCV libs opencv-natives-linux-arm-hardfloat.jar: The OpenCV 3.0.0 library natives, and the JavaCPP-generated interface libraries NetworkTables-3.0.0-natives-arm.jar: libntcore.so, built for linux-armhf

[ThadHouse]

Did you cross compile any of those? Or were they all compiled natively? If cross, how'd you get GCC5.3 instead of 4.9.3, which is the newest I could find looking around. I would like to get rid of the libstdc++.so requirement of my build if possible.

[multiplemonomials]

I cross compiled them with the latest arm-linux-gnueabihf-gcc available from the Arch User Repository.

[ThadHouse]

So I can't get your copy working on raspbian either, without the LD_PRELOAD trick. I end up getting the same error I was getting, where it can't find a libstdc++.so.6 with CXXABI_1.3.9, which it looks like GCC 5 compiles to. Turns out my build was compiling with GCC 5 anyway, so I don't know why you couldn't get it working on yours. It seems like the binaries produced should be identical whether or not the cross compiler is hosted on arch or debian.

Also it looks like your NetworkTables-3.0.0-natives-arm.jar does not include any of the java files, and instead just libntcore.so. Is that right? Everytime I've built ntcore, it get a jar that include both the java files and the native library.

[multiplemonomials]

Can you check what libstdc++ ABI version your Raspberry Pi is using? It looks like there's a command line option to set it.

And yeah, I just built the c++ (cmake) part of ntcore because that is all that's needed to run GRIP on armhf. I was having some issues getting the java (gradle) part to run.

[ThadHouse]

Mines using the one from GCC 4.9. I can't find anywhere to install GCC 5 from for Raspbian currently. Thats why I was just grabbing the libstdc++.so.6 from the cross compiler itself, and preloading that one before running. I tried switching to the GCC 4.9 cross compiler, and it compiled, however it was generating an illegal instruction for some reason, so I need to figure that out. I don't know how easy its going to be to get a universal configuration working unless we do the preloading trick, which should work on any device. I'll install arch on my BBB sometime this week and try and find a universal solution that would work for the most devices.

What part of gradle were you having trouble running? Once I got java 8 installed, the only special thing I had to do was set -PcompilerPrefix=arm-linux-gnueabihf-, which changes from the frc prefix to the generic hf prefix.

[multiplemonomials]

Can you post raspbian's libstdc++.so so I can see if I can figure out what ABI it's using?

[multiplemonomials]

Update: Or just run

strings /usr/lib/libstdc++.so | grep CXXABI 

on the Pi and post the output.

EDIT: sorry for the repeated post, GitHub/Pale Moon was being weird.

[ThadHouse]

CXXABI_1.3
CXXABI_1.3.1
CXXABI_1.3.2
CXXABI_1.3.3
CXXABI_1.3.4
CXXABI_1.3.5
CXXABI_1.3.6
CXXABI_1.3.7
CXXABI_1.3.8
CXXABI_TM_1
CXXABI_ARM_1.3.3

Those are what I get. And that is a fresh install of raspbian jessie with apt-get update and upgrade ran.

[multiplemonomials]

OK, so according to this page, anything we compile with GCC 4.9.0 or earlier (not sure about 4.9.3) should be fine. I can also install that older version of libstdc++ into my toolchain.

[ThadHouse]

Oh can I just copy paste that one into the toolchain? or would you need to add other files? I still don't know why the 4.9 cross compiler kept crashing. It compiled correctly, and technically ntcore will compile with GCC 4.8 as well. But I'll try coping in the older libstdc++ into my toolchain too to see if I can fix it. Heres the libstdc++ from my pi if you want to try it.

http://198.199.94.5/ntcore/PiLibStdC++.so/

[multiplemonomials]

I don't know if that would work. You can try. The safest way would be to download gcc 4.9.0 and build and install libstdc++ from inside the source tree. I won't have time to try that until Thursday, though.

[multiplemonomials]

libntcore.so.zip

Alright, try this ntcore library on your Pi. I installed libstdc++ 6.0.20 and used that to build ntcore:

cmake -DCMAKE_SHARED_LINKER_FLAGS="-L/usr/arm-linux-gnueabihf/oldc++/lib" -DCMAKE_CXX_FLAGS="-I/usr/arm-linux-gnueabihf/oldc++/include/c++/4.9.0 -I/usr/arm-linux-gnueabihf/oldc++/include/c++/4.9.0/arm-linux-gnueabihf" ..

It doesn't seem to require the CXXABI_1.3.9 symbol. If this works, I'll try rebuilding opencv.

[ThadHouse]

You didn't link a library.

[multiplemonomials]

Just updated my post.

[ThadHouse]

Ok I'll try it in an hour or so hopefully. We'll want to build with optimizations if we do an official one, but that should hopefully work for testing.

[JLLeitschuh]

Thanks you guys for taking this on. We have a lot we are trying to fix. This would be an awesome feature to implement if we could get it automated as part of our deployment.

[multiplemonomials]

I did some research, and I think the correct optimizations are -march=armv7-a -mtune=cortex-a7 -mfpu=neon (plus -O3 added by CMake in release mode). I attached the optimized version. libntcore.so.zip

[multiplemonomials]

Oh and by the way, I still can't build the Java part of ntcore.

➜  ntcore git:(master) ✗ gradle build
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel

FAILURE: Build failed with an exception.

* Where:
Script '/home/jamie/dev/ntcore/java/java.gradle' line: 21

* What went wrong:
A problem occurred evaluating script.
> Could not find property 'binaries' on task ':arm:jar'.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 6.451 secs

[ThadHouse]

Oh. You want to be using the gradle wrapper ./gradlew. In addition, you want to set the custom compiler prefix by using -PcompilerPrefix=yourcompilerprefixhere. Also did you custom compile your 4.9? If so I might want some help setting it up, as if your build works I want to be able to build it.

[multiplemonomials]

I downloaded GCC 4.9.0 and configured only in the libstdc++v3 subdirectory. I figured out I had to copy libgcc/gthr-posix.h to libgcc/gthr-default.h for this to work. Then, I installed to the /usr/arm-linux-gnueabi/oldc++ directory.

The commands I ran:

cd gcc-4.9.0
cp libgcc/gthr-posix.h libgcc/gthr-default.h
cd libstdc++-v3
mkdir build
cd build
../configure --prefix=/usr/arm-linux-gnueabihf/oldc++ --host=arm-linux-gnueabihf --enable-libstdcxx-threads
make
sudo make install

You still have to patch ntcore, though, because the LLVM headers included with ntcore try to use std::is_trivially_copyable because GCC is version 5. This doesn't work with the old c++ headers. Therefore, in ntcore/include/llvm/type_traits.h, you have to change lines 33 and 34 to

#if 0

[ThadHouse]

Oh is it still using GCC 5 only with libstdc++ for 4.9? Interesting.

[multiplemonomials]

Using gradlew did fix it, but I have no idea why. The one I have installed is v2.10, and gradlew uses v2.8. Does the newer version not work???

[ThadHouse]

So with that build, I get an _undefined symbol: ZdlPvj when trying to load it. And I think the gradlew has some custom mods to it to enable some hidden settings. I think thats why you need the wrapper.

[ThadHouse]

So I think if we do it right, the deploy can handle the LD_LOAD_LIBRARY required for a GCC 5 build. With the deploy changes is shouldn't be too difficult. However its going to require a separate GRIP build jar for sure. It will also probably require a button in the UI to switch between the RoboRIO and other devices. I don't know enough about Gradle or JavaFX to enable both of those things. @ThomasJClark would you be able to help with those? If so, I can get the deploy script added to where it should deploy right to the RIO and other ARM devices.

[ThomasJClark]

So I think if we do it right, the deploy can handle the LD_LOAD_LIBRARY required for a GCC 5 build. With the deploy changes is shouldn't be too difficult. However its going to require a separate GRIP build jar for sure. It will also probably require a button in the UI to switch between the RoboRIO and other devices. I don't know enough about Gradle or JavaFX to enable both of those things. @ThomasJClark would you be able to help with those? If so, I can get the deploy script added to where it should deploy right to the RIO and other ARM devices.

If I can get a link to whatever native libraries are working and what commands we need to run them, I think I can add it to the project

[BrianAtlanta]

Just got our Pi. Which linux distro should we use. I thought I read Raspbian over on ChiefDelphi.

[ThadHouse]

I only have the ntcore lib, which is here http://198.199.94.5/ntcore/cb4cc63/arm-linux-gnueabihf/java/ntcore-arm.jar. You should be able to use the OpenCV lib found here, made by multiplemonomials https://www.dropbox.com/s/flmcrzhzjou6b9y/GRIP-linux-arm-libs.zip?dl=0.

To make these compatible with most devices, you need to deploy this file http://198.199.94.5/ntcore/libstdc++.so.6 to the same directory you deploy the grip jar to. Then, when you run GRIP, your run command needs to become env LD_LIBRARY_PATH=deploydirectory:LD_LIBRARY_PATH java -jar GRIP.jar (make sure to replace deploydirectiory with the directory you deployed to.) and then whatever other arguments you need after that.

We'd been trying to get an ntcore build that didn't require the libstdc++.so.6, however everyway we tried it didn't work. We'll keep looking, but for now it looks like thats going to be the required solution.

[ThomasJClark]

:+1:

[bathtubed]

So I got GRIP to run on my Pi 2 Jessie, however it gives me this error at startup SEVERE: NetworkTables: TCPConnector.cpp:157 select() to port 1735 error 111 - Connection refused It's also unable to capture frames off my webcam, but that could be unrelated. Is this related to a dependency issue? I can ping the address that publishAddress is set to in my project.grip settings from the pi, and the project successfully connects and publishes on my computer.

[ThadHouse]

@EdWard680 Do you have the jar you attempted to deploy? ntcore works with me manually on my Pi, but I've never been able to build grip with the jar. Don't know enough about the Java build system. That's why I was hoping Thomas could get the build system working so I could test it.

[bathtubed]

Here it is I used the method mentioned above to run it env LD_LIBRARY_PATH=.:LD_LIBRARY_PATH java -jar core-1.1.1-all.jar

[multiplemonomials]

Btw, I wrote a batch script to automate deploying the GRIP profile and starting GRIP. You guys might find it useful.

[multiplemonomials]

Also, I'm pretty sure you're getting that Connection refused error because your NetworkTables publish address is wrong in GRIP.

[bathtubed]

I wish it were wrong but I've double checked it and it's right. I can ping that address from the Pi and I can open client viewer client on my computer for that address, and the values are successfully pushed from the UI

[multiplemonomials]

Actually, you need to run the server at that address.

[multiplemonomials]

Wait, I read your post wrong. You're running the server at the publish address, right? And you can access data on it from the client?

[bathtubed]

I have a roborio running the server on my lan. I have a laptop from which I composed the test grip file on my lan. And I have the Pi 2 to which I deployed that project. I can access the data in outline viewer client on my laptop when it's running in the UI. The Pi loads the file but shows that connection refused error. I can ping the deploy address from the pi

[vScourge]

Thanks for all the efforts here! Has anyone got GRIP fully working/capturing on the Raspberry Pi 2? I think we almost have GRIP 1.1.1 running on ours (using Linux Arm) but it's failing to initialize the camera (Logitech C920). The same camera works fine on the Pi 2 using ffmpeg. I've verified it's dev/video0, and our grip file is set to use camera 0. Any suggestions?

From the .grip file:

<sources>
  <grip:Camera>
    <property name="deviceNumber" value="0"/>
  </grip:Camera>
</sources>

Errors:

Loading Dependency Injection Framework
Jan 24, 2016 9:47:30 PM java.util.logging.LogManager$RootLogger log
CONFIG: Configuration done.
Jan 24, 2016 9:47:31 PM java.util.logging.LogManager$RootLogger log
CONFIG: GRIP Version: 1.1.1
platform: /Linux/arm/
Jan 24, 2016 9:47:40 PM edu.wpi.grip.core.Main start
INFO: Loading file test2.grip
Jan 24, 2016 9:47:44 PM edu.wpi.grip.core.Source initializeSafely
WARNING: Failed to initialize CameraSource
java.io.IOException: A problem occurred trying to start the frame grabber for Webcam 0
    at edu.wpi.grip.core.sources.CameraSource.start(CameraSource.java:191)
    at edu.wpi.grip.core.sources.CameraSource.initialize(CameraSource.java:174)
    at edu.wpi.grip.core.Source.initializeSafely(Source.java:98)
    at edu.wpi.grip.core.serialization.SourceConverter.unmarshal(SourceConverter.java:60)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
    at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readItem(AbstractCollectionConverter.java:71)
    at com.thoughtworks.xstream.converters.collections.CollectionConverter.addCurrentElementToCollection(CollectionConverter.java:98)
    at com.thoughtworks.xstream.converters.collections.CollectionConverter.populateCollection(CollectionConverter.java:91)
    at com.thoughtworks.xstream.converters.collections.CollectionConverter.populateCollection(CollectionConverter.java:85)
    at com.thoughtworks.xstream.converters.collections.CollectionConverter.unmarshal(CollectionConverter.java:80)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.unmarshallField(AbstractReflectionConverter.java:480)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doUnmarshal(AbstractReflectionConverter.java:412)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.unmarshal(AbstractReflectionConverter.java:263)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
    at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:134)
    at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
    at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1206)
    at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1190)
    at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1061)
    at edu.wpi.grip.core.serialization.Project.open(Project.java:68)
    at edu.wpi.grip.core.serialization.Project.open(Project.java:62)
    at edu.wpi.grip.core.Main.start(Main.java:59)
    at edu.wpi.grip.core.Main.main(Main.java:41)
Caused by: org.bytedeco.javacv.FrameGrabber$Exception: cvCreateCameraCapture() Error: Could not create camera capture.
    at org.bytedeco.javacv.OpenCVFrameGrabber.start(OpenCVFrameGrabber.java:179)
    at org.bytedeco.javacv.FrameGrabber.restart(FrameGrabber.java:430)
    at edu.wpi.grip.core.sources.CameraSource.start(CameraSource.java:189)
    ... 28 more