[enhancement] Noetic / python3 support

[tkazik]

I recently naively tried to get a versavis running under Noetic. It turned out that the rosserial dependency of versavis in this bundle is not yet capable to handle python3. However, in the meantime, rosserial should be able to cope with noetic/python3 thanks to this and this. I tried to update the dependency with a newer version, but that did unfortunately not work out of the box...so I guess there might be a little bit (much?) more work involved. What is your guesstimate on that? Thx!

[mcamurri]

We have also investigated the problem, but we couldn't get the VersaVIS board to communicate successfully with the ADIS IMU. Here is what we have come up with:

• Operating System: Ubuntu Focal 20.04
• Architecture: amd64
• ROS version: Noetic
• flir_camera_driver branch fix/newspinnaker
• flir library version: 2.2.0.48 (see this discussion)
• versavis branch master

Modifications

• erase the rosserial dependency folder and compile rosserial with branch noetic-devel
• Modify the following line from rosserial_arduino from:

  from SerialClient import *

  to:

  from .SerialClient import *

When compiling the versavis firmware, we encountered this error:

    In file included from /home/lintong/code/versavis/firmware/libraries/ros_lib/ros.h:43:0,
                 from /home/lintong/code/versavis/firmware/versavis/versavis.ino:5:
    /home/lintong/code/versavis/firmware/libraries/ros_lib/ArduinoHardware.h: In constructor 'ArduinoHardware::ArduinoHardware()':
    /home/lintong/code/versavis/firmware/libraries/ros_lib/ArduinoHardware.h:79:19: error: cannot convert 'Uart*' to 'Serial_*' in assignment
       iostream = &Serial;

To fix that, we changed the line 56 of ArduinoHardware.h from:

#define SERIAL_CLASS Serial_

to:

#define SERIAL_CLASS Uart

• Change the run_versavis.launch file from:

  <node name="rosserial_python" pkg="rosserial_python" type="serial_node.py"
      args="_port:=/dev/versavis _baud:=250000" respawn="true" output="screen" />

  to:

  <node name="rosserial_arduino" pkg="rosserial_arduino" type="serial_node.py"
      args="_port:=/dev/versavis _baud:=250000" respawn="true" output="screen" />

After all of this, if we compile in DEBUG mode we see a constant error complaining about the IMU being inaccessible.

Irrespective of this issue, is there a recommended working combination of branches/operating system/ROS version etc. to make an ADIS IMU and a BlackFly S working together?

I'm happy to try older versions of Ubuntu for the time being, but so far we couldn't get it to work even on Melodic.

[floriantschopp]

Thanks a lot for investigating.

Removing my version of rosserial will remove the option of syncing with the host using the EKF. But that is a quite encapsulated feature that could easily be added again.

Regarding running VersaVIS under 20.04, I will look into the issue as soon as I have time and a machine with 20.04 at my disposal.

My working configuration is

• OS version: 18.04
• Architecture: amd64
• ROS version: melodic
• VersaVIS branch: master
• flir_camera_driver branch: devel/versavis
• Spinnaker SDK version: 1.13.0.31 (should come together with the above mentioned flir_camera_driver) or 1.24.0.60 (which I am using on my system)
• Arduino IDE version: 1.8.2

[mcamurri]

Thank you very much, @floriantschopp , I'll try your configuration to see if I make my setup work with it.

@tkazik if you had an already working setup with Ubuntu 18.04 you are welcome to try my fixes to make the same setup work with Ubuntu 20.04 as well.

It might be that my system doesn't work for other causes, as I couldn't make it work with 18 as well (although I didn't check I had the exact combination as @floriantschopp )

[mcamurri]

Note that the small rosserial fix for noetic has been merged yesterday: https://github.com/ros-drivers/rosserial/pull/554

[lintong-zhang]

Hi @floriantschopp what linux kernel version are you using? Spinnaker driver says use Linux kernel version 4.15.0-20 or later. We have 5.4.0-65.

[lintong-zhang]

We used an Ubuntu 18 machine and the exact same working setup as you mentioned. Versavis compiles, Arduino also has no problem, but there is problem with spinnaker sdk 1.24.0.60 (the default 1.13 installation in CMake is no longer working). When directly running roslaunch spinnaker_camera_driver camera.launch is gives the following error error.txt How did you resolve this? (Edit: after set sudo sh -c 'echo 1024 > /sys/module/usbcore/parameters/usbfs_memory_mb', the error disappears but still no image published) When trying to debug with spinView from /usr/bin/, but it is giving error ./SpinView_QT: error while loading shared libraries: libswscale-ffmpeg.so.3: cannot open shared object file: No such file or directory, and we found libswscale-ffmpeg is from Ubuntu 16 not in 18 anymore?

[mantelt]

I created a new WIP (work in progress) branch called feature/noetic/ekf_timesync (that actually just updates the submodule rosserial), that works on my computer with ROS noetic.

As far as I can tell right now, things seem to build and run. I'm not sure if the time sync is running properly though. Might need some tweaking in the initialization of the EKF (https://github.com/ethz-asl/rosserial/blob/0a345bc46b32b48b2b4c764f799549040a4e5051/rosserial_client/src/ros_lib/ros/node_handle.h#L372) or some bugfixing.

I build the VersaVis firmware using the most recent Arduino IDE (version 1.8.13) .

@floriantschopp , I guess the residual should not just steadily decrease, right?

I will continue to look into this at some point in the next couple of days / weeks.

[floriantschopp]

I finally found some time to do some tests. I used the branch ekf_evaluation and for rosserial, @mantelt's feature/noetic/ekf_timesync.

So far, after some minor retuning, it seems to work reasonably well (see figures which correspond to Fig. 8 & 9 from the original paper). Of course, convergence speed can be further be improved with tuning, but I guess for most applications this is fine. Also, more aggressive tuning would lead to sensitivity in case of a host clock jump (e.g. with GPS sync).

@mantelt Maybe we were just very unlucky with the initialization and the EKF never actually did anything in your case? If the setup is in initialization mode for a long time, this could happen I think... Maybe we should remove this condition?

BTW I used the following setup on a clean install of Ubuntu 20.04:	Software	Version
Ubuntu	20.04.2 LTS
Kernel	5.8.0-53-generic
Spinnacker SDK	2.4.0.143
flir_camera_driver	master
VersaVIS	ekf_evaluation
--> includes rosserial	feature/noetic/ekf_timesync
Arduino IDE	1.8.14
VersaVIS Board support	1.1.0

[mcamurri]

Thank you very much @floriantschopp !

The new FLIR cameras we have ordered have arrived a few days ago so I will be testing this soon (hopefully next week).

[mcamurri]

Unfortunately, it seems that the two camera + IMU setup doesn't work.

The problem we are getting is the same specified here, however the period is way below the exposure time of the camera.

Actually we know that the camera is not being triggered at all, because there is nothing being published on the topic of the non timestamped image (/usb_BFS_1)

It seems the second camera, no matter which one it is, just sits and waits for a trigger that never comes.

It's not clear whether its a problem specific to this branch or something else, but here is what we have tried:

• each camera and/or individually connected with the IMU work fine
• the board in debug mode prints the same messages for both cameras and indicate triggering is being sent
• we have made sure the correct topics are in the firmware header file
• we have checked the little switches on the board are in the correct position
• we have tried all possible port combinations on the board as well as USB ports on the computer
• the two cameras can work together on the FLIR camera viewer

Any suggestion on how to debug this?

[floriantschopp]

That is weird, would you mind having a troubleshooting session sometime early next week @mcamurri ?

[lintong-zhang]

Thank you @floriantschopp for the trouble shooting session today. Here is a summary of what we found.

First, we checked various settings and connections and they were correct, but the second camera (cam1) cannot continue produced synced image after initialisation. The E0 light (exposure) is continuously flashing but not E1 (T0, T1 are trigger signals and the LED flashing cannot be seen easily). So we used an oscilloscope to measure the triggering signal, which is Pin 2 in the camera socket (Pin4 is Ground). We could see there is a signal at 20Hz of 3.3V, this meant the versavis board is producing the right signals but the camera is not responding with the 20Hz frequency trigger.

Important note here is, all cameras require USB3 connection and dmesg will tell you what is being connected, and we are looking for new SuperSpeed Gen 1 USB device number.... The problem we have is that one of USB3 cameras gets disconnected after we plugging in the versavis usb. We are using IntensePC2 and perhaps it could not produce enough power to support these many high speed USBs.

We will switch to a NUC and reinstall all software to try again. Hopefully the USB issue would be sorted.

[lintong-zhang]

An update: after we changed to an Intel NUC, the USB problem is gone so we have a working stereo setup! We tried 3 camera setup but found one of the camera doesn't produce any image with external trigger signal (verified with a signal generator), but it works on its own. I guess this should be quite a rare problem with Flir BF S camera? We will try purchase another camera to replace.