Enhance camera features, stability and performance

lukicdarkoo commented 4 years ago

Description Use GPU to perform YUV422 to RGB conversion and publish camera_info topic.

Related Issues Fixes #15

Tasks

[x] Utilise GPU for YUV422 to perform RGB conversion
[x] Create a stable way to handle switching between compressed, raw and none
[x] Delete OpenCV dependecy
[x] Perform a stress test
[x] Parametrise output image resolution
[x] Add camera_info topic
[x] Estimate camera intrinsic parameters
[x] Investigate performances of RGB raw topic

Additional context camera_info requires a camera calibration with a checkboard (for calibration matrix, with Webots it was easy, the focal length is known and there are no distortions). Since Corona forbids me from going to EPFL to print the checkboard it may take a while. Therefore, I will estimate parameters now and calibrate them as soon as I print the checkboard (open a separated issue).

lukicdarkoo commented 4 years ago

Changing output encoding to RGB24 or any other RGB variation (RGBA, BGR...) is failing: https://github.com/cyberbotics/epuck_ros2/blob/bfd303751e2645344687dcb7280cf3a29bdbd7b7/epuck_ros2_camera/src/pipuck_jpeg.c#L82

Looking for the workaround...

lukicdarkoo commented 4 years ago

This is great, I have just managed to do it with vc.ril.isp component. It seems we can use it in parallel with compression. Even better, vc.ril.isp has two outputs, so we could potentially serve two resolutions simultaneously.

lukicdarkoo commented 4 years ago

Successfully publishes camera images to two topics simultaneously

lukicdarkoo commented 4 years ago

Camera info data is read from a YAML file (with a default one provided with the package), so it can be easily modified in the future.

lukicdarkoo commented 4 years ago

Many very minor polishing comments but the general design is very good. Have you been able to measure quantitatively a performance improvment?

JPEG performance should not be impacted. I will check for raw RGB since we were getting only like 4fps onboard it should be much better now.

Update 1

Unfortunately, there are no improvements, it's about the same. This is strange. Investigating...

Update 2

I suspected that copying an array in the following line takes a lot of time: https://github.com/cyberbotics/epuck_ros2/blob/699228187cb07b922059b38c268f637c4396aeb9/epuck_ros2_camera/src/camera.cpp#L148 so I changed it, but the results are about the same (3.5fps - 4fps).

Update 3

Looking into ROS2 middlewares if it makes sense changing them. Default one, eProsima’s FastRTPS doesn't support shared memory: https://discourse.ros.org/t/is-there-any-dds-implementation-using-shared-memory-inside-system/7609/2 It means it sends the whole image through the "network". On the other hand, RTI Connext DDS supports the shared memory, but it doesn't work on ARM32 platform: https://github.com/ros-infrastructure/rep/blob/master/rep-2000.rst#eloquent-elusor-november-2019---november-2020 Note that eProsima’s FastRTPS and RTI Connext DDS are Tier 1 ROS2 middlewares.

Update 4

This is a CPU usage:

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                                  
 2165 pi        20   0  157404  30640  22620 S  44.7   8.1  14:47.04 camera                                                                                                                                   
 2249 pi        20   0  165436  60328  26908 R  41.4  16.0   0:52.28 ros2

in which camera is our camera node and ros2 is a process fired by ros2 topic hz image_raw to measure the publishing rate.

Update 5

It seems there are a lot of message drops because it takes less than 100ms to convert an image and execute publisher->publish(msg): More details on the following link: https://github.com/lukicdarkoo/epuck-ros2-various-analyses/blob/master/UnderhoodCameraPerfomances.ipynb

cyberbotics / epuck_ros2