Reduce (communcation) delay for high-dim observations (e.g. images)

[bheijden]

We lose a lot of performance in the service communication of high-dimensional observations: ~112 fps (only rendering, no sending)-> ~24 fps (only sending an empty dummy message of the required size without any rendering). If we include pybullet rendering, the fps is reduced to ~24 fps. Then, if we include the required (slow) preprocessing steps required for Pybullet (conversion of np array to list) it even reduces to ~16 fps.

• Rendering rgba & depth with resolution [1024, 544] in Pybullet excluding rest of EAGER ~112 fps.
• Single camera ([1024, 544, 4]), no actual rendering (sending empty buffer), we only achieve ~24 fps.
• Single camera ([1024, 544, 4]), with rendering (But no buffer update & without np processing in Pybullet), we only achieve ~20 fps.
• Single camera ([1024, 544, 4]), with rendering (with np processing in Pybullet), we only achieve ~16 fps.

This was run in environment with 1 UR5e and a plane object.

[AlexanderKeijzer]

Does this also happen when you send high dimensional data using topics? I can't imagine this is generally so slow in ROS.

[AlexanderKeijzer]

This seems to be python related after looking online. I found a webots issue about a similar issue. https://github.com/cyberbotics/webots_ros2/pull/160#issuecomment-707204002

https://answers.ros.org/question/208858/python-reading-messages-very-slow/

[bheijden]

Ah, so it seems that our data that we provide as a list is first converted to a numpy array which takes a considerable amount of time. In Pybullet, we actually already get a numpy array out from the render function. As of now, I am forced to convert it to a list. But apparently, this list is then again converted to a numpy array. Very inefficient...