fujitatomoya / ros_k8s

Kuberenetes / ROS&ROS2 Cluster Samples
Creative Commons Attribution 4.0 International
192 stars 29 forks source link

Add example deployment with Data Sharing via LoanedMessage #31

Closed fujitatomoya closed 10 months ago

fujitatomoya commented 1 year ago

I had this question during ROSCon 2023 presentation. I believe that this is one of the use case to show how we can use the data sharing zero copy even with container and Kubernetes. Certain constraints that we need to assign the containers in the same pod as localhost, but for the explanation this makes sense to add the example here.

fujitatomoya commented 10 months ago

export ROS_DISABLE_LOANED_MESSAGES=0 this needs to be done, since https://github.com/ros2/rcl/pull/1110

some experimental check... using docker container.

tomoyafujita@~/DVT >docker run -it --network host -v /dev/shm:/dev/shm tomoyafujita/ros:rolling

root@tomoyafujita:~# ls -lt /dev/shm/
total 0

root@tomoyafujita:~# ros2 daemon status
The daemon is not running

root@tomoyafujita:/# ros2 run demo_nodes_cpp talker_loaned_message
[INFO] [1703119123.500535244] [loaned_message_talker]: Publishing: '1.000000'
[INFO] [1703119123.500835352] [rclcpp]: Currently used middleware can't loan messages. Local allocator will be used.
[INFO] [1703119123.500871185] [loaned_message_talker]: Publishing: 'Hello World: 1'
[INFO] [1703119124.500472323] [loaned_message_talker]: Publishing: '2.000000'
[INFO] [1703119124.500608551] [loaned_message_talker]: Publishing: 'Hello World: 2'
...
tomoyafujita@~/DVT >docker run -it --network host -v /dev/shm:/dev/shm tomoyafujita/ros:rolling

root@tomoyafujita:/# ros2 node list

root@tomoyafujita:/# ros2 topic list
/chatter
/chatter_pod
/parameter_events
/rosout

interesting ths case, we cannot see the nodes but topic list. i think Simple Participant Discovery Protocol (SPDP) uses shm transport, so that participants cannot discover each other. on the other hand, Simple Endpoint Discovery Protocol (SEDP) falls back to multicast based default discovery, that can discover the DataReader and DataWriter, so we can see the topics.

https://github.com/eProsima/Fast-DDS/pull/3753 related to this behavior, see more details for https://fast-dds.docs.eprosima.com/en/latest/fastdds/discovery/simple.html#simple-discovery-settings

Note:

root@da343f3b6b7d:/# fastdds shm clean
shm.clean:
0 ports in use
0 segments in use
3 zombie ports cleaned
2 zombie segments cleaned
fujitatomoya commented 10 months ago

closing in favor of https://github.com/fujitatomoya/ros_k8s/pull/33