fkie / multimaster_fkie

ROS stack with FKIE packages for multi-robot (discovering, synchronizing and management GUI)
BSD 3-Clause "New" or "Revised" License
267 stars 107 forks source link

Gateway IP on non-roscore computers #169

Closed derektan95 closed 2 years ago

derektan95 commented 2 years ago

Hi,

First of all, thanks for this amazing package. It's really useful and intuitive to use. I wanted to seek your opinions on a specific robot setup I have. Let's use the example you have on your technical report as an illustration.

image

All 3 robots are in separate subnets, and have their own roscores and corresponding gateway IPs. However, each of the subnet's roscore is not on the gateway device, but on another computer that is linked to the same roscore in the same subnet. Both master_discovery and master_sync nodes are located on the computer where the roscore is located (not where gateway is located).

For my use case, I have a external PC that is connected to each of the 3 gateway devices (not hosting roscore). I would simply need to subscribe / publish to the ROS topics on these gateway devices, and wouldn't involve the PCs with the roscores.

1) Is this an efficient use of bandwidth, given that our gateway IP and roscore are on separate computers? Will subscribed topics have to move from gateway computer --> roscore computer --> gateway IP before it is being sent out to the subscriber (and vice versa for publisher), or will it be directly gateway computer --> gateway IP --> subscriber subnet?

2) Are there any other unforseen issues with this setup? From this post, it seems that I would need to the sync_remote_nodes parameters?

Thanks alot!

atiderko commented 2 years ago

Hi, thank you, i am glad if the package is helpful for you! The master_dsicovery node causes a constant load on the network (some bytes/sec) by sending the heartbeat messages for the discovery. The load of the master_sync depends on changes in the ROS-Network (start/stop of nodes or create/destroy topics/services). In this case the registration information are get from remote master_discovery and registered on local roscore.

  1. The topics (subscriber and publisher) are then connected by roscore and the data should be send to subscriber directly. Without going through roscore or one of the multimaster nodes. (computer --> gateway IP --> subscriber subnet)
  2. If you run roscore (with master_discovery and master_sync) on each host you use ROS-node, you don't need to change the sync_remote_nodes parameter ;-). Beside this all hostnames on all host should be resolvable or you set ROS_IP environment variable. Otherwise ROS will try to resolve the name and if it does not work, the messages will not be delivered.
  3. Try to avoid syncing the / tf topic! this causes a lot of traffic.
derektan95 commented 2 years ago

Hi @atiderko,

Thanks for your prompt response. What you said made perfect sense, so I am going to go ahead with having roscores and gateway IPs hosted on different computers on each of the subnet.

Unfortunately, I'm having some issues with this setup. I currently have a laptop connected to a radio, which is then connected to another radio within the same subnet. This other radio is connected to a subnet with 2 PCs.

IMG20211014184700

I've followed your technical report very closely to ensure that I've done everything I should have. The routing commands can be seen in the image above. I've also made sure to enable multicasting and IP forwarding on all of the computers.

From my laptop (roscore at 10.168.1.102), I am able to ping PC2 (roscore at 192.168.123.12). However, I can't ping in the opposite direction. In fact, I've issues pinging from 192.168.123.12 to the radio at 10.168.1.4 (I can ping 10.168.1.104).

Here are my questions: 1) Do we need to be able to ping in both directions before we can have the master_discovery nodes find each other? I attempted but they can't locate each other.

2) if it's necessary, do you have any insights on how I can ping from 192.168.123.12 to 10.168.1.2?

Alternatively, I can consider moving the roscore from 192.168.123.12 to PC1 @ 192.168.123.98. They'll be able to discover each other then. Looking forward to hearing from you!

p.s. there shouldn't be any firewalls set

Thanks alot, Derek

derektan95 commented 2 years ago

Ah, after testing this out more, I think it's all working now. Thanks again for this wonderful package!

derektan95 commented 2 years ago

Hi @atiderko,

May I know why you mentioned that:

Try to avoid syncing the / tf topic! this causes a lot of traffic.

/tf topic seems rather lightweight. Is it due to the fact that some tf publishers are publishing at very high rates?

atiderko commented 2 years ago

Hi @derektan95 ,

this is because of the high data rates and because of the NxN relationship with TCP unicast connections. This is also true for every ROS topic, but hardly any other topic has so many subscribers/publishers.

derektan95 commented 2 years ago

Hi @atiderko,

That's great! Thanks for the explanation!