Implement downlink with multicast address

[thanasipantazides]

Why

To send data to the ground we have two routes:

1. a direct Ethernet connection while on the launch rail,
2. a radio transmitter (used some of the time on launch rail, and once flying).

The transmitter will only listen to multicast UDP packets. But when the GSE is physically connected to the Formatter, it will be able to listen directly to the Formatter and indirectly through the radio. We should make sure we have a predictable way to handle duplicate packets (from the radio and from the direct connection) in the GSE—a mode that changes once the radio turns on, or something.

On the ground side, I think we should have two Logger instances running—one listening to the radio, and one listening directly to the Formatter over physical connection. Each will output a different log file. The GSE software can include a toggle to read data from either file (activate when radio turns on).

To do

• [ ] Pick a multicast address and port to use. I think the address should be routable and not commonly used for other things.
• [ ] Decide whether to give different address/port to the GSE interface than the radio interface. Kris may have ideas about this. Is it better to just use a unicast/standard IP address for the physical GSE connection?
• [ ] Create a 3-machine setup to test (can write these in Python, see examples in /util:
  • [x] One multicast talker (1) outputs arbitrary packets (/util/udpgarbage.py does something like this with random data).
  • [x] One listener (2) receives packets and forwards them to the third machine.
  • [x] One listener (3) receives packets from both (1) and (2) and prints them or something (/util/printudp.py does the printing for a single sender).
• [ ] Open an issue in the GSE repository to implement some (port-based or IP-based) downlink log file filter that is tied to the SystemConfiguration.flight global setting.

[yixianz]

I don't think duplicate packets would be a problem. I did some tests with one or two laptops - with only one laptop, I just opened different terminal windows for sending and receiving separately; with two laptops in the same local network (e.g. my home Wifi), I could send messages from one to the other. Below are some example scripts that I used and the terminal outputs.

Simple sender and receiver

Starting from something very basic in updgarbage.py, I randomly generate 10 bytes of garbage data every second and send them out by multicasting. Here I'm not mimicking the 18 Mbps downlink rate because I want outputs to be short and clean in my terminal, and I've tested that such rate would work just the same.

# udpgarbage.py
import socket, time, random

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)

# multicast group address and port
mcast_grp = '224.1.1.0'
mcast_port = 3000

while True:
    # make some random data to transmit
    data = random.randbytes(10)
    print("sending data to multicast group " + mcast_grp + ":" + str(mcast_port))
    sock.sendto(data, (mcast_grp, mcast_port))
    time.sleep(1) 

On the receiver side, printudp.py looks like:

# printudp.py
import socket, struct

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

# multicast group address and port
mcast_grp = '224.1.1.0'
mcast_port = 3000

# bind socket to mcast_grp on mcast_port
sock.bind((mcast_grp, mcast_port))
# convert multicast address to binary form, INADDR_ANY means any interface
mreq = struct.pack("4sl", socket.inet_aton(mcast_grp), socket.INADDR_ANY)
# join the multicast group (adding membership)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)

while True:
    data, sender_endpoint = sock.recvfrom(1500)  #receive data
    print(str(sender_endpoint[0]) + ":" + str(sender_endpoint[1]) + " sent" + data.hex())

Testing with one laptop (running udpgarbage.py and printudp.py in two separate terminal windows), I get:

% python udpgarbage.py
sending data to multicast group 224.1.1.0:3000
sending data to multicast group 224.1.1.0:3000
sending data to multicast group 224.1.1.0:3000
sending data to multicast group 224.1.1.0:3000
sending data to multicast group 224.1.1.0:3000

% python printudp.py
192.168.1.149:52535 sent 1d14ac7bd5f1e26db5cc
192.168.1.149:52535 sent 2f09ca6f06c192cf3794
192.168.1.149:52535 sent 80ff21e931dc64c8da68
192.168.1.149:52535 sent 506c10a97c20e56c6ac1
192.168.1.149:52535 sent 740d1a72300e015e1a26

Note that in those scripts I don't explicitly write out the standard or loopback IP of my computer - I just tell the sender and receiver to both use the multicast address (i.e. "224.1.1.0") and it works. This is also true when I try multicasting from one computer to another.

Ways to avoid duplicate packets

Using different multicast groups/ports

If there are two routes from the sender to the receiver, one way to avoid duplicate packets is to use different multicast addresses and/or ports for each route. Then you can bind your receiver's socket only to the address and port that you want.

For example, on the sender side, this time I create two multicast groups "224.1.1.0" and "224.1.1.1" both with port 3000. (Using the same group but changing the port would also work.) I send random bytes again through "224.1.1.0" and send "0x01" repeatedly through "224.1.1.1".

# udpgarbage2.py (two groups)
import socket, time, random

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)

# multicast group address and port
mcast_grp1 = '224.1.1.0'
mcast_grp2 = '224.1.1.1'
mcast_port = 3000

while True:
    # make some random data to transmit in group 1
    data = random.randbytes(10)
    # send 0x01 in group 2
    data2 = bytes([0x01])
    print("sending data to multicast group " + mcast_grp1 + ":" + str(mcast_port))
    sock.sendto(data, (mcast_grp1, mcast_port))
    print("sending data to multicast group " + mcast_grp2 + ":" + str(mcast_port))
    sock.sendto(data2, (mcast_grp2, mcast_port))
    time.sleep(1) 

On the receiver side, I use the same printudp.py script as in the simple example. As expected, I only receive the random garbage data because the receiver joins group 1.

% python udpgarbage2.py
sending data to multicast group 224.1.1.0:3000
sending data to multicast group 224.1.1.1:3000
sending data to multicast group 224.1.1.0:3000
sending data to multicast group 224.1.1.1:3000
sending data to multicast group 224.1.1.0:3000
sending data to multicast group 224.1.1.1:3000

% python printudp.py
192.168.1.149:50099 sent 5a1d16e59975a494cb54
192.168.1.149:50099 sent 3fcf8bb4414b8ef026a0
192.168.1.149:50099 sent 78abaa589a9563a2af0c

If I change mcast_grp = '224.1.1.0' to mcast_grp = '224.1.1.1' in printudp.py, I get:

% python printudp.py
192.168.1.149:52122 sent 01
192.168.1.149:52122 sent 01
192.168.1.149:52122 sent 01

because now the receiver only listens to group 2.

The receiver can listen to more than one multicast groups if need. If you want the socket to be reachable by any address that the machine happens to have, you can do sock.bind(('', mcast_port)) instead of sock.bind((mcast_grp, mcast_port)).

Filtering based on interface IPs

Alternatively, even using just one multicast group and port, the receiver can do some filtering based on the interface IPs (I haven't tested this though).

In the above printudp.py example, when adding membership I do:

mreq = struct.pack("4sl", socket.inet_aton(mcast_grp), socket.INADDR_ANY)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)

where INADDR_ANY means that it would listen to all the interfaces. We can change that parameter to a specific IP to only listen to packets from that specific interface. This will be something like:

sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, socket.inet_aton(mcast_grp) + socket.inet_aton(ip_interface))

Forwarding example

I did a forwarding test with two computers using my home WiFi, which is very similar to the 3-machine test but using one of the computers as both the sender and the receiver (with different terminal windows). So data can be transmitted via two routes: computer A (sender) --> computer A (receiver), or computer A (sender) --> computer B (redirector) --> computer A (receiver).

I again use the simple sender and receiver scripts, while the redirector is essentially a combination of receiver and sender but with a port number incremented by 1 (same multicast address).

# udpredirect.py
import socket, time, random, struct

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

# multicast group address and port
mcast_grp = '224.1.1.0'
mcast_port = 3000

# I have to bind the socket to all multicast groups instead of the specific mcast_grp to get it to work. Not sure why.
sock.bind(('',mcast_port))
# convert multicast address to binary form, INADDR_ANY means any interface
mreq = struct.pack("4sl", socket.inet_aton(mcast_grp), socket.INADDR_ANY)
# join the multicast group
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)

while True:
    # receive data from source
    data, sender_endpoint = sock.recvfrom(1500)
    # data will be redirected to the same multicast address but with a port number incremented by 1
    addr_redirect = (mcast_grp, mcast_port+1)
    print("Redirecting data from "+ str(sender_endpoint) + " to " + str(addr_redirect))
    # forward data
    sent = sock.sendto(data, addr_redirect)

Now the outputs look like: Sender:

% python udpgarbage.py
sending data to multicast group 224.1.1.0:3000
sending data to multicast group 224.1.1.0:3000
sending data to multicast group 224.1.1.0:3000
sending data to multicast group 224.1.1.0:3000

Redirector:

% python udpredirect.py
Redirecting data from ('192.168.1.149', 59965) to ('224.1.1.0', 3001)
Redirecting data from ('192.168.1.149', 59965) to ('224.1.1.0', 3001)
Redirecting data from ('192.168.1.149', 59965) to ('224.1.1.0', 3001)
Redirecting data from ('192.168.1.149', 59965) to ('224.1.1.0', 3001)

Receiver, if mcast_port = 3000 in printudp.py:

% python printudp.py
192.168.1.149:56532 sent 6131211b134ed10b5a0c
192.168.1.149:56532 sent d74a9fac7a0b91105e34
192.168.1.149:56532 sent 2ed95b3dfa530b233945
192.168.1.149:56532 sent c6d3e9708fb54fb3b538

If mcast_port = 3001 inprintudp.py:

% python printudp.py
192.168.1.196:3000 sent 43061d9b3d5f983f8115
192.168.1.196:3000 sent 8cb6ddf754a5be3e91cd
192.168.1.196:3000 sent 017e9cac62c2c99a64de
192.168.1.196:3000 sent e02d9402d8b9dae0524f

I didn't get any duplicate packets because the receiver can choose which route it wants to receive data from.

[thanasipantazides]

Notes to self for implementation:

• Want to infer endpoint type from systems.json.
• In Python, use ipaddress.is_multicast(), then open socket accordingly.
• In C++, use boost::asio::ip::address::is_multicast(), after constructing from ::make_address(std::string address).
  • See this asio example and this post.

[thanasipantazides]

In the above boost::asio example, sending to a multicast address is as simple as:

1. creating an address with ::make_address(), supplying a multicast address as std::string argument,
2. sending to the endpoint.

This is the exact same workflow as sending to a single endpoint.

It will be a little trickier to get the Listener set up correctly in the GSE, but that is in Python and will be quick to debug. Should be possible to determine address type from provided JSON, and conditionally open the correct type of socket and bind or join.

[thanasipantazides]

Closing, this was resolved on the Formatter side by edits in foxsi4-commands.