search

roc-streaming / roc-toolkit

Real-time audio streaming over the network.
https://roc-streaming.org
Mozilla Public License 2.0
1.06k stars 213 forks source link

Add multicast support #247

Closed dshil closed 4 years ago

dshil commented 5 years ago

253

gavv commented 5 years ago

It would be nice if you explain these changes in the commit messages. It would be also nice to add the PR number to the commit title (since we have no github issue for this).

gavv commented 5 years ago

A few questions:

gavv commented 5 years ago

Do you have examples of other software that automatically join/leave multicast group if the address is multicast? Is it a common practice?

gavv commented 5 years ago

May the user want to control the interface on which to join the group?

dshil commented 5 years ago

Do you have examples of other software that automatically join/leave multicast group if the address is multicast?

vlc

Is it a common practice?

I guess yes since I saw this in 2 projects (vlc is one of them)

dshil commented 5 years ago

May the user want to control the interface on which to join the group?

Well, there are 2 most common use cases of multicast:

I am not sure if the user even want to care about join/leave mechanism. All the user wants to do is send data over the network.

dshil commented 5 years ago

A few questions:

* What happens if a single process joins the same group multiple times?

10:15:03.324 [err] roc_netio: udp receiver: uv_udp_set_membership(): [EADDRINUSE] address already in use

* What happens if multiple processes join the same group?

If more than one process joins the same multicast group on the same interface, they will all receive the datagrams sent to that group via that interface. The host remains member of the group, until all the processes decide to leave the group, a.k.a. ref counting mechanism. The funny thing that if we leave the multicast group but keep binding to the socket and there is at least one process that already has joined to the same multicast group we will still receive multicast traffic.

* What happens if the process crashes without leaving a group?

I tried to kill roc via SIGKILL and after that check if multicast group still exists with netstat -gn. The group was released.

We can even omit the leave_multicast_group() call and when process finishes the group will be released (if it is not used by other processes).

gavv commented 5 years ago

10:15:03.324 [err] roc_netio: udp receiver: uv_udp_set_membership(): [EADDRINUSE] address already in use

How is it reproduced?

I just checked out your branch and run this command:

roc-recv -vv -s rtp+rs8m:224.0.0.1:10001 -r rs8m:224.0.0.1:10002 -s rtp+ldpc:224.0.0.1:10003 -r ldpc:224.0.0.1:10004

The receiver joined the group four times and I got no error.

gavv commented 5 years ago

If more than one process joins the same multicast group on the same interface, they will all receive the datagrams sent to that group via that interface. The host remains member of the group, until all the processes decide to leave the group, a.k.a. ref counting mechanism. The funny thing that if we leave the multicast group but keep binding to the socket and there is at least one process that already has joined to the same multicast group we will still receive multicast traffic.

Great.

I tried to kill roc via SIGKILL and after that check if multicast group still exists with netstat -gn. The group was released.

Great.

We can even omit the leave_multicast_group() call and when process finishes the group will be released (if it is not used by other processes).

We shouldn't: the user may want to unbind port without terminating the process. We don't have unbind() in the API so far, but we'll add it later.

gavv commented 5 years ago

I am not sure if the user even want to care about join/leave mechanism. All the user wants to do is send data over the network.

I think the user may want to control the interface for security reasons. Imagine that the user has two interfaces: one "public", connected to the Internet, and one "private", connected to the private network. Then the user starts Roc receiver and wants to ensure that Roc will accept only packets from the private network and will be protected (at the kernel level) from the Internet.

In case of unicast, the user just specifies the address of the private interface as the bind address, and it works as expected.

But in case of multicast, if I understand correctly, your implementation will automatically join the group on all interfaces and in our example it will be vulnerable to packets coming from the public interface? Is it true? If you're not sure, could you please test it?

If it's true, I think we need a way to restrict the interface. BTW, it seems that VLC has --miface option for that.

We could also add such an option, but it would be better to be able to control the interface on per-port basic, so that you can bind some ports to one interface, and some ports to another.

To solve this, we could implement a syntax like:

roc-recv -vv -s rtp+rs8m:224.0.0.1@eth0:10001 -r rs8m:224.0.0.1@eth0:10002

It should be also possible to specify interface address instead of interface name, like:

roc-recv -vv -s rtp+rs8m:224.0.0.1@192.168.0.101:10001 -r rs8m:224.0.0.1@192.168.0.101:10002

In case of unicast, the @... part should be omitted. In case of multicast, the @... part likely should be strictly required for security reasons.

It should be also possible to explicitly request to join group on all interfaces, something like:

rtp+rs8m:224.0.0.1@0.0.0.0:10001
dshil commented 5 years ago

10:15:03.324 [err] roc_netio: udp receiver: uv_udp_set_membership(): [EADDRINUSE] address already in use

How is it reproduced?

I just checked out your branch and run this command:

roc-recv -vv -s rtp+rs8m:224.0.0.1:10001 -r rs8m:224.0.0.1:10002 -s rtp+ldpc:224.0.0.1:10003 -r ldpc:224.0.0.1:10004

The receiver joined the group four times and I got no error.

I was wrong. I got this error because I tried to join to the multicast group on the same interface + on the same port.

What happens if a single process joins the same group multiple times?

I guess that nothing special. We will only see one record in /proc/net/igmp

dshil commented 5 years ago

We shouldn't: the user may want to unbind port without terminating the process. We don't have unbind() in the API so far, but we'll add it later.

OK. As I wrote previously, in case of multicast group we will have some kind UB for the user who is not well-informed about multicast usage. If you unbind the port but still there is at least one process joined to this group all previously joined processes will still receive the multicast traffic (even after unbind call).

I guess that providing the unbind API is not enough. Lets keep in mind it.

dshil commented 5 years ago

I am not sure if the user even want to care about join/leave mechanism. All the user wants to do is send data over the network.

I think the user may want to control the interface for security reasons. Imagine that the user has two interfaces: one "public", connected to the Internet, and one "private", connected to the private network. Then the user starts Roc receiver and wants to ensure that Roc will accept only packets from the private network and will be protected (at the kernel level) from the Internet.

In case of unicast, the user just specifies the address of the private interface as the bind address, and it works as expected.

But in case of multicast, if I understand correctly, your implementation will automatically join the group on all interfaces and in our example it will be vulnerable to packets coming from the public interface? Is it true? If you're not sure, could you please test it?

You are right.

I join to the multicast group as:

bool UDPReceiverPort::join_multicast_group_() {
    if (int err = uv_udp_set_membership(
            &handle_, packet::ip_to_str(address_).c_str(), NULL, UV_JOIN_GROUP)) {
        roc_log(LogError, "udp receiver: uv_udp_set_membership(): [%s] %s",
                uv_err_name(err), uv_strerror(err));
        return false;
    }

    return (multicast_group_joined_ = true);
}

And if we look into the libuv code we see the following:

UV_EXTERN int uv_udp_set_membership(uv_udp_t* handle,
                                    const char* multicast_addr,
                                    const char* interface_addr,
                                    uv_membership membership);

static int uv__udp_set_membership4(uv_udp_t* handle,
                                   const struct sockaddr_in* multicast_addr,
                                   const char* interface_addr,
                                   uv_membership membership) {
  struct ip_mreq mreq;
  int optname;
  int err;

  memset(&mreq, 0, sizeof mreq);

  if (interface_addr) {
    err = uv_inet_pton(AF_INET, interface_addr, &mreq.imr_interface.s_addr);
    if (err)
      return err;
  } else {
    mreq.imr_interface.s_addr = htonl(INADDR_ANY);
  }

  // ...
}

As you might see we join to the multicast group on all available interfaces.

dshil commented 5 years ago

It would be nice if you explain these changes in the commit messages. It would be also nice to add the PR number to the commit title (since we have no github issue for this).

I extracted it into the separate issue.

gavv commented 5 years ago

We shouldn't: the user may want to unbind port without terminating the process. We don't have unbind() in the API so far, but we'll add it later.

OK. As I wrote previously, in case of multicast group we will have a some kind UB for the user who is not well-informed about multicast usage. If you unbind the port but still there is at least one process joined to this group all previously joined processes will still receive the multicast traffic (even after unbind call).

I guess that providing the unbind API is not enough. Lets keep in mind it.

I don't get it. If our unbind() implementation will leave the group and then close the socket, what problem exactly do we have?

gavv commented 5 years ago

What happens if a single process joins the same group multiple times?

I guess that nothing special. We will only see one record in /proc/net/igmp

Is reference counting working as expected inside a single process too? I mean if a single process joins the same group two times, it should leave it two times, not a single time, right?

gavv commented 5 years ago

I think we also need an option not to join any group at all, if the user wants to control this manually.

Not sure about the syntax. One possibility is to omit @... part, but it probably will be confusing, because most users will try it first and at the same time they are not aware that they need to join/leave manually. So probably if the @... is omitted, we should instead provide a meaningful error message, and add some special syntax to disable auto-join/leave.

One idea is it use empty interface name for this:

rtp+rs8m:224.0.0.1@:10001
dshil commented 5 years ago

We shouldn't: the user may want to unbind port without terminating the process. We don't have unbind() in the API so far, but we'll add it later.

OK. As I wrote previously, in case of multicast group we will have a some kind UB for the user who is not well-informed about multicast usage. If you unbind the port but still there is at least one process joined to this group all previously joined processes will still receive the multicast traffic (even after unbind call). I guess that providing the unbind API is not enough. Lets keep in mind it.

I don't get it. If our unbind() implementation will leave the group and then close the socket, what problem exactly do we have?

I missed that you mentioned that unbind will also leave a group. I didn't assume it (don't know why). Never mind.

gavv commented 5 years ago

I missed that you mentioned that unbind will also leave a group. I didn't assume it (don't know why). Never mind.

Actually unbind() should close and remove the port. It's an opposite of adding port. It's called unbind only because "adding port" is called "bind" in terms of our C API.

dshil commented 5 years ago

What happens if a single process joins the same group multiple times?

I guess that nothing special. We will only see one record in /proc/net/igmp

Is reference counting working as expected inside a single process too? I mean if a single process joins the same group two times, it should leave it two times, not a single time, right?

Correct

dshil commented 5 years ago

As discussed privately we won't introduce per port multicast interface configuration due to 4fc2b5723e7ad28b331514cf50f86c055606d51c.

New option was added to roc-recv, --miface, that allows to configure multicast interface for both source and repair ports:

roc-recv -s rtp:225.1.2.3:10001 --miface ffee::
roc-recv -s rtp:[ffee::]:10001 --miface 0.0.0.0
roc-recv -s rtp:192.169.18.1:10001 --miface 0.0.0.0
dshil commented 5 years ago

Due to #258 we need to discuss how we will merge address to string formatting. Now I use the old variant of the network address format:

16:34:34.788 [inf] roc_pipeline: receiver: adding port rtp+rs8m:225.1.2.3@0.0.0.0:10001
16:34:34.788 [dbg] roc_netio: udp receiver: joined multicast group 225.1.2.3 at interface 0.0.0.0
dshil commented 5 years ago

@gavv,

Could you please test ipv6 multicast?

roc-recv -s rtp+rs8m:\[ff00::\]:10001 -r rs8m:\[ff00::\]:10002 --miface \:\: -vv
16:42:28.475 [dbg] roc_netio: udp receiver: joined multicast group ff00:: at interface ::
16:42:28.475 [inf] roc_netio: udp receiver: opened port [ff00::]@[::]:10001
16:42:28.475 [inf] roc_pipeline: receiver: adding port rtp+rs8m:[ff00::]@[::]:10001
16:42:28.475 [dbg] roc_netio: udp receiver: joined multicast group ff00:: at interface ::
16:42:28.475 [inf] roc_netio: udp receiver: opened port [ff00::]@[::]:10002
16:42:28.475 [inf] roc_pipeline: receiver: adding port rs8m:[ff00::]@[::]:10002
arch% netstat -gn
IPv6/IPv4 Group Memberships
Interface       RefCnt Group
--------------- ------ ---------------------
wlp2s0          2      ff00::
roc-send -s rtp+rs8m:\[ff00::\]:10001 -r rs8m:\[ff00::\]:10002 -i loituma.wav -vv

And I get the silence...

gavv commented 5 years ago

Could you please test ipv6 multicast?

This works for me:

roc-send -vv -s rtp+rs8m:[ff03::1]:10001 -r rs8m:[ff03::1]:10002 -i stash/loituma.wav
roc-recv -vv -s rtp+rs8m:[ff03::1]:10001 -r rs8m:[ff03::1]:10002 --miface ::

ff00:: is probably reserved. I don't have much experience with IPv6.

gavv commented 5 years ago

Due to #258 we need to discuss how we will merge address to string formatting. Now I use the old variant of the network address format:

https://github.com/roc-project/roc/pull/247#discussion_r332086338