After upgrade to 2.5, suddenly getting error 1007 for system behind router (worked with 2.4)

[lightweight]

Recently ran scripts/upgrade and then the newer scripts/upgrade-and-build script which ran without issue except that I had to apply the fix described in #226 to get MediaSoup to build (I specified version 3.10.3). This is running on a Docker host behind a router, which is forwarding all the relevant ports for COTURN, Greenlight and BBB (this same configuration was running happily using the same configuration via the 2.4 Docker install prior to the upgrade).

After the upgrade, all the containers run without any obvious error, but any attempt to connect the mike or video results in a 1007 error, and, in the Javascript console and error of WebRTC: ICE failed, your TURN server appears to be broken, see about:webrtc for more details.

Looking at the BBB issue queue, I note https://github.com/bigbluebutton/bigbluebutton/issues/14460 and investigated how these services are configured via the Docker install. Eventually, found that they are configured via MS_WEBRTC_LISTEN_IPS and MS_RTP_LISTEN_IP specified in the docker-compose.yml based on the template and the .env file value for EXTERNAL_IPv4. I also note the following comments directly above this in the template:

# TODO: add mediasoup IPv6
# TODO: can listen to 0.0.0.0 for nat support? https://github.com/versatica/mediasoup/issues/487

I don't have any IPv6 in this environment, so that's not an issue (unless a blank IPv6 external address creates issues!), but I'm wondering if this configuration accommodate an installation of Mediasoup behind a router (where the server IP isn't on the same network as the external IP)?

Is it possible that, given I'm using Mediasoup 3.10.3 that I might be running into issues related to https://github.com/bigbluebutton/bigbluebutton/issues/15472?

As a short term fix, can I disable Mediasoup and revert to the previous solution?

[qubex22]

It looks like mediasoup works differently than kurento. The deafult configuration doesn't work with NAT and the notes here https://github.com/bigbluebutton/docker/blob/develop/docs/behind-nat.md are outdated. Check my workaround here https://github.com/bigbluebutton/docker/issues/222 I also opened a pull request to modify the NAT notes https://github.com/bigbluebutton/docker/pull/223 I got it fully working this way, however the devs should address the NAT problem

[ffdixon]

Thanks for the feedback. Will let @alangecker take a quick look at the pull request and merge if it looks good.

[lightweight]

We've also managed to fix this. We specified a newer version of MediaSoup (3.10.5) to allow the containers to build successfully, and tweaked the scripts. Here's our diff:

diff --git a/docker-compose.tmpl.yml b/docker-compose.tmpl.yml
index a65ec62..08c0f88 100644
--- a/docker-compose.tmpl.yml
+++ b/docker-compose.tmpl.yml
@@ -119,6 +119,7 @@ services:
     environment:
       DOMAIN: ${DOMAIN}
       EXTERNAL_IPv4: ${EXTERNAL_IPv4}
+      ANNOUNCED_IPv4: ${ANNOUNCED_IPv4}
       EXTERNAL_IPv6: ${EXTERNAL_IPv6:-::1}
       SIP_IP_ALLOWLIST: ${SIP_IP_ALLOWLIST:-}
       DISABLE_SOUND_MUTED: ${DISABLE_SOUND_MUTED:-false}
@@ -236,11 +237,11 @@ services:
       # TODO: add mediasoup IPv6
       # TODO: can listen to 0.0.0.0 for nat support? https://github.com/versatica/mediasoup/issues/487
     {{ if .Env.EXTERNAL_IPv6 }}
-      MS_WEBRTC_LISTEN_IPS: '[{"ip":"{{ .Env.EXTERNAL_IPv6 }}", "announcedIp":"{{ .Env.EXTERNAL_IPv6 }}"}, {"ip":"${EXTERNAL_IPv4}", "announcedIp":"${EXTERNAL_IPv4}"}]'
+      MS_WEBRTC_LISTEN_IPS: '[{"ip":"{{ .Env.EXTERNAL_IPv6 }}", "announcedIp":"{{ .Env.EXTERNAL_IPv6 }}"}, {"ip":"${EXTERNAL_IPv4}", "announcedIp":"${ANNOUNCED_IPv4}"}]'
     {{else}}
-      MS_WEBRTC_LISTEN_IPS: '[{"ip":"${EXTERNAL_IPv4}", "announcedIp":"${EXTERNAL_IPv4}"}]'
+      MS_WEBRTC_LISTEN_IPS: '[{"ip":"${EXTERNAL_IPv4}", "announcedIp":"${ANNOUNCED_IPv4}"}]'
     {{end}}
-      MS_RTP_LISTEN_IP: '{"ip":"0.0.0.0", "announcedIp":"${EXTERNAL_IPv4}"}'
+      MS_RTP_LISTEN_IP: '{"ip":"0.0.0.0", "announcedIp":"${ANNOUNCED_IPv4}"}'
     volumes:
       - vol-mediasoup:/var/mediasoup
     tmpfs:
diff --git a/mod/freeswitch/conf/vars.xml.tmpl b/mod/freeswitch/conf/vars.xml.tmpl
index b31edeb..f7b2e5e 100644
--- a/mod/freeswitch/conf/vars.xml.tmpl
+++ b/mod/freeswitch/conf/vars.xml.tmpl
@@ -256,7 +256,7 @@
        If unspecified, the bind_server_ip value is used.
        Used by: sofia.conf.xml dingaling.conf.xml
   -->
-  <X-PRE-PROCESS cmd="set" data="external_rtp_ip={{ .Env.EXTERNAL_IPv4 }}"/>
+  <X-PRE-PROCESS cmd="set" data="external_rtp_ip=autonat:{{ .Env.ANNOUNCED_IPv4 }}"/>

   <!-- external_sip_ip
       Used as the public IP address for SDP.
@@ -269,7 +269,7 @@
        If unspecified, the bind_server_ip value is used.
        Used by: sofia.conf.xml dingaling.conf.xml
   -->
-  <X-PRE-PROCESS cmd="set" data="external_sip_ip={{ .Env.EXTERNAL_IPv4 }}"/>
+  <X-PRE-PROCESS cmd="set" data="external_sip_ip=autonat:{{ .Env.ANNOUNCED_IPv4 }}"/>

   <!-- unroll-loops
        Used to turn on sip loopback unrolling.
@@ -395,4 +395,4 @@
   <X-PRE-PROCESS cmd="set" data="video_mute_png=$${images_dir}/default-mute.png"/>
   <X-PRE-PROCESS cmd="set" data="video_no_avatar_png=$${images_dir}/default-avatar.png"/>

-</include>
\ No newline at end of file
+</include>
diff --git a/sample.env b/sample.env
index f83534d..7dd7f33 100644
--- a/sample.env
+++ b/sample.env
@@ -59,6 +59,12 @@ DOMAIN=bbb.example.com
 EXTERNAL_IPv4=144.76.97.10
 EXTERNAL_IPv6=

+# If your BBB system is NAT'd behind a router, define the 
+# internal IP of your docker host on the NAT'd network. 
+# If this isn't done, you'll get 1004 or 1007 errors when trying
+# to connect from outside.
+INTERNAL_IPv4=
+
 # STUN SERVER
 # stun.freeswitch.org
 STUN_IP=216.93.246.18