Unable to use IP abstraction in front of Bono

[dmoneil2]

Problem: Unable to use IP abstraction in front of Bono

Many cloud deployment tools such as kubernetes, openstack mesos, etc employ the concept of floating IPs or "IP Abstraction", possibly with NAT and then forward the incoming packets to the required destination.

This does not work with BONO and the user is forced to use a external IP of a physical machine (PUBLIC_IP) for the deployment. This creates a hard coupling for USER -> Bono IP endpoints.

The client reports a non-routable addresses in SIP Messages received by the server.

As the endpoint behind the Gate/floating/etc is a non routable address

[mirw]

@dmoneil2, thanks for raising this. Are you able to share a SIP flow (and where it goes wrong), so that we can see which SIP header it is that is at fault?

[dmoneil2]

Somewhat crude drawing of the network @mirw. Hope this helps.

note, when changing the public IP to the private address of the k8s service (iptables gateway) also break ellis.

Not sure what else breaks when changing the PUBLIC_IP to private gateway

BONO LOG: capture4.txt

[dmoneil2]

[mirw]

Thanks for sharing this. It looks like Bono is record-routing itself with IP address 10.254.207.54. It presumably should be record-routing itself with 192.168.96.6. For the topology you're proposing, this should be set via the public_hostname option in /etc/clearwater/shared_config. Please can you check what this is set to?

I suspect by default it will be set to the same as local_ip, which is probably what is causing the problem. You can override this by setting the PUBLIC_IP environment variable when spinning up the Bono container.

(Note that for clearwater-docker testing, we often just configure the SIP phone to have a proxy that points at the public IP address, which is why this default often works.)

Note that, although K8s supports services that are fronted by a load-balancer, I'd be a little nervous about doing that here (although it might work). Specifically, the SIP/TCP connection from the SIP phone to Bono can only be established inbound - Bono can't connect back out because almost all SIP phones are behind some kind of firewall or NAT. As a result, Bono anchors these connections and keeps them alive for long periods of idle time. I don't know what the load balancer's behavior would be for long-lived TCP connections - it's possible that it would clean them up from time to time.

[mirw]

@dmoneil2, please can you check the public_hostname option/PUBLIC_IP environment variable and let me know how you get on?

[eleanor-merry]

Ported to https://github.com/Metaswitch/project-clearwater-issues/issues/18

[TaiSHiNet]

Did you manage to get it working? While trying to test this within a local k8s cluster, NodePort can't be assigned to ports under 30000