IOTstack internal networks being configured with non-RFC1918 addresses.

[yknivag]

Internal networks iotstack_nw, iotstack_nw_internal & vpn_nw are being configured with publicly routeable IP addresses (192.16.238.0/24, 192.16.230.0/24 & 192.16.200.0/24) belonging to "Shared Services Canada" and "SURFwim, Utrecht" (AS1101) respectively.

I'm no networking expert but shouldn't these should be internal addresses? Maybe they should all begin 172?

This could surely cause a problem for users trying to access the public services offered by those who own these addresses in the public space.

The issue can be found in: IOTstack/.templates/env.yml @ line nos: 9, 19, 28 https://github.com/SensorsIot/IOTstack/blob/master/.templates/env.yml

[Slyke]

@yknivag you are correct, but it would only affect services attached to these networks. I will still change them regardless, just in case there's an off change it could adversely affect someone, and so that it's to standard. I have found that a lot of services use Class B address range, so I'll change it to Class A 10.64.32.0/24 address schema, unless there's any reason you think it should be specifically 172.

[yknivag]

Hi @Slyke

Thank you for your speedy response! I don't think it matters which ranges are used, so long as they are RFC1918 ones. Whatever address you pick there will be someone using it in their network somewhere...

Incidentally there is a similar problem in the Wireguard package.

IOTstack/.templates/wireguard/service.yml line 16 https://github.com/SensorsIot/IOTstack/blob/master/.templates/wireguard/service.yml

This is setting the peer IP range to 100.64.0.0/24 which is reserved for ISPs to use for carrier grade NAT. Again these should really be a /24 from the RNF1918 range, as in the example at https://hub.docker.com/r/linuxserver/wireguard

[Paraphraser]

I'll chime in here to make some points I'd like @slyke to consider while reflecting on the earlier discussion. Nothing I say should be heard as either "complaint" or "criticism" of the negative kind. It's all just one person's opinion. I'm happy for anyone to disagree or point out flaws in my reasoning.

1. I agree RFC1918 addresses should be employed. The issue is not so much what happens in the user's local network but the small (finite) risk of a collision if something like a NodeRed flow is trying to reach an external service which happens to sit on the same allocated range. Most IOTstackers would have a lot of trouble figuring out what was going on.

2. Rather than choose from any particular RFC1918 range (in the sense of hard-coding anything into docker-compose.yml), I think it would be better to leave it to Docker to allocate. I've just done a test. I removed all the subnet definitions from my (new-menu) docker-compose.yml. Docker happily brought up all the networks, allocating from 172.16/12.

3. In the absence of any documentation as to the purpose of all these extra networks (a comment which should read as a hint that more explanation would be useful), I can only guess as to design intention. When I see:

   iotstack_nw: # Exposed by your host.

   and experiment by pinging within the RPi but outside container space, I can see that the network is indeed exposed. But my next question is, "so what?". I can't imagine a situation where knowing the IP range of an exposed network solves a problem that couldn't be solved by the loopback address plus a well-known port, à la old-menu. I really need help understanding what problem is being solved by this "exposed network" structure because, right now, it seems a bit like a feature searching for a rationale.

4. As far as I can see, the "exposed network" is only exposed inside the RPi. Unless I'm doing something wrong, it seems to me that for exposure to be useful beyond the RPi (to, say, IoT devices) would need either a routing protocol to advertise reachability or static routes, likely implemented in the home router. That means an IoT device will forward to the home router, which will redirect to the RPi. I see that as an extra hop for a benefit I'm yet to discern (maybe I'm being obtuse). It may also cause asymmetric traffic if the RPi and external devices share the same broadcast domain. Asymmetry isn't a big deal in and of itself. I'm just thinking of how the average user will keep track of what's happening. Again, I come back to the IP address of the RPi (preferably discovered via the DNS) and a well-known port being sufficient (keep in mind that I see reverse proxies as serious overkill too).

5. On Discord I noticed this about zigbee2mqtt_assistant not working. I'm don't think that would have happened under old menu. Sure, it can be dismissed as an oversight or bug which is easily fixed but, for me, it's another reason to raise the larger question of why all these discrete networks are necessary? Don't get me wrong. I spent years working in data-comms doing complex network designs. My knowledge is well out-of-date but I'm still a bit of a comms geek (my home network has multiple VLANs, wired & wireless, DMZs, LAN-to-LAN and dial-up VPNs, etc). But "comms geek" isn't how I read the average IOTstack user. I could be wrong but the impression I get is that what the average user wants from IOTstack is a stable and reliable scaffolding that just works, where their "thinking" starts at NodeRed flows and HA linkages, how to get data into a database, and how to visualise stuff in Grafana. Comms nuts and bolts are things they don't want to have to think about too much, if at all. The further we move from that as a design goal, the more we say "only experts should enter" and I, for one, don't see upping the minimum technical knowledge required to run IOTstack as a Good Thing.

6. Acknowledging again that I don't understand the design intention, I tried asking the "why have multiple networks?" question from a performance perspective. In a "real" network (physical hardware) there are plenty of situations where segmenting into multiple broadcast domains is advisable but I don't see parallels in the virtual networking environment of Docker. It really doesn't matter whether a packet is forwarded over net A or net B, the Docker engine still has to do all the forwarding. These are bridged networks in the "layer two switching" sense so unicast traffic only involves the two endpoints and a single replicant. There's no multicast support so that only leaves broadcast packets. I just ran a test on my live (old-menu) Pi. Nine broadcast datagrams in ten minutes, all BOOTP/DHCP, coming from a MAC associated with a virtual interface, at the rate of one every 63 seconds. Restarting the stack only produced a handful of ARPs. Even if 20 containers were attached to the internal network, broadcast traffic would still be negligible in a traffic-load sense so I see no performance benefit from multiple broadcast domains. I'm stumped.

7. This message during up/down really troubles me:

   WARNING: Some networks were defined but are not used by any service: nextcloud_internal, iotstack_nw_internal

   Axiomatically, if a network isn't needed, it shouldn't be defined. I'm the kind of person who tracks down and resolves every single compiler warning in my code (the only good compile is a clean compile!). I look at something like the 70 tons of warning messages during SQLite compilation in Node-Red and I'm absolutely aghast. I don't think we should be setting the kind of precedent for IOTstack where users simply have to know which errors and warnings mean "there's a problem" and which can be ignored.

Here endeth my AUD2¢.

[Paraphraser]

Another networking problem on Discord.

[Slyke]

Hello,

I've updated the networks on my local and will push it shortly. Just testing some other fixes before I do.

All good points, the reason some of the networks need to be hardcoded is because the VPN needs to know how to forward traffic to your LAN (bridging on your RPi), so it must know the network and subnet.

There's 2 networks because it's possible you don't want to expose your database ports to your localhost, but instead just have it accessible to internal services running on IOTstack, that's what the iotstack_nw_internal is for. Anything running on iotstack_nw should be exposed to anything on your LAN, unless there's some iptables or another setting that's been changed.

In nextcloud's case, it has its own database that spins up when it does. It could be made to run on the standard IOTstack's network, but there's no need to expose it since it's only nextcloud that will be using it, and it'd need to use a different port.

The network warning is annoying, and it is fixable, but it'd require a bunch of logic to scan through and see what networks all the services are using, and then just get those definitions from the networks section in env.yml. I do plan to implement exactly this. At the moment though, having unused networks shouldn't cause a problem, apart from the warnings.

I could also just put everything on the default network inside docker and that'll remove that warning, but all services will be able to see each other. Nextcloud and it's MariaDB will need to change ports (since there's another MariaDB service that users could possibly use).