Wifi router backup

[emizzle]

Purchase wifi router and SIM dongle (or maybe one that's integrated) that supports around 100 simultaneously connected devices. This can be used as a backup if the workshop venue wifi starts to lag/fail with a lot of connections.

[veaceslavdoina]

I have first two devices and also worked slightly with the Mikrotik and UniFi

#	Device	Wi-Fi	Moble	Clients	Coverage	Price
1	NETGEAR Nighthawk M6 Pro (MR6450)	4, 5, 6, 6E	5G	32	?	~ 1000 €
2	ZTE MU5120	4, 5, 6	5G	32 + 32	?	~ 300 $
3	Mikrotik Chateau 5G ax	4, 5, 6	5G	?	?	~ 595 $
4	UniFi Dream Router	4, 5, 6	-	300	150 m² (?)	~ 220 €
5	UniFi Express*	4, 5, 6	-	50	140 m²	~ 140 €

• Wi-Fi - Versions and generations

• Multiple UniFi Express can be deployed in a mesh configuration which reduce Wi-Fi throughput but can help to increase coverage area.

[emizzle]

Slava to order a UniFi Express.

Connection limits per SIM may be an issue. Slava to send Netgear router to Ben. Ben will try running 100+ nodes in Belgium.

[veaceslavdoina]

UniFi Express arrived and the following setup was tested

• NETGEAR Nighthawk M6 Pro set in IP Passthrough mode and powered using internal battery
• UniFi Express is managed via Cloud with almost all defaults and powered via type-c port using a power bank
• NETGEAR Nighthawk M6 Pro and UniFi Express are connected via Ethernet cable
• UniFi Express get ISP Public IP on its WAN interface

images

  

Basic tests

• Speed test
• Traceroute
• Public IP check
• TCP Port forwarding
• IPFS Desktop
  screenshots

Speed test

Traceroute

Public IP check

TCP Port forwarding

IPFS Desktop

Quick conclusions

• Setup is very portable and lightweight and we can use power banks and place devices in required locations
• In case we need to extend Wi-Fi coverage, we can add more UniFi devices and interconnect them wirelessly
• We can pass Public IP to the Wi-Fi access point
• Moldtelecom most probably is using Carrier-grade NAT which leads to the asymmetric communication
• TCP Port forwarding on UniFi works as expected but it is not so usefully for our case with CGNAT
• We can make hundreds of connections via mobile network, but it should be tested additionally using a different approach

Next steps

• Check UDP Port forwarding
• Check multiple simultaneous outgoing connections
• Check multiple Wi-Fi clients

[veaceslavdoina]

Wi-Fi coverage test

Environment

• Test was performed on a public stadium
• Distance was 0, 10, 20, 25 and 30 m
• AP is in direct view without any obstacles
• AP Wi-Fi Transmit Power = High
• Client device is Google Pixel 8 Pro with WiFiman app

Note: Stride length is ~ 70 cm, but they were longer then ussual and ~ 1 m.

images

**Equipment in the backpack on the tree**  **Distance - 30 m** 

Results

#	Time	Distance	Wi-Fi	Signal	Latency	Download	Upload
1	10:25	0 m	Wi-Fi 6 / 48	-34 dBm	42 ms	48.4 Mbps	32.3 Mbps
2	10:26	10 m	Wi-Fi 6 / 48	-60 dBm	42 ms	60.7 Mbps	46.0 Mbps
3	10:27	20 m	Wi-Fi 6 / 48	-66 dBm	42 ms	69.7 Mbps	39.9 Mbps
4	10:28	25 m	Wi-Fi 6 / 48	-68 dBm	40 ms	26.9 Mbps	42.6 Mbps
5	10:29	30 m	Wi-Fi 6 / 48	-68 dBm	38 ms	42.8 Mbps	38.5 Mbps

screenshots

    

Conclusions

1. Even on a 30 m distance signal is strength and phone was connected using Wi-Fi 6
2. Internet latency and speed is more or less similar and does not depend on the distance
3. With that setup we should be able to cover 20m x 20m open space

[veaceslavdoina]

Internet 5G in Belgium

• List of mobile network operators of Europe - Belgium
• 5G NR frequency bands
• spectrum-tracker.com/Belgium
• NETGEAR Nighthawk M6 Pro (MR6450) (n1, n2, n3, n5, n7, n8, n20, n28, n38, n40, n41, n71, n77, n78)

Carriers

#	Carier	5G Bands (?)	5G Coverage	Plans	FAQ
1	Proximus	n1, n3, n8, n28	map	17-35 € /month	FAQ 5G
2	Orange	n1, n3, n8, n28	map	?	FAQ 5G
3	Base	n1, n3, n8, n28	map	12-32 € /month	All about 5G

How to get a mobile phone number and SIM card in Belgium / Prepaid vs mobile contracts

As with many countries, expats have the choice of prepaid SIM cards or mobile contracts in Belgium. However, be aware that you won’t be able to take out a mobile contract unless you have registered at your local town hall. For many expats planning a longer stay, registration will take place soon after you arrive. However, if you’re only staying for the short term you will be limited to prepaid SIMs.

Quick conclussions

• Our 5G modem is compatible with Belgium operators
• Proximus probably can be most convenient, but that should be checked
• We might have port forwarding issue with mobile connection
• It is not clear if we can get right mobile plan in Belgium - should it be a contract and what documents do we need for that?

[emizzle]

eSIM option: https://www.airalo.com/belgium-esim physical SIM option (tether with android?): https://simcorner.com/collections/belgium-sim-card/products/belgium-travel-sim-card-20gb

[veaceslavdoina]

Did a check with a generic (from ZTE MU5120) Type-C to Ethernet adapter and Pixel 8 Pro with eSIM card

Laptop --> UniFi AP <--> Type-C-to-Ethernet <--> Phone

Internet works fine but we have a double NAT, because UniFi AP NAT is behind Android NAT. And some discussions around that - How to assign Mobile Data's public IP to host connected on hotspot?

An option would be to use a VPN with port forwarding to UniFi and port forwarding on UniFi to the clients

        p.fw       p.fw
Clients <--> UniFi <--> Cloud VPN --> Cloud Public IP

NETGEAR Nighthawk M6 Pro does not support eSIM and we can use for that type of connection only mobile Android phone.

And a list of the eSIM providers/services in case we decide to follow that way

providers/services

1. https://esim.holafly.com 2. https://bytesim.com 3. https://mobimatter.com 4. https://travel.vodafone.com 5. https://travel.orange.com 6. https://www.simoptions.com/esim-europe 7. https://www.esim.net 8. https://rapidesim.com 9. https://etravelsim.com 10. https://microesim.com 11. https://www.airhubapp.com 12. https://esimdb.com

We should be aware that not all of them provide an option to share the internet and especially on the unlimited plans.

[veaceslavdoina]

Looks like with UniFi we can't currently forward ports to the WireGuard interface and only to the WAN. So, we can't use a workaround witр the Cloud VPN tunnel, at least with the UniFi devices.

Make port forwarding between my VPN client wireguard and specific divice.

[emizzle]

Maybe we should look at buying a router that supports this? Routers running ASUS Merlin can do this (OVPN instructions but likely similar for wireguard)

I’m personally running out of other ideas on how we can work around not having hole punching. It seems like even if we get hole punching, that it will only work in limited scenarios, hardly ideal for a workshop. So this workaround is important.

[gmega]

I think discussions on the client team are converging towards using VMs for the workshop again, @emizzle. I don't think we'll be able to figure out reliable NAT traversal in time (the pieces are simply not there) and the uncertainty with fiddly routers and heterogeneous mobile provider network constraints are too risky to deal with.

[veaceslavdoina]

Performed a test with a Cloud VPN based on Mikrotik CHR and Mikrotik hAP ac³ using SSTP VPN

• That solution works
• TCP/UDP port forwarding works as expected
• VPN connection add an additional latency

[veaceslavdoina]

We found a workaround for devices which does not support eSIM and that need to be tested

• eSIM.me
• 5ber eSIM

[veaceslavdoina]

Belgium Internet testing plan

We discussed an option to go to the Brussels and perform 5G Internet connection and equipment we would like to use for a workshop.

More details

#### Plan - [ ] Take with you ~ 10 minutes - [ ] Check Mobile Internet packages/options ~ 1 hour - [ ] Prepare hardware for 5G tests ~ 10 minutes - [ ] Check Mobile Internet speed ~ 10 minutes - [ ] Check Mobile Internet port forwarding ~ 10 minutes - [ ] Check Mobile Internet simultaneous connection ~ 10 minutes - [ ] Check Wi-Fi in a workshop location ~ 1 hour Duration ~ 2:50 hours #### Take with you - Laptop - fully charged - Android phone - fully charged - eSIM adapter with an eject tool - NetGear modem - fully changed battery - UniFi - with a power bank and type-c cable - Ethernet patch cable to connect NetGear and UniFi - Mikrotik Chateau 5G ax with power adapter #### Check Mobile Internet packages/options ~ 1 hour In Belgium there are [3 mobile carrier](https://github.com/codex-storage/nim-codex/issues/781#issuecomment-2143828383): - Go to Proximus/Orange/Base and ask how we can purchase a package for 5G and 50/100/150GB of Internet traffic - Ask if we get a public IP - do not expect for technical answer - Ask if port forwarding will work - do not expect for technical answer #### Prepare hardware for 5G tests ~ 10 minutes - Insert eSIM into the Android phone - Install eSIM into the eSIM adapter using [Android App](https://play.google.com/store/apps/details?id=esim.me) - Put eSIM adapter into NetGear - Power on NetGear ~ 2 minutes - Power on UniFi using power bank and type-c cable ~ 3 minutes - Connect UniFi to NetGear using ethernet cable #### Check Mobile Internet speed ~ 10 minutes 1. Connect to the UniFi Wi-Fi from a Android 2. Open [WiFiman](https://play.google.com/store/apps/details?id=com.ubnt.usurvey) and authenticate using _UI Account_ account 3. Run 3-5 tests in a row - results will be saved and available for further screenshots ---- 1. Connect to the UniFi Wi-Fi from a laptop 2. Open https://www.speedtest.net in a browser 3. Run 3-5 tests in a row and save results via screenshots or Share --> Web --> Copy/Paste #### Check Mobile Internet port forwarding ~ 10 minutes 1. Connect to the UniFi Wi-Fi from a laptop 2. Check your Public IP - [ip.codex.storage](https://ip.codex.storage) 3. Check your Private IP - `ipconfig` 4. Open TCP port on laptop using Docker, based on your IP latest octet - `x.x.x.20'100` ```shell docker run \ --rm \ -p 20100:80 \ nginx ``` Connect to Public IP from Cloud VM ```shell curl public-ip:20100 ``` 5. Open UDP port on laptop using Docker, based on your IP latest octet - `x.x.x.20'100` ```shell docker run --rm \ -p 0.0.0.0:20100:5005/udp \ --name udp-listener \ -e UDPPORT=5005 \ mendhak/udp-listener ``` Connect to Public IP from Cloud VM ```shell # Connect nc -u localhost 20100 # Send data Test --> Enter # Check result on the server side Test ``` #### Check Mobile Internet simultaneous connection ~ 10 1. Run test using [iperf3](https://iperf.fr/iperf-doc.php) with 30, 50, 100 and save results ```shell docker run \ --rm \ networkstatic/iperf3 \ --client 168.119.50.205 \ --port 5201 \ --parallel 30 ``` #### Check Wi-Fi in a workshop location ~ 1 hour 1. Find a workshop location 2. Place eSIM card into Mikrotik router 3. Place Mikrotik to the table and plug into power outlet 4. Check Wi-Fi signal at the longest distance in the room using [WiFi Analyzer (open-source)](https://play.google.com/store/apps/details?id=com.vrem.wifianalyzer) - note signal level 5. Check Internet speed at the longest distance in the room via https://speedtest.net - save the results 6. Check Wi-Fi signal and speed in other parts of the room - save the results

[veaceslavdoina]

We recently discussed about UPnP support on Cloud VPN side.

And a quick recap

• It is supported on CHR
• It works using Multicast and will work only at same L2
• Quick experiments with IGMP Proxy/PIM-SM and EoIP didn't work and need to be investigated
• Zerotier is an interesting option and should be tested

Discussions on Mikrotik forum

- [Setting TTL for Outgoing Traffic](https://forum.mikrotik.com/viewtopic.php?t=144140) [SOLVED] - [PIM/IGMP over OVPN](https://forum.mikrotik.com/viewtopic.php?t=52758) - [UPnP for VPN users](https://forum.mikrotik.com/viewtopic.php?t=103579&sid=f438b5003475e622a511ce1c63cf6b5d) - [Multicast over VPN for site-to-site](https://forum.mikrotik.com/viewtopic.php?t=104667) - [Working DLNA routing example (basic)](https://forum.mikrotik.com/viewtopic.php?t=135405) - [Network discovery over wireguard](https://forum.mikrotik.com/viewtopic.php?t=194634) - [HOW TO: mDNS and SSDP over Wireguard](https://forum.mikrotik.com/viewtopic.php?t=194842) - [The complete SSDP + mDNS solution for network segmentation](https://forum.mikrotik.com/viewtopic.php?t=197542) - [mDNS between VLANs with just bridge filters - Look Mum, no containers!](https://forum.mikrotik.com/viewtopic.php?t=204025)

[veaceslavdoina]

We have successfully tested with Ben

1. eSIM to SIM adapter with a Best Europe 30 GB package and enabled roaming

2. Netgear with eSIM adapter and UniFi

3. Mikrotik with eSIM adapter

[veaceslavdoina]

We ended up with the following solution

• Mikrotik Chateau 5G ax and local Wi-Fi network
• For Internet we use eSIM.me and Best Europe 30 GB package
• Codex Bootstrap node in local Wi-Fi network running on NUC
• Codex Storage nodes in local Wi-FI network running on NUC and laptops
• External Geth network to be able to use Faucets/Discordbot/Explorer