DtxdF
commented
5 months ago Hi @OrvilleQ
I am currently not at home which has 80 MB/s (rx) and 30 MB/s (tx), a big difference if we compare the ADSL link where I am.
Excerpt from net/speedtest-go:
# speedtest-go
...
✓ Latency: 42.665045ms Jitter: 117.528µs Min: 42.406088ms Max: 42.883802ms
✓ Download: 4.51Mbps (used: 5.38MB) (latency: 61ms jitter: 37ms min: 48ms max: 205ms)
✓ Upload: 0.76Mbps (used: 0.91MB) (latency: 73ms jitter: 21ms min: 47ms max: 139ms)But if we compare the result of speedtest-go executed in a jail:
# appjail jail list -j speedtest-go
STATUS NAME TYPE VERSION PORTS NETWORK_IP4
UP speedtest-go thin 13.2-RELEASE - 10.0.0.5
# appjail cmd jexec speedtest-go speedtest-go
...
✓ Latency: 31.867667ms Jitter: 845.172µs Min: 31.07015ms Max: 33.496017ms
✓ Download: 4.87Mbps (used: 5.80MB) (latency: 87ms jitter: 111ms min: 35ms max: 451ms)
✓ Upload: 0.76Mbps (used: 0.91MB) (latency: 91ms jitter: 63ms min: 51ms max: 250ms)It's even better, but this doesn't mean that the performance in jails is better, it's probably just better because speedtest-go used two different servers. Although there is an overhead when using virtual networks, it is actually so small that you should not notice any difference. Of course, in your case there is a problem since you notice a performance loss. Check the bridge's MTU to confirm if it matches the host's MTU currently used:
# appjail network list
NAME NETWORK CIDR BROADCAST GATEWAY MINADDR MAXADDR ADDRESSES DESCRIPTION MTU
ajnet 10.0.0.0 10 10.63.255.255 10.0.0.1 10.0.0.1 10.63.255.254 4194302 AppJail network 576
# ifconfig jext | grep mtu
jext: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 576
# grep MTU /usr/local/etc/appjail/appjail.conf
DEFAULT_VIRTUALNET_MTU=576
# appjail version
3.2.0+15f7db9c07e005d852119b2346b93b9900d4e023Another recommendation is to try to minimize configuration overhead:
- Use
GENERICkernel instead of a custom one as it may or may not affect network performance in jails, but if this happens it is probably a bug. Note: I'm just trying to deduce the cause, not conclude that a custom kernel can affect its performance. - If you have configured your firewall, try to minimize the configuration to a minimum:
# cat /etc/pf.conf
nat-anchor 'appjail-nat/jail/*'
nat-anchor "appjail-nat/network/*"
rdr-anchor "appjail-rdr/*"This is what AppJail requires to work when using Virtual Networks, no more, no less.
- Use another networking feature (for discard only) like VNET, Bridges or even Netgraph. If you have more than one interface or you don't have a problem with a jail stealing your interface, I recommend using VNET. VNET will steal the target interface from the host and you can make any allowed changes to that interface, so you have more control and no overhead. Bridges and Netgraph have a bit of overhead and under normal circumstances you shouldn't see any real difference as mentioned above.
Bridges and Virtual Networks use if_bridge(4) and if_epair(4) drivers, and most jail frameworks and users use the same drivers for jail networking, so if you notice a performance overhead, this will affect to many people unless it is just a problem in your system.
- AppJail is used more on AMD64 platforms, but I know some users who have deployed AppJail on ARM platforms and there are no problems. Test on AMD64 platforms (for discard only).
I hope the above can help you, tell me any results you find.
OrvilleQ
Just created a server on my Hetzner server located at Germany. I self built the kernel to enable bbr and it's now on 14.0-RELEASE-p6.
I followed your document and set pf, virtual networks and dns up, then I create a jail with
appjail quick packager virtualnet=":packager"(nat is enabled on the virtualnet), but when I download ports using gitup I notice extremely slow network speed.Here's the result if I fetch a 10gb test file outside of jail:
And here's the result inside the jail:
It's 1500x times slower.
I'm new to FreeBSD, so I'm not sure where to start with debugging and digging useful information. If there are some specific details required, please inform me, and I will provide them as promptly as possible.