Open goodboy opened 11 months ago
Hey,
Thanks, there's a lot of good questions here.
So for the PtP Wireguard use-case you'd want to be able to run something like:
# On client end
$ vopono exec --provider ptp --protocol wireguard --custom ptpclient.conf firefox
# On server end
$ vopono exec --provider ptp --protocol wireguard --custom ptpclient.conf webserver
And have them able to communicate only inside the network namespace e.g. in this case the client could see the webserver, but only inside the netns.
So you just need point-to-point connection and don't need those outgoing connections to also be over a VPN, etc.?
I think this should be doable, one thing I want to add before the next release is an empty
/ None
provider and protocol to just create a blank namespace with basic connectivity for testing (with --keep-alive
). It'd also help with testing stuff like the Cloudflare Warp issues - #235
I'll try this out when I get some time (it'd be really helpful for me as my ISP disallows port forwarding) but I do want to fix the Cloudflare Warp issues first.
I can't for the life of me figure out why it's not when using vopono, even when using -o 80 to forward the http port over the tunnel.. I've caved to just finally believing that my firewall knowledge is just so terrible I must be missing something obvious..
What were you using as the vopono command? As any of the VPN configs will restrict traffic to the VPN tunnel which doesn't make sense for a LAN (i.e. you'll connect to Mullvad, and won't be able to connect back from there to your LAN (assuming no public access, etc.)). Note you can use -f
to let the host proxy to the port, but that shouldn't be necessary here anyway.
Also note that simulating a LAN with network namespaces is not the same, as one of the main parts here is that we're dealing with traffic sent over the chosen adapter (i.e. your ethernet adapter - to the LAN / Internet) - specifically with the masquerade rule. That said, it should be doable to get working with different firewall rules (see the veth diagram on the Red Hat article you linked for example), but it is a very different problem from the normal LAN since you aren't using the same network interface, etc.
I actually wanted to provide a library to do this - see issue #49 . Separating vopono into the part creating the network namespace and basic connectivity, and the VPN provider config part. But the main issue I hit was dealing with RTNETLINK for setting Nftables rules. Also OpenVPN provides no library until OpenVPN3 (which also breaks other parts of the config we currently use :joy_cat: - see issue #47 ) - but you can see the messy prototype code in libvopono - https://github.com/jamesmcm/libvopono/blob/master/src/lib.rs
Another issue was working out how the API should be, you can't really do anything async since you need to guarantee that the right threads are in the right namespaces, which is a bit of a pain (it's more like C at this point, and a lot of network stuff is async). FWIW in that code the idea is you spawn a network namespace, and can then run closures in it by forking the process.
The permissions are also a bit more complicated and it might really force switching to a daemon (running as root) and client (local user) model like other apps (e.g. Cloudflare Warp) - but this is a huge change and really affects how easy it is to just download and run for most users. There is a difficult trade-off in not worsening the user experience for most users i.e. just running a browser in Mullvad, etc. and also spending time on these features vs. things which more of a direct effect for more users like port forwarding in ProtonVPN #226 or even a GUI.
One thing to look at for the NfTables stuff is the Mullvad Rust code for their desktop client - https://github.com/mullvad/mullvadvpn-app/blob/97c9771bde9a01b327cc040eee60b35f169cc84e/talpid-routing/src/unix/linux.rs#L267-L283
But for an existing network namespace, you should just be able to do - https://github.com/jamesmcm/libvopono/blob/master/src/lib.rs#L95-L113 to move a thread / process to it.
why for the veth tunnel is the default route always via-ed to 10.200.1.1?
It's arbitrary (but needs to be a local address). Also for the host to netns addresses they need to be unique, so this was just an easy way to mostly ensure that (up to 255 addresses) and also easily pair and convert from one to the other.
why not use macvlans for the namespace briding instead?
I'd never heard of it, I don't think it existed when I first pieced together the bash scripts. But this is a really good idea to try out, it'd simplify some of the steps at least (at a cost of maybe dropping support for earlier Linux versions, unless it were an option like iptables vs. nftables). Here's an example I found btw - https://unix.stackexchange.com/questions/742684/how-to-use-network-namespaces-for-vpn-split-tunneling
There's a lot of things to fix like this, like IPv6 endpoint support too (it breaks the masquerade address translation :crying_cat_face: ) - #181 , and fixing the use of network interface altnames #230
this doesn't bode well for multi-peer setups (addr collisions) and I'm wondering would be it be sane to think about a way in which vopono could detect a (custom) multi-peer config
These IP addresses are for the network namespaces though. What do you mean by a multi-peer config in this case? Like a VLAN over Wireguard?
As mentioned above, doing this between local network namespaces will be difficult due to needing to allow the network namespaces to communicate to one another (instead of just out over the host's network interface). If you really want a local "network" of network namespaces, I think it'd be easier to manage with Docker Compose.
So I originally collected some bash scripts together and got it to work with that and some Red Hat articles. Unfortunately I can't find them now, but this was very close - https://bbs.archlinux.org/viewtopic.php?id=215085
But I agree the documentation situation isn't great in general. I'll see if I can at least add the commands and order of steps for a normal run with Wireguard and OpenVPN for example and put them in the CONTRIBUTING.md
@jamesmcm Yo!
No, thank you so much for the in depth response and clarification.
Sorry i've ghosted; I WILL be coming back to this as soon as I can, just been hit recently by boring life stuff the past couple weeks.
A couple projects i work on are extremely interested in getting the PtP stuff going and obviously I (/we) would like to use that motivation to help make vopono
better š
I'm hoping to have a proper per-section breakdown discussion-response going within the next couple weeks šš¼
Absolutely amazing project btw; finally something to help with automating the magic of the linux networking stack with
wireguard
:boom:I'm starting this issue as both source of discussion around learning more about how
vopono
has been designed as well hopefully for content that might drive more advanced usage and features. I will be incrementally updating it as I learn more, start trying to grok the code base, and hopefully get some (good) feedback :smile:.The "Ask"
I realize maybe this isn't (yet) a core use case, but for many purposes being able to share certain (sets) of unsecured protocols over a wg tunnel would be super handy in building and deploying many classes of highly distributed sw applications š
I've tried with little success to get a basic PTP setup that is normally quite easy to configure with std
wg
/wg-quick
as per this lovely post: https://www.procustodibus.com/blog/2020/11/wireguard-point-to-point-config/and the
wg-quick
guide: https://www.wireguard.com/quickstart/Premises
vopono
net ns + wireguard automation around processes that will communicate with each other overwg
tunnels using a (potential ecosystem of) protocol(s).wireguard
configs, (presumably) different then the default firewall rules, possibly additional config options tovopono
itself?Technical questions
To start this is going to be a few questions that hopefully can be explained as just a lack of my own knowledge; i'm pretty rusty with the advanced networking features in linux, though I used to know some of this stuff..
why for the
veth
tunnel is the default route always via-ed to10.200.1.1
?veth
) .veth
tunnels:macvlan
s for the namespace briding instead?inside the net ns the same private addr subnet space is always used, and the same incremental addresses allocated?
vopono
could detect a (custom) multi-peer config and then automatically be allowed (or configured) to partition the subnet space in some way?Basic deployment how to?
i've tried within a LAN deploying a PTP style setup with 2 (metal) hosts and have had zero luck being able to
http
GET a little webserver running in one of the namespaces.python -m http.server
to verify the PTP endpoints are working.vopono
, even when using-o 80
to forward the http port over the tunnel.. I've caved to just finally believing that my firewall knowledge is just so terrible I must be missing something obvious..sure would seem handy to have a guide on how to setup pinging from one namespace to another much like the
wg
quickstart: https://www.wireguard.com/quickstart/More docs and details on custom-config and advanced usage
The
vopono -v exec
debug logging is amazing for figuring out what automation is actually going on behind the scenes :thumbs_up: but it seems there's very little docs on custom config setups in general and definitely very little explanation for the default firewall settings,veth
bridging around the net ns, configuration of the wireguard interfaces.I'd for sure be willing to at least do some in depth writeups on what all these commands mean, are doing, and possibly if more features get added for "advanced custom config" setups, extend the user guide (with more sections) to make grokin advanced usage and config easier.
I think this is really important for both advanced users as well any potential hacker/code-contribs.