Qubes-Community / Contents

Community documentation, code, links to third-party resources, ... See the issues and pull requests for pending content. Contributions are welcome !
258 stars 98 forks source link

VPN page requires rework #103

Closed 3hhh closed 11 months ago

3hhh commented 3 years ago

From my point of view the VPN page is a 2.x or 3.x relic and should be re-written. The large number of questions related to VPN setups on qubes-users or discourse indicate the same.

Reasons:

Instead I'd suggest using the Qubes OS firewall and proper VM segregation as default recommendation: Essentially just describe the required infrastructure, i.e. sys-net -- sys-fw -- sys-vpn -- sys-fw-vpn, the need to limit sys-vpn connections to the VPN provider servers only via qvm-firewall to remain leak-proof and let the user install whatever VPN application inside sys-vpn (with openvpn just being one option).

awokd commented 3 years ago

Suggest edits on the pull request, or I'll commit as is after a while. @tasket, any input is welcomed too.

tasket commented 3 years ago

The original issue for the vpn doc was focused on providing something that could be considered an educational, general guideline that also works. I think this explains its overall structure and content. As it stands, for most cases the user only has to create the VM, cut/paste 3 small scripts, copy their config files under /rw/config/vpn, and maybe do editing for the password file. The suggested edits appear to dispense with that approach (and removes almost everything, actually :) ).

I'd also like to point out that pulling this functionality together can seem deceptively simple, esp. when the target audience is varied and includes users who want "the most secure". Its not simple and that's why VPN providers issue their own special apps (Qubes arch. makes it even less simple).

And noting the doc isn't even long (looking subjectively at the length), what it does it show VPN setup and enhanced security for both GUI and CLI environments.

If we lop that off for brevity and discourage less ideal security, then the AppVM section becomes a bit precarious, too. But it would probably be a mistake not to mention that type of configuration.

I think its OK to introduce a domain-limited term or two into a Howto for the sake of clarity, esp. when its highlighting a distinction that exists in layers beneath the admin UI.

However... the ProxyVM term in the Network Manager segment was vaguely associated with "provides network" in the first step, whereas the CLI-based section is less vague.

My suggestion here is to keep the ProxyVM term and add a sentence at the top of the Network Manager and the CLI-based sections explicitly stating what it is. The first step in the Qubes-vpn-support Readme says "In Qubes 4.0 a proxyVM is called an 'AppVM' with the provides network option enabled; this document will use the more descriptive 'ProxyVM' term...". I think everyone using it seems fine with that.

If you are using anything other than OpenVPN, change the VPN_CLIENT and VPN_OPTIONS variables to match your VPN software.

However, the Network Manager segment has no mention of Openvpn that I can see.

BTW, I'm not against extending examples to include wireguard (I have just recently switched to a VPN provider that offers wg as a first-class option), but I'll also say there is little you can do beyond those two variables to further generalize the solution.

More comments.....

tasket commented 3 years ago

Continued...

Additionally, expanding the Network Manager segment a bit to mention the NM plugins that need to be present for popular protocols like Openvpn and Wireguard, I think would be helpful.


PS - I noticed under CLI segment, Step 4 it does ask the user to edit an Openvpn file with specific commands. This seems to be the one significant failing it has in being openvpn-specific; The qubes-tunnel scripts automate this process.

awokd commented 3 years ago

Thank you, @tasket. I know when I look at the original doc here, it's a bit overwhelming with all the scripts. Wanted to simplify, but probably went too far in that direction. Were you saying it might be better to not expect this to be an educational doc, or Qubes-vpn-support? Also, good call on qubes-tunnel, I think including a link to that towards the top ought to help point people in the right direction. I'll open another pull on this.

tasket commented 3 years ago

@awokd The PR seems to have an editing mistake, since most of the doc past Step 1 disappears. That's what I was referring to in my first paragraph.

I'm leaning toward making the CLI section far shorter and based on qubes-tunnel since that was one of the reasons for creating that project (i.e. cloning and rebranding Qubes-vpn-support with a better name). I'll do a PR over the holidays since I'll have some time.

3hhh commented 3 years ago

@tasket: While I respect you and your work, I don't necessarily agree with you on all points. Btw I think we had this discussion already 3 years back or so.

Let me clarify:

awokd commented 3 years ago

I think we should be able to incorporate both viewpoints into the same document without making it too busy. @3hhh, mind letting tasket take a go at it first so we have something not 2.x/3.x at least to base from, then making sure your points get addressed too?

3hhh commented 3 years ago

mind letting tasket take a go at it first so we have something not 2.x/3.x at least to base from, then making sure your points get addressed too?

Totally fine for me.

awokd commented 3 years ago

@tasket, did you get a chance to come up with a draft doc/edit?

awokd commented 3 years ago

Didn't realize it had been two months already. @tasket you still out there? If not, @3hhh want to give it a go? I'm willing to help and spend the time to consolidate documents, but it's difficult for me to start from scratch as I only rarely use VPN functionality.

tasket commented 3 years ago

@awokd @3hhh I would be comfortable starting a re-write if there were more agreement on the technical points. Relying on sys-firewall is highly problematic as that VM won't have the information needed to block effectively. sys-firewall is not a bastion of Qubes networking correctness and it introduces problems for users who think it gives them the ability to control traffic with an appropriate level of specificity – it usually doesn't.

TBH, sys-firewall and the dom0-administered rule system was a nice idea to begin with, but over time it hasn't grown to effectively handle real situations as it can't cover domains properly (just for starters).

If you dig through the issues and PR comments on doc/vpn, you'll see the failsafe method has been criticized over and over (and over) again and the result has always been that Marek comes into it and disagrees with the requester as the desired firewall change is always worse than what is there now. The method currently used is not conventional because Qubes gives us the opportunity to do something better; to some people that will look odd or wrong.

As for users messing with proxyVM firewalls, there is no upside to that. They can mess with sys-firewall, too, and there are far more examples in the forums of that not working out as the user expected.

Checking for vpn up/down status is covered by the qubes-tunnel configuration; the service is restarted when the link goes down. This relies on the vpn protocol handler to decide when that happens, but the protocol handler is also what decides when/how to add or remove the route, so...

I don't have a problem with deprecating the suggestion that the vpn can be setup in sys-net.


With that said, I agree that changing the doc's overall style to a simpler, non-cut-and-paste method would be welcomed by users; the vehicle for that simplicity would be changing the CLI portion to use qubes-tunnel. Also, adding some guidance for wireguard would be useful, as the interest in that protocol has greatly increased. I think those should be the two goals for a vpn doc re-write.

3hhh commented 3 years ago

@awokd @3hhh I would be comfortable starting a re-write if there were more agreement on the technical points. Relying on sys-firewall is highly problematic as that VM won't have the information needed to block effectively. sys-firewall is not a bastion of Qubes networking correctness and it introduces problems for users who think it gives them the ability to control traffic with an appropriate level of specificity – it usually doesn't.

Do you have examples of real issues? I simply don't agree with the "we trust sys-firewall for everything except for VPN" argument. If it's broken, it should be a bug in Qubes OS and requires fixing.

I'm currently aware of the following quirks:

With that said, I agree that changing the doc's overall style to a simpler, non-cut-and-paste method would be welcomed by users; the vehicle for that simplicity would be changing the CLI portion to use qubes-tunnel. Also, adding some guidance for wireguard would be useful, as the interest in that protocol has greatly increased. I think those should be the two goals for a vpn doc re-write.

Agreed. Not necessarily the only two, but very relevant parts.

tasket commented 3 years ago

The problem with integrating sys-firewall with VPN anti-leak protection stems from the fact that the VPN client itself is providing a rather strong guardian function, wherein it cryptographically verifies the VPN server's authenticity (VPN setups without this cert checking at connection time are pretty rare now). So not only is the firewall in sys-firewall lacking info and completely walled-off from the VPN client, the class of protection provided by the client already goes far beyond what an isolated firewall can do.

Additionally, the risk profile of a VPN client is rather low, IMO. So it makes sense for us to treat the proxyVM like a router device and have the firewall in the same VM configured by the VPN client.

Having a qr-exec based process send configuration info to sys-firewall is probably the most meaningful add-on that could be made in the foreseeable future. However, when thinking about the threat models and risks, its really unclear to me what we would be gaining in security. Any successful attack on the VPN client that could alter internal firewall rules could probably change, for instance, the VPN connection configuration.

Therefore, a true elevation of network security, where secure tunnels are a factor, would mean elevating the VPN configuration process itself to a dom0-administered feature where a VM is not only flagged as "provides network" but also "provides tunnel" and the VPN configuration is imposed on the proxyVM at startup (similar to Qubes firewall).

Also, the VPN VM could only securely supply one set of 'allow' IP addresses at startup – no follow-up IP lists could be accepted upon reconnection; this means reconnection would have to be performed by restarting the VPN VM itself. But this assumes that the VPN client isn't immediately vulnerable during the connection process.

3hhh commented 3 years ago

The problem with integrating sys-firewall with VPN anti-leak protection stems from the fact that the VPN client itself is providing a rather strong guardian function, wherein it cryptographically verifies the VPN server's authenticity (VPN setups without this cert checking at connection time are pretty rare now).

I guess you are referring to certificate revocation checks etc.? Yes, one would have to either live without them or have to allow the check servers as destination as well.

Additionally, the risk profile of a VPN client is rather low, IMO. So it makes sense for us to treat the proxyVM like a router device and have the firewall in the same VM configured by the VPN client.

I'm not so sure about that. OK, openvpn only received 16 CVEs in 2020, but I'm less confident about all these proprietary implementations.

Admittedly the attack vector is less common than e.g. browser-based attacks, but also more promising against users of non-disposable sys-vpn VMs.

Having a qr-exec based process send configuration info to sys-firewall is probably the most meaningful add-on that could be made in the foreseeable future. However, when thinking about the threat models and risks, its really unclear to me what we would be gaining in security. Any successful attack on the VPN client that could alter internal firewall rules could probably change, for instance, the VPN connection configuration.

As you also pointed out below one could implement a qrexec service that only works exactly once at boot time of sys-firewall and not again afterwards. Assuming that the VPN VM is not compromised at startup would be reasonable as long as it's disposable. For persistent VMs however there is no security gain, yes.

Alternatively one could employ another vpn-control VM that does all the necessary configuration in sys-firewall and sys-vpn. That's pretty similar to your below comment, just with vpn-control instead of dom0. Anyway it would require some custom VPN client that is aware of the other VMs (running inside vpn-control) or some proxy kernel driver that instead of executing the kernel-related network requests (create interface X etc.) inside vpn-control would have to execute them inside sys-vpn via qrexec. The latter sounds pretty hard to me, but might be useful for other setups as well.

Therefore, a true elevation of network security, where secure tunnels are a factor, would mean elevating the VPN configuration process itself to a dom0-administered feature where a VM is not only flagged as "provides network" but also "provides tunnel" and the VPN configuration is imposed on the proxyVM at startup (similar to Qubes firewall).

Also, the VPN VM could only securely supply one set of 'allow' IP addresses at startup – no follow-up IP lists could be accepted upon reconnection; this means reconnection would have to be performed by restarting the VPN VM itself. But this assumes that the VPN client isn't immediately vulnerable during the connection process.

I think we're pretty much on the same page there.

3hhh commented 3 years ago

some proxy kernel driver that instead of executing the kernel-related network requests (create interface X etc.) inside vpn-control would have to execute them inside sys-vpn via qrexec. The latter sounds pretty hard to me, but might be useful for other setups as well.

Generalizing it a bit more: Having a qrexec service that could pass a pre-defined set of kernel syscalls to another VM to let them execute there would be really cool to segregate duty to different VMs for a wide range of user applications without having to make these applications aware of the Qubes VM infrastructure. Implementing & configuring such a beast would be pretty hard though I guess (similar to SELinux or so).

deeplow commented 3 years ago

Someone on the forum created a guide for configuring a VPN under Qubes through GUI-only: https://qubes-os.discourse.group/t/how-to-setup-openvpn-fedora-appvm-for-ovpn/3354. Could be interesting this approach in the docs as well.

awokd commented 3 years ago

Maybe I could use that as a starting point, thanks @deeplow .

tasket commented 3 years ago

The problem with integrating sys-firewall with VPN anti-leak protection stems from the fact that the VPN client itself is providing a rather strong guardian function, wherein it cryptographically verifies the VPN server's authenticity (VPN setups without this cert checking at connection time are pretty rare now).

I guess you are referring to certificate revocation checks etc.? Yes, one would have to either live without them or have to allow the check servers as destination as well.

I'm referring to the certs typically supplied with Openvpn configs. They let the client check the authenticity of the server.

Additionally, the risk profile of a VPN client is rather low, IMO. So it makes sense for us to treat the proxyVM like a router device and have the firewall in the same VM configured by the VPN client.

I'm not so sure about that. OK, openvpn only received 16 CVEs in 2020, but I'm less confident about all these proprietary implementations.

The CVE's I've seen often mention Openvpn, but are actually for bugs in proprietary connection managers where they misconfigure openvpn, firewall, etc. This is like looking for "Xen" CVEs and thinking that Qubes is insecure.

Admittedly the attack vector is less common than e.g. browser-based attacks, but also more promising against users of non-disposable sys-vpn VMs.

Having a qr-exec based process send configuration info to sys-firewall is probably the most meaningful add-on that could be made in the foreseeable future. However, when thinking about the threat models and risks, its really unclear to me what we would be gaining in security. Any successful attack on the VPN client that could alter internal firewall rules could probably change, for instance, the VPN connection configuration.

As you also pointed out below one could implement a qrexec service that only works exactly once at boot time of sys-firewall and not again afterwards. Assuming that the VPN VM is not compromised at startup would be reasonable as long as it's disposable. For persistent VMs however there is no security gain, yes.

Alternatively one could employ another vpn-control VM that does all the necessary configuration in sys-firewall and sys-vpn. That's pretty similar to your below comment, just with vpn-control instead of dom0. Anyway it would require some custom VPN client that is aware of the other VMs (running inside vpn-control) or some proxy kernel driver that instead of executing the kernel-related network requests (create interface X etc.) inside vpn-control would have to execute them inside sys-vpn via qrexec. The latter sounds pretty hard to me, but might be useful for other setups as well.

I don't see inter-VM configuration happening, at least not in the Qubes firewall sense. Qubes firewall did not encompass enough functionality to be very useful; it only had one configuration target (iptables). Doing this for generic VPN/tunnel is much harder.

Therefore, a true elevation of network security, where secure tunnels are a factor, would mean elevating the VPN configuration process itself to a dom0-administered feature where a VM is not only flagged as "provides network" but also "provides tunnel" and the VPN configuration is imposed on the proxyVM at startup (similar to Qubes firewall). Also, the VPN VM could only securely supply one set of 'allow' IP addresses at startup – no follow-up IP lists could be accepted upon reconnection; this means reconnection would have to be performed by restarting the VPN VM itself. But this assumes that the VPN client isn't immediately vulnerable during the connection process.

I think we're pretty much on the same page there.

The idea that a remote attack is a big problem here... I have to disagree. The VPN client is performing an essential guard function... its up to the client to take care of link security, not us. A MITM would be spoofing the provider's IP address, which is easy to do – I'd much, much rather rely on the VPN certificate. OTOH, if MITM succeeds, the attacker has access to your cleartext traffic and can direct it out through the known IP address (game over). I think this is the correct threat model for a remote attack against a VPN client.

For the rest (preventing leaks around the link), this is more of an 'accidental spillage' threat model. If the VPN suddenly exits/restarts for example, you don't want cleartext packets getting routed to the outside. The existing iptables rules are extremely effective at this, and they will keep working in most situations; the VPN server's IP address can change 10X a day and it won't be phased or require reconfiguration.

tasket commented 3 years ago

@awokd After posting to the current VPN thread, I just want to reiterate here how the original inclination (which I think was voiced by @marmarek in various issues/PR comments) was to handle Network Manager at the end of a very long stick.... Tell them to refer to VPN provider docs and distro docs. Now that I think about it again, the more I think that including NM at all was a mistake. I do not think a detailed NM walkthrough is a good starting point for a revised VPN doc.

The NM concept of internally capturing necessary VPN config details after translating them via a special plugin failed for many years; it worked in only a minority of cases and that's why most providers still avoid mentioning Network Manager. The potential for half-failure (insecure connections) is probably high. Providers would rather build their own GUI than deal with the OS GUI.

I think the only correct way, GUI or not, is to use (not import) the original config file and touch none of the link-security parameters, running the VPN client (openvpn, etc) directly with that config. So I think we should use qubes-tunnel as the basis for a revised doc.

3hhh commented 3 years ago

The problem with integrating sys-firewall with VPN anti-leak protection stems from the fact that the VPN client itself is providing a rather strong guardian function, wherein it cryptographically verifies the VPN server's authenticity (VPN setups without this cert checking at connection time are pretty rare now).

I guess you are referring to certificate revocation checks etc.? Yes, one would have to either live without them or have to allow the check servers as destination as well.

I'm referring to the certs typically supplied with Openvpn configs. They let the client check the authenticity of the server.

Alright. However there is no blocking related to these certs apart from CRL or OCSP checks even if sys-firewall rules are used. All this happens as part of the regular protocol with the allowed destination server.

Actually I checked my provider cert wrt CRL or OCSP and even noticed that it doesn't have a CRL or OCSP field. Since all my VPN server certificates are signed by a single certificate, I guess the provider would inform me via e-mail in case it gets compromised.

The idea that a remote attack is a big problem here... I have to disagree. The VPN client is performing an essential guard function... its up to the client to take care of link security, not us. A MITM would be spoofing the provider's IP address, which is easy to do – I'd much, much rather rely on the VPN certificate. OTOH, if MITM succeeds, the attacker has access to your cleartext traffic and can direct it out through the known IP address (game over). I think this is the correct threat model for a remote attack against a VPN client.

I consider MITM a remote attack; maybe that's the misunderstanding there.

Anyway your threat nodel is mostly accurate except for the fact that working MITM can be used to obtain persistence inside sys-vpn via some bug in the VPN client. Persistence is quite interesting for an attacker as MITM usually only works well in some, but not all situations (e.g. when the user unexpectedly switches to some other DSL network that you hadn't compromised/sent legal warrants before) and you might miss the traffic you're interested in. Also, interesting traffic may have further encryption inside the VPN tunnel, i.e. gaining persistence on the target host is key for further attacks.

For the rest (preventing leaks around the link), this is more of an 'accidental spillage' threat model. If the VPN suddenly exits/restarts for example, you don't want cleartext packets getting routed to the outside. The existing iptables rules are extremely effective at this, and they will keep working in most situations; the VPN server's IP address can change 10X a day and it won't be phased or require reconfiguration.

Last time I reconfigured my settings was 3+ months ago... As long as the list of VPN servers remains stable, there's no need to reconfigure.

Regardless I don't insist on my proposed implementation method. As long as the initially mentioned points are fixed, we're all good.

tasket commented 3 years ago

The problem with integrating sys-firewall with VPN anti-leak protection stems from the fact that the VPN client itself is providing a rather strong guardian function, wherein it cryptographically verifies the VPN server's authenticity (VPN setups without this cert checking at connection time are pretty rare now).

I guess you are referring to certificate revocation checks etc.? Yes, one would have to either live without them or have to allow the check servers as destination as well.

I'm referring to the certs typically supplied with Openvpn configs. They let the client check the authenticity of the server.

Alright. However there is no blocking related to these certs apart from CRL or OCSP checks even if sys-firewall rules are used. All this happens as part of the regular protocol with the allowed destination server.

Actually I checked my provider cert wrt CRL or OCSP and even noticed that it doesn't have a CRL or OCSP field. Since all my VPN server certificates are signed by a single certificate, I guess the provider would inform me via e-mail in case it gets compromised.

I would put it as, the cert is a public key the provider is giving the user. There was no non-PKI way of updating certs, and TBH I think a lot of Qubes and privacy-seeking users would not want to have public CAs in the process (and VPN providers know this). But this is another reason why custom apps are commonplace, they can update certs on the providers' terms. Apart from having that custom app handling it, I think an email saying 'update your configs' is preferable.

With that said, the VPN client actually is blocking according to whether the cert matches. If verification fails, the link won't be established.

The idea that a remote attack is a big problem here... I have to disagree. The VPN client is performing an essential guard function... its up to the client to take care of link security, not us. A MITM would be spoofing the provider's IP address, which is easy to do – I'd much, much rather rely on the VPN certificate. OTOH, if MITM succeeds, the attacker has access to your cleartext traffic and can direct it out through the known IP address (game over). I think this is the correct threat model for a remote attack against a VPN client.

I consider MITM a remote attack; maybe that's the misunderstanding there.

Anyway your threat nodel is mostly accurate except for the fact that working MITM can be used to obtain persistence inside sys-vpn via some bug in the VPN client. Persistence is quite interesting for an attacker as MITM usually only works well in some, but not all situations (e.g. when the user unexpectedly switches to some other DSL network that you hadn't compromised/sent legal warrants before) and you might miss the traffic you're interested in. Also, interesting traffic may have further encryption inside the VPN tunnel, i.e. gaining persistence on the target host is key for further attacks.

Yes, persistence of a trojan/malware is the threat we're left with. I have myself configured Qubes-VM-hardening to wipe+repopulate the private volume of the VPN VM. But I didn't carry that over to my new system as I felt the benefit was more academic. In Qubes 4.1, I think this could also be addressed by dispVM templates.

FWIW, I don't think adding a Qubes firewall rule is a bad choice for ppl who don't mind reading configs for IP addresses and periodically re-populating. But the gain is too small to present it as more than an option or footnote.

Also note that INPUT is protected in the internal firewall rules for a qubes-tunnel setup. Part of it is the default proxyVM settings that Qubes uses. Only established traffic is going to come in.

For the rest (preventing leaks around the link), this is more of an 'accidental spillage' threat model. If the VPN suddenly exits/restarts for example, you don't want cleartext packets getting routed to the outside. The existing iptables rules are extremely effective at this, and they will keep working in most situations; the VPN server's IP address can change 10X a day and it won't be phased or require reconfiguration.

Last time I reconfigured my settings was 3+ months ago... As long as the list of VPN servers remains stable, there's no need to reconfigure.

I've seen large VPNs like PIA switch IP addresses, and use domain names (with random address selection) on and off for years. For many providers, we're looking at maintaining a list of allowed addresses that can grow quite large.

Regardless I don't insist on my proposed implementation method. As long as the initially mentioned points are fixed, we're all good.

tasket commented 3 years ago

@3hhh @awokd

I have a draft ready for a qubes-tunnel setup. Its not terribly different than the README instructions, but I've left out Network Manager – at least for now. At this point, I'd rather NM be dealt with as a footnote or a separate document if at all.

awokd commented 3 years ago

I have a draft ready for a qubes-tunnel setup.

@tasket Great, mind submitting it as a new document here? I'll merge it so we can work on it. It will help me a lot to have something to reference so I can go through this thread and make sure everything's addressed, even though I'm not an expert on VPNs. Once we're all good with it, I can retire the existing document and replace with this new one.

[Disregard, I either missed the link to it above or see you've added it since I wrote this.]

3hhh commented 3 years ago

A good start @tasket!

Some comments so far:

I didn't follow the instructions (yet?) to test them though.

Side note fyi: https://github.com/QubesOS/qubes-core-agent-linux/pull/303 should make the firewall DNS issue you mentioned before fixable.

3hhh commented 3 years ago

A good beginner's guide that works with most VPN providers (not only Mullvad): https://micahflee.com/2019/11/using-mullvad-in-qubes/

More cautious people could improve upon that blog post guide by a) avoiding DNS & ICMP leaks by using qvm-firewall instead of the GUI. b) creating a disposable sys-vpn.

awokd commented 2 years ago

@tasket , if you don't have time to review my suggested changes, mind if I copy/paste your document over here?

awokd commented 11 months ago

Content moved to https://forum.qubes-os.org/c/guides/14.