search

mirage / qubes-mirage-firewall

A Mirage firewall VM for QubesOS
BSD 2-Clause "Simplified" License
210 stars 28 forks source link

4.1 and Liteqube #174

Open kennethrrosen opened 1 year ago

kennethrrosen commented 1 year ago

I manually unpacked and installed the mirage fw and kernel into a liteqube setup on 4.1 (I realize liteqube also offers it as its own fw). The VM seems to function, but it cannot supply firewall rules. There is a red banner notice reading "Net qube does not support 'qubes-firewall'. Firewall restrictions will not be applied."

palainp commented 1 year ago

I see this too, but it will work as expected (to validate, you can try to restrict to one domain and you won't be able to get traffic outside that domain).

The message may come from a missing signaling part in the communication protocol between Qubes and the unikernel. We can probably leave this issue open until this message is fixed.

palainp commented 1 year ago

Update: I probably checked the issue on an old mirage-firewall installation, but with the new installation procedure, we should run qvm-features mirage-firewall qubes-firewall 1 which sets up the firewall property and removes this red message.

Can you tell me if this solves the problem for you?

kennethrrosen commented 1 year ago

I did this at install, as per the README file instructions. No joy. I've setup the same configuration on a vanilla 4.1 with firewall rules in the mirage VM, and it acts as a perfect vpn killswitch, allowing only VPN traffic. but here it allows both clearnet and VPN traffic, even with the rules.

palainp commented 1 year ago

Would you mind to give the result of qvm-firewall AppVM list?

kennethrrosen commented 1 year ago 
[user@dom0 ~]$ qvm-firewall fw-mirage list
NO  ACTION  HOST  PROTOCOL  PORT(S)  SPECIAL TARGET  ICMP TYPE  EXPIRE  COMMENT
0   accept  -     udp       1194     -               -          -       -
1   accept  -     tcp       5995     -               -          -       -
2   accept  -     tcp       8443     -               -          -       -
3   accept  -     udp       4569     -               -          -       -
4   accept  -     udp       5060     -               -          -       -
5   accept  -     -         -        dns             -          -       -
6   accept  -     icmp      -        -               -          -       -
7   drop    -     -         -        -               -          -       -
palainp commented 1 year ago

Thank you for that output.

These rules (set on mirage-fw in Qubes Manager but not dealt inside mirage-fw, it's up to it's netvm to filter the netflow) should be visible in the netvm with iptables -L (it may worth to check there if something is missing), and I agree that any flow not matching an accept line should be dropped in netvm.

It may also worth to try to set the rules on your vpn VM tab in Qubes Manager to check how mirage-fw deals with that.

kennethrrosen commented 1 year ago

They don't seem to populate in core-net, but of course the rules work when set through the Qubes Manager in sys-vpn, though I'm stumped as to why I can't set all necessary rules in fw-mirage.

palainp commented 1 year ago

Yes I think you should be able to restrict the vpn netflow by setting rules on the mirage-fw tab in Qubes Manager. However this leads into rules being checked by sys-net or core-net which is untrusted, and that sounds odd to me.

Given it seems to work as intended with a vanilla Qubes, is it possible to try with liteqube and a normal linux firewall: setting rules on that VM and see if it works as vpn killswitch? If it works we should fix something into mirage-fw, and if not it's probably something relevant in liteqube (and we may open an issue there with this one as background).

kennethrrosen commented 1 year ago

I created a new default sys-firewall qubesctl state.apply qvm.sys-firewall and attached the qubes: core-net <--> sys-firewall <---> sys-vpn <---> appqube and applied the firewall rules to sys-firewall. This worked as intended.

When I set the rules in sys-vpn, the appqube, and mirage-fw, those rules don't appear to be visible in core-net.

palainp commented 1 year ago

~The rules should only be visible in the uplink VM where you add some rules (so core-net when rules are set on mirage-fw or sys-firewall, mirage-fw or sys-firewall when set on sys-vpn, etc.).~ EDIT: In fact it's very hard to see the rules with iptables inside sys-net, the output of iptables refers to vif+ with few details :(

As it works fine with a linux firewall, I assume the issue comes from mirage-fw, but I don't see what it could be so far :(

palainp commented 1 year ago

To summarize the tests, am I right with the following?

So the situation is that we can't filter traffic outside vpn with mirage-fw directly after core-net, but we can't reproduce it with standard Qubes distribution.

kennethrrosen commented 1 year ago

Correct. Happy to test any other configurations. Is the red text appearing only since the latest Qubes release?

palainp commented 1 year ago

Hi, I found an easy way to check firewall rules in core-net, would you mind to type:

You should get something referring chain qbs-10-137-0-XX which stands for the IP the qubes and see the rules there.

palainp commented 1 year ago

Sorry I forgot the command to type in core-net : sudo nft list table qubes-firewall.

kennethrrosen commented 1 year ago

Hmmmm, that's bizarre: core-net and debian-core are both mssing nftables and thus I can't run nft.

palainp commented 1 year ago

Oh that's probably because liteqube uses the -minimal templates. That may (or may not) be a root cause for this issue.

I'm also thinking about the vpn script (https://github.com/a-barinov/liteqube/blob/main/4.VPN/default/debian-core/etc/protect/template.core-vpn/usrlocal/bin/liteqube-vpn), please, if that's what you use as vpn, can you check that the iptables rules are correctly up ? (this should be visible in the core-vpn VM with sudo iptables -L -v).

kennethrrosen commented 1 year ago

I don't use the liteqube VPN, but rather a variant of this: https://forum.qubes-os.org/t/how-to-setup-openvpn-fedora-appvm-for-ovpn/3354

palainp commented 1 year ago

Ok thanks ! Anyway this is probably not the root cause.

It seems that iptables in debian stands for nft, so you still should be able to see the rules for mirage-fw inside core-net (but the syntax could be different and I havn't liteqube for testing :( ), would you mind to share that?

kennethrrosen commented 1 year ago

Sorry I forgot the command to type in core-net : sudo nft list table qubes-firewall.

root@localhost:~# sudo iptables -L

target               prot       opt   source                                      destination
DROP               all         --     anywhere                                  anywhere             state INVALID
DROP              udp        --     anywhere                                  anywhere             udp dpt:bootpc 
ACCEPT           all          --     anywhere                                  anywhere            ctstate RELATED,ESTABLISHED
ACCEPT           icmp      --     anywhere                                  anywhere  
ACCEPT           all          --     anywhere                                  anywhere  
REJECT            all         --     anywhere                                  anywhere             reject-with icmp-host-prohibited
DROP               all         --     anywhere                                  anywhere  

Chain FORWARD (policy DROP)
target               prot       opt   source                                      destination
DROP               all         --     anywhere                                  anywhere            state INVALID
ACCEPT           all         --     anywhere                                  anywhere            ctstate RELATED,ESTABLISHED
QBS-FORWARD     all         --     anywhere                                  anywhere
DROP           all         --     anywhere                                  anywhere
ACCEPT           all         --     anywhere                                  anywhere
DROP           all         --     anywhere                                  anywhere

Chain OUTPUT (policy ACCEPT)
target               prot       opt   source                                      destination

Chain QBS-FORWARD (1 references)
target               prot       opt   source                                      destination
palainp commented 1 year ago

So I have exactly the same (classical Qubes, fedora sys-net). So there's probably a way to check into core-net the filter rules you set for mirage-fw. Sadly I didn't find something useful yet :(

palainp commented 9 months ago

@arkenoi, @froeb I recently saw that you've been trying to adapt liteqube to Qubes 4.2.

I'm throwing a bottle into the sea in case you might have an idea of what could cause issues with mirage-fw (not) acting as a killswitch when used in a VPN context. TLDR: on a classic Qubes 4.2 mirage-fw correctly acts as a killswitch as well as sys-fw, on liteqube mirage-fw doesn't but core-fw does.

palainp commented 2 weeks ago

@arkenoi you seem to have fully updated to Qubes 4.2 recently, do you know if this issue is still present in latest Qubes release?

arkenoi commented 1 week ago

I did not try yet, but I will.