Windows 10 Pro HVM does not work with Mirage Firewall

mirage / qubes-mirage-firewall

A Mirage firewall VM for QubesOS

BSD 2-Clause "Simplified" License

211 stars 28 forks source link

Windows 10 Pro HVM does not work with Mirage Firewall #56

Closed cgchinicz closed 5 years ago

cgchinicz commented 5 years ago

Hi Thomas,

First of all, not being much technical may prevent me from realizing all you've done to create this qubes alternative firewall but from what I've read about Mirage unikernel and tried with v0.5, I can say it's a really great advance for the Qubes community. I hope your work will be incorporated in future Qubes releases as an alternative to sys-firewall.

I have a Mirage Firewall VM v.05 that works with HVM (Linux Mint) and Debian/Fedora template-based PVMs.

My Windows 10 HVM works with sys-firewall using auto DHCP, while on Linux Mint I have to setup IP, mask and gateway (works with both sys-firewall and mirage-firewall as well).

On Windows Mirage did not work and I've tried setting up networking manually by adding its IP, mask and gateway and rebooting but it did not work.

I've followed all the ideas from here (https://www.windowscentral.com/how-regain-internet-access-after-installing-update-windows-10) and it still did not work.

One last piece of information, my Windows 10 Pro was successfully activated using a key I provided.

Any ideas? This is not critical, since I can continue using sys-firewall, but would love to free some memory by using Mirage.

Below I've added logs from guest-mirage-firewall.log. My Windows VM is 10.137.0.21.

Best Regards,

Claudio Chinicz

2019-04-18 11:20:10 -00:00: INF [client_net] Client 18 (IP: 10.137.0.21) ready 2019-04-18 11:20:10 -00:00: INF [ethernet] Connected Ethernet interface 00:16:3e:5e:6c:00 2019-04-18 11:20:11 -00:00: INF [client_net] add client vif {domid=17;device_id=0} 2019-04-18 11:20:11 -00:00: INF [qubes.db] got rm "/qubes-iptables-domainrules/" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/qubes-iptables-header" = "# Generated by Qubes Core on Thu Apr 18 14:20:11 2019\nfilter\n:INPUT DROP [0:0]\n:FORWARD DROP [0:0]\n:OUTPUT ACCEPT [0:0]\n-A INPUT -i vif+ -p udp -m udp --dport 68 -j DROP\n-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A INPUT -p icmp -j ACCEPT\n-A INPUT -i lo -j ACCEPT\n-A INPUT -j REJECT --reject-with icmp-host-prohibited\n-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A FORWARD -i vif+ -o vif+ -j DROP\nCOMMIT\n" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/10" = "filter\n-A FORWARD -s 10.137.0.18 -j ACCEPT\n-A FORWARD -s 10.137.0.18 -j DROP\nCOMMIT\n" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/17" = "filter\n-A FORWARD -s 10.137.0.21 -j ACCEPT\n-A FORWARD -s 10.137.0.21 -j DROP\nCOMMIT\n" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/14" = "filter\n-A FORWARD -s 10.137.0.13 -j ACCEPT\n-A FORWARD -s 10.137.0.13 -j DROP\nCOMMIT\n" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/9" = "*filter\n-A FORWARD -s 10.137.0.8 -j ACCEPT\n-A FORWARD -s 10.137.0.8 -j DROP\nCOMMIT\n" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/qubes-iptables" = "reload" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-ip" = "10.137.0.21" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-gateway" = "10.137.0.23" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/0000" = "action=accept" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/policy" = "drop" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21" = "" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got rm "/qubes-firewall/10.137.0.21/" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/0000" = "action=accept" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/policy" = "drop" 2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21" = "" 2019-04-18 11:20:22 -00:00: INF [client_eth] who-has 10.137.0.1? 2019-04-18 11:20:22 -00:00: INF [client_eth] unknown address; not responding 2019-04-18 11:20:22 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.21 (dropping) 2019-04-18 11:20:22 -00:00: WRN [client_net] Incorrect source IP 10.137.0.1 in IP packet from 10.137.0.21 (dropping) 2019-04-18 11:20:22 -00:00: INF [client_eth] who-has 10.137.0.1? 2019-04-18 11:20:22 -00:00: INF [client_eth] unknown address; not responding 2019-04-18 11:20:22 -00:00: INF [client_eth] who-has 10.137.0.21? 2019-04-18 11:20:22 -00:00: INF [client_eth] ignoring request for client's own IP 2019-04-18 11:20:22 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 46e6, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-18 11:20:22 -00:00: INF [client_eth] who-has 10.137.0.21? 2019-04-18 11:20:22 -00:00: INF [client_eth] ignoring request for client's own IP 2019-04-18 11:20:22 -00:00: INF [client_eth] who-has 10.137.0.1? 2019-04-18 11:20:22 -00:00: INF [client_eth] unknown address; not responding 2019-04-18 11:20:22 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 46e7, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-18 11:20:22 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 46e8, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-18 11:20:22 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.21 (dropping) 2019-04-18 11:20:22 -00:00: WRN [client_net] Incorrect source IP 10.137.0.1 in IP packet from 10.137.0.21 (dropping) 2019-04-18 11:20:22 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 46e9, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-18 11:20:22 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id e7de, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-04-18 11:20:22 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id 211e, off 0 proto 17, ttl 1, options UDP port 53180 -> 5355) 2019-04-18 11:20:22 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id e7df, off 0 proto 17, ttl 1, options

talex5 commented 5 years ago

2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-ip" = "10.137.0.21"
2019-04-18 11:20:11 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-gateway" = "10.137.0.23"

I think this means that Windows should expect its gateway (the firewall) to appear at 10.137.0.23.

However, Windows seems to be trying to use 10.137.0.1:

 2019-04-18 11:20:22 -00:00: INF [client_eth] who-has 10.137.0.1?
 2019-04-18 11:20:22 -00:00: INF [client_eth] unknown address; not responding

Check that the gateway IP address is configured correctly in Windows. The firewall doesn't currently run a DHCP service, so that won't work.

cgchinicz commented 5 years ago

Hi Thomas,

Thanks for your attention on this issue. The logs were produced with Windows networking configured for auto DHCP, which now I understand cannot work since Mirage does not provide DHCP server services.

What really intrigues me is that I've also tried setting up Windows networking manually (IP, mask and gateway) and it did not work. This is what I do with Linux Mint (also a HVM), where I have to set up manually IP, mask and gateway and it works with Mirage FW.

I've also tried with my Personal VM (based on the Debian 9 template) and it works automatically, which I presume is due to the QubesDB integration, from where the VM can get its IP, mask and its gateway (FW) IP's.

One point to note is that both my Windows and Linux Mint HVMs show up as yellow in Qubes manager, meaning that not all 3 services/integrations are available. Obviously "Xterminal" works but either or both of "qrexec" (this I know for sure is not available for both Windows and Linux since when I try to attach an USB a get the same error message on both saying qrexec is not present in the VM) or/and "QubesDB" are not integrated between the HVM and Qubes. Maybe here lies the difference in the behavior between Windows and Linux Mint (both 64 bits) HVMs.

Can it be related to the fact that Mirage is PV and not PVH?

Would love to see this issue solved and be able to use Mirage for both Windows and Linux Mint HVMs. I use two pairs of sys-net/FW, one for eth (cabled) and another one for wifi and the reason is that, at the office, I need the cabled connection in order to have connectivity to corporate resources like net printers. Neither Mirage-cabled/Mirage-wifi work with Windows and both of the work with Linux Mint and my Personal VM (Debian 9 based).

Best Regards,

Claudio

talex5 commented 5 years ago

You'll need to get the logs from a run where Windows was configured manually. That will probably show what the problem is.

cgchinicz commented 5 years ago

Hi Thomas,

I've configured windows networking manually (IP=10.137.0.21; mask=255.255.255.255 and gateway=10.137.0.23). Please see the logs below.

Best Regards,

Claudio

2019-04-23 17:36:08 -00:00: INF [client_net] Client 18 (IP: 10.137.0.21) ready 2019-04-23 17:36:08 -00:00: INF [ethernet] Connected Ethernet interface 00:16:3e:5e:6c:00 2019-04-23 17:36:09 -00:00: INF [client_net] add client vif {domid=17;device_id=0} 2019-04-23 17:36:09 -00:00: INF [qubes.db] got rm "/qubes-iptables-domainrules/" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/qubes-iptables-header" = "# Generated by Qubes Core on Tue Apr 23 20:36:09 2019\nfilter\n:INPUT DROP [0:0]\n:FORWARD DROP [0:0]\n:OUTPUT ACCEPT [0:0]\n-A INPUT -i vif+ -p udp -m udp --dport 68 -j DROP\n-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A INPUT -p icmp -j ACCEPT\n-A INPUT -i lo -j ACCEPT\n-A INPUT -j REJECT --reject-with icmp-host-prohibited\n-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A FORWARD -i vif+ -o vif+ -j DROP\nCOMMIT\n" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/9" = "filter\n-A FORWARD -s 10.137.0.18 -j ACCEPT\n-A FORWARD -s 10.137.0.18 -j DROP\nCOMMIT\n" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/17" = "filter\n-A FORWARD -s 10.137.0.21 -j ACCEPT\n-A FORWARD -s 10.137.0.21 -j DROP\nCOMMIT\n" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/11" = "filter\n-A FORWARD -s 10.137.0.8 -j ACCEPT\n-A FORWARD -s 10.137.0.8 -j DROP\nCOMMIT\n" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/qubes-iptables" = "reload" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-ip" = "10.137.0.21" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-gateway" = "10.137.0.23" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/0000" = "action=accept" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/policy" = "drop" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21" = "" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got rm "/qubes-firewall/10.137.0.21/" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/0000" = "action=accept" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/policy" = "drop" 2019-04-23 17:36:09 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21" = "" 2019-04-23 17:36:22 -00:00: INF [client_eth] who-has 169.254.154.187? 2019-04-23 17:36:22 -00:00: INF [client_eth] unknown address; not responding 2019-04-23 17:36:23 -00:00: INF [client_eth] who-has 169.254.154.187? 2019-04-23 17:36:23 -00:00: INF [client_eth] unknown address; not responding 2019-04-23 17:36:24 -00:00: INF [client_eth] who-has 169.254.154.187? 2019-04-23 17:36:24 -00:00: INF [client_eth] unknown address; not responding 2019-04-23 17:36:25 -00:00: INF [client_eth] who-has 169.254.154.187? 2019-04-23 17:36:25 -00:00: INF [client_eth] unknown address; not responding 2019-04-23 17:36:25 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 169.254.154.187 -> 224.0.0.22: id 3dbb, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:36:25 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 169.254.154.187 -> 224.0.0.22: id 3dbc, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:36:25 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 169.254.154.187 -> 224.0.0.22: id 3dbd, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:36:25 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 169.254.154.187 -> 224.0.0.22: id 3dbe, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:36:25 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:25 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:25 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:25 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:25 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:25 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:26 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 169.254.154.187 -> 224.0.0.22: id 3dbf, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:36:26 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:26 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:26 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:27 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:27 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:27 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:28 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:28 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:28 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:36 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:36 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:36 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:36 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:36 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:36 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:36 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:37 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:37 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:37 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:38 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:38 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:38 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:38 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:39 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:39 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:39 -00:00: INF [client_eth] who-has 10.137.0.23? 2019-04-23 17:36:39 -00:00: INF [client_eth] responding to: who-has 10.137.0.23? 2019-04-23 17:36:39 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:39 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:39 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:39 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:39 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:39 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:40 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:40 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:40 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:40 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 169.254.154.187 -> 224.0.0.22: id 3dc0, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:36:40 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:40 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:41 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:41 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:41 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:41 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:41 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:41 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 169.254.154.187 -> 224.0.0.22: id 3dc1, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:36:41 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:41 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:41 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:41 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:42 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:42 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:36:42 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:37:24 -00:00: INF [client_eth] who-has 10.137.0.23? 2019-04-23 17:37:24 -00:00: INF [client_eth] responding to: who-has 10.137.0.23? 2019-04-23 17:37:54 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 169.254.154.187 -> 224.0.0.22: id 3dc2, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:37:54 -00:00: INF [client_eth] who-has 10.137.0.23? 2019-04-23 17:37:54 -00:00: INF [client_eth] responding to: who-has 10.137.0.23? 2019-04-23 17:37:54 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 169.254.154.187 -> 224.0.0.22: id 3dc3, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:37:54 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:37:54 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:37:54 -00:00: WRN [client_net] Incorrect source IP 169.254.154.187 in IP packet from 10.137.0.21 (dropping) 2019-04-23 17:37:54 -00:00: INF [client_eth] who-has 10.137.0.21? 2019-04-23 17:37:54 -00:00: INF [client_eth] ignoring request for client's own IP 2019-04-23 17:37:54 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 169.254.154.187 -> 224.0.0.22: id 3dc4, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:37:55 -00:00: INF [client_eth] who-has 10.137.0.23? 2019-04-23 17:37:55 -00:00: INF [client_eth] responding to: who-has 10.137.0.23? 2019-04-23 17:37:55 -00:00: INF [client_eth] who-has 10.137.0.21? 2019-04-23 17:37:55 -00:00: INF [client_eth] ignoring request for client's own IP 2019-04-23 17:37:56 -00:00: INF [client_eth] who-has 10.137.0.23? 2019-04-23 17:37:56 -00:00: INF [client_eth] responding to: who-has 10.137.0.23? 2019-04-23 17:37:56 -00:00: INF [client_eth] who-has 10.137.0.21? 2019-04-23 17:37:56 -00:00: INF [client_eth] ignoring request for client's own IP 2019-04-23 17:37:57 -00:00: INF [client_eth] who-has 10.137.0.23? 2019-04-23 17:37:57 -00:00: INF [client_eth] responding to: who-has 10.137.0.23? 2019-04-23 17:37:57 -00:00: INF [client_eth] who-has 10.137.0.21? 2019-04-23 17:37:57 -00:00: INF [client_eth] ignoring request for client's own IP 2019-04-23 17:37:57 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 164b, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:37:57 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 164c, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:37:57 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 164d, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:37:57 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 164e, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-23 17:37:57 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 91fb, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-04-23 17:37:57 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id 2bf3, off 0 proto 17, ttl 1, options UDP port 50914 -> 5355) 2019-04-23 17:37:57 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 91fc, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-04-23 17:37:57 -00:00: INF [client_eth] who-has 10.137.0.23? 2019-04-23 17:37:57 -00:00: INF [client_eth] responding to: who-has 10.137.0.23?

yomimono commented 5 years ago

2019-04-23 17:36:22 -00:00: INF [client_eth] who-has 169.254.154.187? 2019-04-23 17:36:22 -00:00: INF [client_eth] unknown address; not responding

It looks very much like the Windows VM is not respecting your manual IP settings, and rather replacing them with a link-local IPv4 address.

cgchinicz commented 5 years ago

Hi,

I may be an "advanced" (??) Windows user but far from a technical person myself. So, I'll add below what I've configured, hoping I missed something and the issue lies with the user..

I've opened control panel --> Network & Internet --> Network and Sharing Center --> clicked on Connections: Ethernet --> Properties --> marked Internet Protocol Version 4 (TCP/IPv4) --> opened Properties and set up (in General) and filled up as below:

Please let me know if I miss something.

Regards,

Claudio

talex5 commented 5 years ago

I don't use Windows, but Google turns up various articles with people having similar problems setting a static IP, e.g. https://superuser.com/questions/987479/how-to-force-windows-to-use-assigned-static-ip-and-not-allow-default-ip-to-be-us

The logs do show some use of the correct IP address, e.g.

2019-04-23 17:37:57 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 91fb, off 0 proto 17, ttl 1, options
UDP port 5353 -> 5353)

(looks like it's checking for local devices using Bonjour and getting blocked, which I think is fine)

Things you could try:

Disable APIPA as in the superuser.com question above to avoid confusion.
Install https://www.wireshark.org/ and see if that shows anything interesting from the Windows site. e.g. is Windows receiving the ARP responses the firewall says it is sending above?

cgchinicz commented 5 years ago

Hi Thomas,

I've disabled APIPA, setup networking manually (as above), downloaded/installed wireshark and rebooted. After rebooting I've double checked and APIPA is disabled ("DWORD value named IPAutoconfigurationEnabled and set its value to 0").

It did not work (Ethernet connected but no internet) and I do not know how to get the required information from wireshark. It did not even start automatically as I thought it would in order to capture the initial traffic during boot/login.

Can you provide some basic guidance on how to get this information?

Thanks, Claudio

talex5 commented 5 years ago

Can you send the mirage logs with the new configuration?

It's OK if wireshark isn't running at boot. Just run it, start capturing on the virtual ethernet device (or "any" if you're not sure which that is). On Linux, I double-click on "eth0" in the window that appears initially to start capturing. Then try to browse to http://www.google.com or similar in a web-browser. Watch the wireshark output and the firewall logs as you do this. If you can save the captured data as a .pcap file and post that somewhere, that would be great too!

In a fully working system, you will should see ARP requests and responses, DNS queries and responses, and HTTPS requests and responses.

cgchinicz commented 5 years ago

Hi Thomas,

Mirage logs:

2019-04-25 08:17:05 -00:00: INF [client_net] add client vif {domid=16;device_id=0} 2019-04-25 08:17:06 -00:00: INF [client_net] Client 16 (IP: 10.137.0.21) ready 2019-04-25 08:17:06 -00:00: INF [ethernet] Connected Ethernet interface 00:16:3e:5e:6c:00 2019-04-25 08:17:06 -00:00: INF [client_net] add client vif {domid=15;device_id=0} 2019-04-25 08:17:06 -00:00: INF [qubes.db] got rm "/qubes-iptables-domainrules/" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/qubes-iptables-header" = "# Generated by Qubes Core on Thu Apr 25 11:17:06 2019\nfilter\n:INPUT DROP [0:0]\n:FORWARD DROP [0:0]\n:OUTPUT ACCEPT [0:0]\n-A INPUT -i vif+ -p udp -m udp --dport 68 -j DROP\n-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A INPUT -p icmp -j ACCEPT\n-A INPUT -i lo -j ACCEPT\n-A INPUT -j REJECT --reject-with icmp-host-prohibited\n-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A FORWARD -i vif+ -o vif+ -j DROP\nCOMMIT\n" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/9" = "filter\n-A FORWARD -s 10.137.0.18 -j ACCEPT\n-A FORWARD -s 10.137.0.18 -j DROP\nCOMMIT\n" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/15" = "filter\n-A FORWARD -s 10.137.0.21 -j ACCEPT\n-A FORWARD -s 10.137.0.21 -j DROP\nCOMMIT\n" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/11" = "filter\n-A FORWARD -s 10.137.0.8 -j ACCEPT\n-A FORWARD -s 10.137.0.8 -j DROP\nCOMMIT\n" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/qubes-iptables" = "reload" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-ip" = "10.137.0.21" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-gateway" = "10.137.0.23" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/policy" = "drop" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/0000" = "action=accept" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21" = "" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got rm "/qubes-firewall/10.137.0.21/" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/policy" = "drop" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/0000" = "action=accept" 2019-04-25 08:17:06 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21" = "" 2019-04-25 08:17:16 -00:00: INF [client_eth] who-has 10.137.0.23? 2019-04-25 08:17:16 -00:00: INF [client_eth] responding to: who-has 10.137.0.23? 2019-04-25 08:17:16 -00:00: INF [client_eth] who-has 10.137.0.21? 2019-04-25 08:17:16 -00:00: INF [client_eth] ignoring request for client's own IP 2019-04-25 08:17:16 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id f322, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-25 08:17:16 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id e1d2, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-04-25 08:17:16 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id e1d3, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-04-25 08:17:16 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id f323, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-25 08:17:16 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id f324, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-25 08:17:16 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id e1d4, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-04-25 08:17:16 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id e1d5, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-04-25 08:17:16 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id ed7d, off 0 proto 17, ttl 1, options UDP port 63277 -> 5355) 2019-04-25 08:17:16 -00:00: INF [client_eth] who-has 10.137.0.23? 2019-04-25 08:17:16 -00:00: INF [client_eth] responding to: who-has 10.137.0.23? 2019-04-25 08:17:16 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id f325, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-25 08:17:16 -00:00: INF [client_eth] who-has 10.137.0.21? 2019-04-25 08:17:16 -00:00: INF [client_eth] ignoring request for client's own IP 2019-04-25 08:17:16 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id f326, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-25 08:17:16 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id f327, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-25 08:17:16 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id e1d6, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-04-25 08:17:16 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id e1d7, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-04-25 08:17:16 -00:00: INF [client_eth] who-has 10.137.0.23? 2019-04-25 08:17:16 -00:00: INF [client_eth] responding to: who-has 10.137.0.23? 2019-04-25 08:17:16 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id f328, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-25 08:17:16 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id f329, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-25 08:17:16 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id f32a, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-25 08:17:16 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id e1d8, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-04-25 08:17:16 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id ed7e, off 0 proto 17, ttl 1, options UDP port 51148 -> 5355) 2019-04-25 08:17:16 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id e1d9, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-04-25 08:17:16 -00:00: INF [client_eth] who-has 10.137.0.21? 2019-04-25 08:17:16 -00:00: INF [client_eth] ignoring request for client's own IP 2019-04-25 08:17:16 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id f32b, off 0 proto 2, ttl 1, options 94 04 00 00 2019-04-25 08:17:17 -00:00: INF [client_eth] who-has 10.137.0.23? 2019-04-25 08:17:17 -00:00: INF [client_eth] responding to: who-has 10.137.0.23?

The wireshark logs you can see them here:

https://drive.google.com/file/d/1TIlDyqraODyd1pVPTTiHIxKw3Qc9FYUA/view?usp=sharing

Best Regards, Claudio

talex5 commented 5 years ago

All the packets in the wireshark log seem to be for the loopback interface rather than the ethernet device. Did you select any as the interface (in which case it just means there weren't any sent over ethernet) or did you select loopback (in which case please try again with the ethernet interface)?

I suspect we're not seeing the ethernet device here, because it doesn't show Windows sending any ARP requests, but the firewall is receiving lots of them.

Oddly, Windows seems to be sending ICMP Host unreachable messages to itself for src=10.137.0.21, dst=8.8.4.4, which is odd if it has a default gateway configured. According to the Internet, the command route print should display this on Windows, and arp -a will display the ARP table (which should indicate if Windows got any responses).

cgchinicz commented 5 years ago

Hi Thomas,

I've generated another wireshark logs file because I'm not sure how I did it before. This time I made sure it was for all interfaces. Please see it here: https://drive.google.com/open?id=1vO4DJx_2AlA3WPqy7gOuHX989xjyigeR

The result of route print:

C:\Windows\system32>route print
===========================================================================
Interface List
  4...00 16 3e 5e 6c 00 ......Realtek RTL8139C+ Fast Ethernet NIC
  8...02 00 4c 4f 4f 50 ......Npcap Loopback Adapter
  1...........................Software Loopback Interface 1
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0      10.137.0.23      10.137.0.21    281
      10.137.0.21  255.255.255.255         On-link       10.137.0.21    281
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    331
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    331
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    331
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    331
        224.0.0.0        240.0.0.0         On-link       10.137.0.21    281
        224.0.0.0        240.0.0.0         On-link                 8    281
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    331
  255.255.255.255  255.255.255.255         On-link       10.137.0.21    281
  255.255.255.255  255.255.255.255         On-link                 8    281
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
          0.0.0.0          0.0.0.0      10.137.0.23  Default
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
 If Metric Network Destination      Gateway
  1    331 ::1/128                  On-link
  4    281 fe80::/64                On-link
  8    281 fe80::/64                On-link
  4    281 fe80::8815:204b:88c9:9abb/128
                                    On-link
  8    281 fe80::b45e:f5f2:a0f5:fd77/128
                                    On-link
  1    331 ff00::/8                 On-link
  4    281 ff00::/8                 On-link
  8    281 ff00::/8                 On-link
===========================================================================
Persistent Routes:
  None

The result of arp -a:

C:\Windows\system32>arp -a

Interface: 10.137.0.21 --- 0x4
  Internet Address      Physical Address      Type
  224.0.0.22            01-00-5e-00-00-16     static
  224.0.0.251           01-00-5e-00-00-fb     static
  224.0.0.252           01-00-5e-00-00-fc     static
  239.255.255.250       01-00-5e-7f-ff-fa     static

Regards,

Claudio

talex5 commented 5 years ago

The new wireshark logs show Windows sending ARP queries and not getting any replies. The ARP table doesn't suggest it got any replies either. I assume that the firewall was logging that it was sending replies, as in earlier traces?

Unfortunately, that probably means there is some disagreement between the Windows and Mirage network device drivers, which will be difficult to debug without a Windows driver expert.

Are you using the Windows PV network driver, or the virtual PCI devices provided by QEMU? There seems to be a choice, according to https://wiki.xen.org/wiki/Xen_Windows_GplPv. It's possible that switching to the other option will help. Note: it seems that the PV driver on that page has been replaced by https://xenproject.org/windows-pv-drivers/

cgchinicz commented 5 years ago

Hi Thomas,

There are 5 options, which one (or ones, or all) should I install?

WINDOWS PV 8.2.2 BUS DRIVER (XENBUS.TAR) WINDOWS PV 8.2.2 INTERFACE (XENIFACE.TAR) WINDOWS PV 8.2.2 NETWORK CLASS DRIVER (XENVIF.TAR) WINDOWS PV 8.2.2 NETWORK DEVICE DRIVER (XENNET.TAR) WINDOWS PV 8.2.2 STORAGE HOST ADAPTER DRIVER (XENVBD.TAR)

Thanks,

Claudio

talex5 commented 5 years ago

It might be best to ask on the Qubes list. I don't think we have any Windows devs reading here. But my guess would be everything except the storage one. It looks like these also come with the Qubes Windows Tools... but it seems that doesn't work properly on Windows 10 yet. You might be able to install just the networking bits that way, though.

https://www.qubes-os.org/doc/windows/

cgchinicz commented 5 years ago

I've asked on the qubes list and tried to install all 4 packages (except the one related to storage) and they all failed to install. Once I've tried the qubes windows tools and lost the VM after a few boots. Actually, before it stopped booting it appeared "green" on the qubes manager, meaning it was fully integrated. But after turning off it did not boot again.

I have two sys-net VMs, one for wireless and another one for eth cabled because my Linux Mint VM would loose networking when I connected the eth cable. So, I already use and benefit from Mirage Firewall since I use it with Linux Mint (wifi) VM. It's just that I wanted to also replace the sys-firewall that I use with windows with another mirage firewall.

Thanks for trying.

Regards,

Claudio

talex5 commented 5 years ago

I can reproduce this problem on HVM OpenBSD if I relink the kernel without the xnf (PV) driver so that it uses the re (Realtek) driver instead.

I also found I could use sudo xl console to get a console on the stub domain. Removing eth0 from the bridge, configuring eth0 manually, and trying to use that also showed the same problem. The guest logs for the -dm domain also show a surprising number of kernel stack dumps (but without symbols), e.g

Linux version 4.14.68-xen-stubdom (user@build-fedora4) (gcc version 6.4.1 20170727 (Red Hat 6.4.1-1) (GCC)) #1 Tue Oct 2 03:34:17 UTC 2018
Command line: debug console=hvc0
x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
Released 0 page(s)
e820: BIOS-provided physical RAM map:
Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
Xen: [mem 0x0000000000100000-0x0000000008ffffff] usable
NX (Execute Disable) protection: active
Hypervisor detected: Xen PV
tsc: Fast TSC calibration failed
tsc: Unable to calibrate against PIT
tsc: No reference (HPET/PMTIMER) available
e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
e820: remove [mem 0x000a0000-0x000fffff] usable
e820: last_pfn = 0x9000 max_arch_pfn = 0x400000000
x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WC  WP  UC  UC  
Base memory trampoline at [ffff88000009a000] 9a000 size 24576
BRK [0x01b47000, 0x01b47fff] PGTABLE
RAMDISK: [mem 0x01c00000-0x033b1fff]
Zone ranges:
  DMA32    [mem 0x0000000000001000-0x0000000008ffffff]
  Normal   empty
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000000001000-0x000000000009ffff]
  node   0: [mem 0x0000000000100000-0x0000000008ffffff]
Initmem setup node 0 [mem 0x0000000000001000-0x0000000008ffffff]
On node 0 totalpages: 36767
  DMA32 zone: 576 pages used for memmap
  DMA32 zone: 21 pages reserved
  DMA32 zone: 36767 pages, LIFO batch:7
p2m virtual area at ffffc90000000000, size is 200000
Remapped 0 page(s)
e820: [mem 0x09000000-0xffffffff] available for PCI devices
Booting paravirtualized kernel on Xen
Xen version: 4.8.4 (preserve-AD)
clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
random: get_random_bytes called from 0xffffffff81a5e5a1 with crng_init=0
pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
pcpu-alloc: [0] 0 
Built 1 zonelists, mobility grouping on.  Total pages: 36170
Kernel command line: debug console=hvc0
PID hash table entries: 1024 (order: 1, 8192 bytes)
Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
Inode-cache hash table entries: 16384 (order: 5, 131072 bytes)
Memory: 105912K/147068K available (6152K kernel code, 343K rwdata, 368K rodata, 656K init, 280K bss, 41156K reserved, 0K cma-reserved)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Using NULL legacy PIC
NR_IRQS: 4352, nr_irqs: 24, preallocated irqs: 0
xen:events: Using FIFO-based ABI
console [hvc0] enabled
clocksource: xen: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
Xen: using vcpuop timer interface
installing Xen timer for CPU 0
tsc: Unable to calibrate against PIT
tsc: No reference (HPET/PMTIMER) available
tsc: Detected 2194.936 MHz processor
Calibrating delay loop (skipped), value calculated using timer frequency.. 4389.87 BogoMIPS (lpj=8779744)
pid_max: default: 4096 minimum: 301
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes)
ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
mce: CPU supports 2 MCE banks
Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8
Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
CPU: Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz (family: 0x6, model: 0x3d, stepping: 0x4)
Spectre V2 : Mitigation: Full generic retpoline
Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
Spectre V2 : Spectre v2 mitigation: Enabling Indirect Branch Prediction Barrier
Spectre V2 : Enabling Restricted Speculation for firmware calls
Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
Performance Events: unsupported p6 CPU model 61 no PMU driver, software events only.
Not enabling interrupt remapping due to skipped IO-APIC setup
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at 0xffffffff8100de2e
CPU: 0 PID: 1 Comm: swapper Not tainted 4.14.68-xen-stubdom #1
task: ffff880000025200 task.stack: ffffc90000200000
RIP: e030:0xffffffff8100de2e
RSP: e02b:ffffc90000203ed0 EFLAGS: 00010203
RAX: ffffffff8100de2e RBX: a9e8a33010c5ae52 RCX: ffffffff81a675cd
RDX: 1845987f2a63fd6f RSI: ffffc90000203d44 RDI: 0000000000000000
RBP: ffffc90000203ee8 R08: 000000000000000a R09: ffffffff81b1a570
R10: 0000000000000001 R11: 0000000000000060 R12: 0000000000000000
R13: 5ca2c32f5e78e9b8 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffffffff81a2a000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000001a12000 CR4: 0000000000042660
Call Trace:
 ? 0xffffffff81a78e8f
 ? 0xffffffff81a675cd
 ? 0xffffffff81a78fdd
 ? 0xffffffff81a5ed95
 ? 0xffffffff81217a24
 ? 0xffffffff81217a29
 ? 0xffffffff814001b5
Code: c2 48 83 f8 ff 48 0f 45 c2 c3 48 c7 c0 a8 5a 80 81 c3 48 c7 06 ff 00 00 00 c3 48 89 f8 b9 00 04 00 00 48 89 f7 48 89 c6 f3 a5 c3 <0f> 0b 89 f8 c3 89 f8 c1 e8 18 c3 31 c0 c3 31 c0 c3 0f 0b c3 31 
---[ end trace 0e37b648b53d7277 ]---
register: 20, value: 0
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at 0xffffffff8100deb2
CPU: 0 PID: 1 Comm: swapper Tainted: G        W       4.14.68-xen-stubdom #1
task: ffff880000025200 task.stack: ffffc90000200000
RIP: e030:0xffffffff8100deb2
RSP: e02b:ffffc90000203ed0 EFLAGS: 00010286
RAX: 0000000000000016 RBX: a9e8a33010c5ae52 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffffc90000203d44 RDI: 0000000000000200
RBP: ffffc90000203ee8 R08: 000000000000020a R09: ffffffff81b1ac04
R10: 0000000000000001 R11: 0000000000000038 R12: 0000000000000000
R13: 5ca2c32f5e78e9b8 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffffffff81a2a000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000001a12000 CR4: 0000000000042660
Call Trace:
 ? 0xffffffff81a78e9b
 ? 0xffffffff81a675cd
 ? 0xffffffff81a78fdd
 ? 0xffffffff81a5ed95
 ? 0xffffffff81217a24
 ? 0xffffffff81217a29
 ? 0xffffffff814001b5
Code: c3 48 63 f6 31 c0 48 0f a3 37 0f 92 c0 c3 81 ff 40 03 00 00 75 05 e9 a0 06 00 00 89 f2 89 fe 48 c7 c7 9f 3c 83 81 e8 5f ec 03 00 <0f> 0b c3 55 53 89 f5 89 fb 48 83 ec 18 65 48 8b 04 25 28 00 00 
---[ end trace 0e37b648b53d7278 ]---
pmu_apic_update: pmudata not initialized
register: 80, value: 0
[ ... more of the same ... ]
---[ end trace 0e37b648b53d727d ]---
devtmpfs: initialized
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
futex hash table entries: 16 (order: -4, 384 bytes)
NET: Registered protocol family 16
xen:grant_table: Grant tables using version 1 layout
Grant table initialized
PCI: setting up Xen PCI frontend stub
PCI: pci_cache_line_size set to 64 bytes
xen:balloon: Initialising balloon driver
PCI: System does not support PCI
clocksource: Switched to clocksource xen
NET: Registered protocol family 2
TCP established hash table entries: 2048 (order: 2, 16384 bytes)
TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
UDP hash table entries: 128 (order: 0, 4096 bytes)
UDP-Lite hash table entries: 128 (order: 0, 4096 bytes)
Unpacking initramfs...
Freeing initrd memory: 24264K
workingset: timestamp_bits=62 max_order=15 bucket_order=0
io scheduler noop registered (default)
xen:xen_evtchn: Event-channel device installed
Invalid max_queues (4), will use default max: 1.
tun: Universal TUN/TAP device driver, 1.6
xen_netfront: Initialising Xen virtual ethernet driver
NET: Registered protocol family 17
sched_clock: Marking stable (36075481, 0)->(1687993761363499, -1687993725288018)
random: fast init done
blkfront: xvda: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled;
 xvda: xvda1 xvda2 xvda3
blkfront: xvdb: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled;
blkfront: xvdc: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled;
blkfront: xvdd: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled;
Freeing unused kernel memory: 656K
Write protecting the kernel read-only data: 10240k
Freeing unused kernel memory: 2032K
Freeing unused kernel memory: 1680K
+ mount -t devtmpfs none /dev
+ mount -t sysfs /sys /sys
+ mount -t proc /proc /proc
+ mount -t tmpfs -o 'size=1m,nodev,noexec' /tmp /tmp
+ mount -o remount,ro /
+ echo 1
+ test -e /sys/class/net/eth0
+ ip link set eth0 address fe:ff:ff:ff:ff:fe
+ ip addr flush eth0
+ ip link set eth0 up
+ brctl addbr br0
+ brctl addif br0 eth0
br0: port 1(eth0) entered blocking state
br0: port 1(eth0) entered disabled state
device eth0 entered promiscuous mode
+ ip link set br0 up
br0: port 1(eth0) entered blocking state
br0: port 1(eth0) entered forwarding state
+ xenstore-read device/vif/0/backend
+ xenstore-read /local/domain/261/backend/vif/286/0/ip
+ client_ip=10.137.0.4
+ client_ip=10.137.0.4
+ net_prefix=10.137.0
+ /bin/xenstore-read target
+ udhcpd -f -I 10.137.0.1 -
udhcpd: started, v1.29.3
+ domid=285
+ xenstore-read /local/domain/285/vm
+ vm_path=/vm/68f52574-e6c8-4a09-bdcb-984036f4ce81
+ xenstore-read -R /vm/68f52574-e6c8-4a09-bdcb-984036f4ce81/image/dmargs
+ dm_args='-xen-domid.285.-nodefaults.-no-user-config.-name.test-hvm.-display.none.-append.root=/dev/mapper/dmroot ro nomodeset console=hvc0 rd_NO_PLYMOUTH rd.plymouth.enable=0 plymouth.enable=0 nopat.-device.VGA,vgamem_mb=16.-boot.order=dc.-device.usb-ehci,id=ehci.-device.usb-tablet,bus=ehci.0.-smp.2,maxcpus=2.-device.rtl8139,id=nic0,netdev=net0,mac=00:16:3e:5e:6c:00.-netdev.type=tap,id=net0,ifname=vif285.0-emu,script=no,downscript=no.-display.qubes-gui.-machine.xenfv.-m.3984.-device.mptsas1068,id=scsi0.-drive.file=/dev/xvda,if=none,id=disk0,format=host_device,cache=writeback,readonly=off.-device.scsi-hd,bus=scsi0.0,drive=disk0,wwn=0x3525400051756265.-drive.file=/dev/xvdb,if=none,id=disk1,format=host_device,cache=writeback,readonly=off.-device.scsi-hd,bus=scsi0.0,drive=disk1,wwn=0x3525400051756266.-drive.file=/dev/xvdc,if=none,id=disk2,format=host_device,cache=writeback,readonly=off.-device.scsi-hd,bus=scsi0.0,drive=disk2,wwn=0x3525400051756267.-drive.file=/dev/xvdd,if=none,id=disk3,format=host_device,cache=writeback,readonly=on.-device.scsi-hd,bus=scsi0.0,drive=disk3,wwn=0x3525400051756268'
+ mkdir /tmp/qmp
+ mkdir /tmp/qmp/req
+ mkdir /tmp/qmp/res
+ kernel=
+ '[' -b /dev/xvdd ]
+ mkdir /tmp/boot
+ mount /dev/xvdd /tmp/boot -o ro
EXT4-fs (xvdd): mounting ext3 file system using the ext4 subsystem
EXT4-fs (xvdd): mounted filesystem with ordered data mode. Opts: (null)
+ '[' -f /tmp/boot/vmlinuz ]
+ kernel='-kernel./tmp/boot/vmlinuz'
+ '[' -f /tmp/boot/initramfs ]
+ kernel='-kernel./tmp/boot/vmlinuz.-initrd./tmp/boot/initramfs'
+ mkfifo /tmp/qmp/qemu.in /tmp/qmp/qemu.out /tmp/qmp/qemu_res.out
+ IFS='.'
+ set -f
+ set +f
+ unset IFS
+ qemu_pid=48
+ true
+ printf '==== Press enter for shell ====\n'
==== Press enter for shell ====

However, that seems to happen before initialising netfront, so maybe it's not our fault...

cgchinicz commented 5 years ago

Hi, happy to know that you could reproduce the issue and that it seems to confirm it is related to the PV driver.

Is there something else I can do or just wait till they fix qubes windows tools and make it Windows 10 compatible? Anyway, Windows 7 EOL is 2020..

Regards,

Claudio

talex5 commented 5 years ago

If you have time, you could try this patch: https://github.com/mirage/qubes-mirage-firewall/pull/61

cd qubes-mirage-firewall
git pull origin pull/61/head
sudo ./build-with-docker.sh

It fixed my OpenBSD VM anyway!

cgchinicz commented 5 years ago

Hi, I'd love to try it but I'll need help.. for instance, the three lines commands you wrote, should I issue them from dom0?

On the link you sent above (#61), there are two commands to issue on dom0, should I do it "as is" or do I need to replace part of it with particulars from my environment?

If you'd prefer, you can reply to my email cchinicz@gmail.com and move off-line from here until I can test this and then return and publish here the findings.

Regards, Claudio

talex5 commented 5 years ago

Oh, I was assuming you'd built the firewall from source. In that case, you run those commands in your build VM to update the source code to the new version and build it (deleting any existing _build directory first is a good idea too). See https://github.com/mirage/qubes-mirage-firewall/#build-from-source for details.

The dom0 commands were just to explain the problem - you don't need to run them.

cgchinicz commented 5 years ago

Hi,

I've not built it but rather downloaded the binary on dom0 from here https://github.com/mirage/qubes-mirage-firewall/releases/tag/v0.5

Is the binary mirage-firewall-bin-0.5.tar.bz2 updated with these changes (patch #61)?

If I download it again and install, will it replace the code on /var/lib/qubes/vm-kernels or do I need to remove directory /mirage-firewall before installing?

Sorry for bothering you with some basic stuff, its just that I lack on the technical side, although being an enthusiast of this technology.

Regards, Claudio

talex5 commented 5 years ago

You need to build from source to test these changes. You follow the instructions at https://github.com/mirage/qubes-mirage-firewall/#build-from-source, but after the cd qubes-mirage-firewall step you do git pull origin pull/61/head to add in the changes from #61 before building.

The result is a mirage-firewall.tar.bz2, which you install as before. It will overwrite the old files (actually, it's just the vmlinuz file inside the archive that needs to be updated). Then reboot the mirage-firewall VM.

cgchinicz commented 5 years ago

Hi Thomas,

I followed the instructions but there were errors and warnings that I extracted from the console (I've skipped most of the lines and included just the errors/warnings and some lines close to them to help you figure out where they happened). Please see below.

One question, it's written "Note: the object files are stored in the _build directory....". I could not find this _build directory, maybe because of the errors. Should it be under /docker or /qubes-mirage-firewall?

Thanks, Claudio

Get:15 http://deb.debian.org/debian stretch/main amd64 xml-core all 0.17 [23.2 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 16.1 MB in 2s (6060 kB/s)

[ERROR] The compilation of xenstore failed at "/home/opam/.opam/4.07/bin/dune build -p xenstore -j 71".
[ERROR] The compilation of shared-memory-ring failed at "/home/opam/.opam/4.07/bin/dune build -p shared-memory-ring -j 71".
[ERROR] The compilation of xen-gnt failed at "/home/opam/.opam/4.07/bin/dune build -p xen-gnt -j 71".

#=== ERROR while installing mirage-stack.1.4.0 ================================#
# path                 ~/.opam/4.07/.opam-switch/build/mirage-stack.1.4.0
# command              /usr/bin/install -m 0644 /home/opam/.opam/4.07/.opam-switch/build/mirage-stack.1.4.0/_build/install/default/lib/mirage-stack/mirage_stack.cmt /home/opam/.opam/4.07/lib/mirage-stack/mirage_stack.cmt
# exit-code            1
# env-file             ~/.opam/log/log-8-05304b.env
# output-file          ~/.opam/log/log-8-05304b.out

#=== ERROR while installing mirage-protocols-lwt.2.0.0 ========================#
"install": command not found.

#=== ERROR while installing functoria.2.2.3 ===================================#
# path                 ~/.opam/4.07/.opam-switch/build/functoria.2.2.3
# command              /usr/bin/install -m 0644 /home/opam/.opam/4.07/.opam-switch/build/functoria.2.2.3/_build/install/default/lib/functoria/functoria.cmti /home/opam/.opam/4.07/lib/functoria/functoria.cmti
# exit-code            1
# env-file             ~/.opam/log/log-8-01ae51.env
# output-file          ~/.opam/log/log-8-01ae51.out

#=== ERROR while compiling xenstore.2.1.0 =====================================#
# context              2.0.3 | linux/x86_64 | ocaml-base-compiler.4.07.1 | git+file:///home/opam/opam-repository
# path                 ~/.opam/4.07/.opam-switch/build/xenstore.2.1.0
# command              ~/.opam/4.07/bin/dune build -p xenstore -j 71
# exit-code            1
# env-file             ~/.opam/log/xenstore-8-ddda1d.env
# output-file          ~/.opam/log/xenstore-8-ddda1d.out
### output ###
#     ocamlopt .ppx/fae5e047b89b86eceafabfbc9ce59292/ppx.exe (exit 2)
# (cd _build/default && /home/opam/.opam/4.07/bin/ocamlopt.opt -o .ppx/fae5e047b89b86eceafabfbc9ce59292/ppx.exe -I /home/opam/.opam/4.07/lib/base/caml -I /home/opam/.opam/4.07/lib/ocaml-migrate-parsetree -I /home/opam/.opam/4.07/lib/ocaml/compiler-libs -I /home/opam/.opam/4.07/lib/parsexp -I /home/opam/.opam/4.07/lib/ppx_cstruct -I /home/opam/.opam/4.07/lib/ppx_derivers -I /home/opam/.opam/4.07[...]
# collect2: error: ld returned 1 exit status
# File "caml_startup", line 1:
# Error: Error during linking

#=== ERROR while compiling xen-gnt.4.0.0 ======================================#
# context              2.0.3 | linux/x86_64 | ocaml-base-compiler.4.07.1 | git+file:///home/opam/opam-repository
# path                 ~/.opam/4.07/.opam-switch/build/xen-gnt.4.0.0
# command              ~/.opam/4.07/bin/dune build -p xen-gnt -j 71
# exit-code            2
# env-file             ~/.opam/log/xen-gnt-8-f736a3.env
# output-file          ~/.opam/log/xen-gnt-8-f736a3.out

[cmdliner.1.0.3] synchronised from git+https://github.com/talex5/cmdliner.git#repro-builds
[WARNING] Failed checks on cmdliner package definition from source at git+https://github.com/talex5/cmdliner.git#repro-builds:
             error 57: Synopsis and description must not be both empty
cmdliner is now pinned to git+https://github.com/talex5/cmdliner.git#repro-builds (version 1.0.3)
 ---> 2674c4075db6

#=== ERROR while compiling shared-memory-ring.3.1.0 ===========================#
# context              2.0.3 | linux/x86_64 | ocaml-base-compiler.4.07.1 | git+file:///home/opam/opam-repository
# path                 ~/.opam/4.07/.opam-switch/build/shared-memory-ring.3.1.0
# command              ~/.opam/4.07/bin/dune build -p shared-memory-ring -j 71
# exit-code            2
# env-file             ~/.opam/log/shared-memory-ring-8-9acf8d.env
# output-file          ~/.opam/log/shared-memory-ring-8-9acf8d.out
### output ###
# Error: Assembler error, input left in file /tmp/camlstartup5f01c1.s
# [...]
# (cd _build/default && /home/opam/.opam/4.07/bin/ocamlmklib.opt -g -o lib/shared_memory_ring_stubs lib/barrier_stubs.o)
# collect2: error: ld returned 1 exit status
# Fatal error: exception Sys_error("No space left on device")
# Raised by primitive operation at file "stdlib.ml", line 377, characters 19-27
# Called from file "src/stdune/exn.ml", line 30, characters 19-28
# Called from file "list.ml", line 106, characters 12-15
# Called from file "src/hooks.ml", line 22, characters 4-50
# Called from file "stdlib.ml", line 545, characters 62-65
# Called from file "stdlib.ml" (inlined), line 548, characters 20-39
# Called from file "stdlib.ml", line 551, characters 2-15
# Called from file "bin/main_dune.ml", line 1, characters 9-21

The former state can be restored with:
    opam switch import "/home/opam/.opam/4.07/.opam-switch/backup/state-20190501133041.export"

talex5 commented 5 years ago

# Fatal error: exception Sys_error("No space left on device")

You might be out of disk space. Use df -h to see if some partition needs more. You might need to use Qubes Manager to give the build VM more.

cgchinicz commented 5 years ago

I had to delete that builder VM because I found out the space issue was not in the user partition but rather on the system partition and could not change it (it is greyed after VM creation). So, I've changed Fedora template system partition and set up to 20GB instead of 10GB and created a new builder VM.

Again, I got errors, mainly permission related. Please see below.

What should I do?

Thanks, Claudio

Get:15 http://deb.debian.org/debian stretch/main amd64 xml-core all 0.17 [23.2 kB] debconf: delaying package configuration, since apt-utils is not installed Fetched 16.1 MB in 2s (6985 kB/s)

[cmdliner.1.0.3] synchronised from git+https://github.com/talex5/cmdliner.git#repro-builds [WARNING] Failed checks on cmdliner package definition from source at git+https://github.com/talex5/cmdliner.git#repro-builds: error 57: Synopsis and description must not be both empty cmdliner is now pinned to git+https://github.com/talex5/cmdliner.git#repro-builds (version 1.0.3) ---> 9cb324f48bc8

Exception Sys_error("/home/opam/qubes-mirage-firewall/config.ml: Permission denied"). The command '/bin/sh -c opam config exec -- mirage configure -t xen && make depend' returned a non-zero code: 1

talex5 commented 5 years ago

It looks like it's nearly done! I haven't seen that error before. Try this to investigate (from the source code directory):

$ ls -ld . config.ml
drwxr-xr-x 6 user user 4096 May  1 17:38 ./
-rw-r--r-- 1 user user 1125 Apr 28 16:09 config.ml

$ docker run --rm -it 9cb324f48bc8

opam@...:~/qubes-mirage-firewall$ ls -ld . config.ml
drwxr-xr-x 1 opam opam 4096 May  1 14:56 .
-rw-r--r-- 1 root root 1125 Apr 28 16:09 config.ml

opam@...:~/qubes-mirage-firewall$ cat config.ml
[...]

(9cb324f48bc8 is the ID from your log output)

Check the permissions on the file and directory match what is shown above. If not, exit from the Docker container and adjust. If you started with a new Qube and followed the instructions I don't see why it wouldn't match, though.

(by the way, you can use ``` around output to stop GitHub trying to format it)

cgchinicz commented 5 years ago

I've found the issue sitting between the chair and the notebook.. I had issued commands from an elevated terminal, I did "sudo su" when I opened terminal on the builder VM to save typying sudo for each command.

I've deleted that builder and created a new one, where I've succeeded creating mirage-firewall.tar.bz2, copied it to dom0, resetted mirage-firewall VM and tested with Linux Mint and sys-whonix (which I also run with mirage) and all is working as before.

Then I changed networking for my Windows 10 HVM to work with mirage and started it. Unfortunately, it did not work. No network..

Any idea to investigate? thanks in advance for extra patience to handle this issue.

Best Regards, Claudio

talex5 commented 5 years ago

Same as before: check the firewall logs and wireshark as you try to view a web-site. See if it looks different this time.

Note, the format of the log messages has changed a bit. The old format looks like this:

2019-04-23 17:37:24 -00:00: INF [client_eth] who-has 10.137.0.23?
2019-04-23 17:37:24 -00:00: INF [client_eth] responding to: who-has 10.137.0.23?

The new format looks like this:

2019-05-01 14:10:36 -00:00: INF [client_eth] who-has 10.137.0.23? responding with fe:ff:ff:ff:ff:ff

If you see the old format message then you're still using the old version. In that case, check that you didn't forget the git pull step. If you did, go back and do that now and then continue from there (you don't need to delete the whole builder qube).

cgchinicz commented 5 years ago

Hi Thomas,

Please see below the mirage logs. It's working according to the new format (2019-05-02 09:45:24 -00:00: INF [client_eth] who-has 10.137.0.23? responding with fe:ff:ff:ff:ff:ff).

The wireshark file you can see here: https://drive.google.com/file/d/1STP04yyUA-1ONoengUQy6H5vlfdBUVie/view?usp=sharing

Looking forward to your return and next step.

Regards, Claudio

2019-05-02 09:38:20 -00:00: INF [net-xen:backend] Frontend asked to close network device dom:20/vif:0 2019-05-02 09:38:20 -00:00: INF [client_net] client {domid=20;device_id=0} has gone 2019-05-02 09:45:24 -00:00: INF [client_eth] who-has 10.137.0.23? responding with fe:ff:ff:ff:ff:ff 2019-05-02 09:46:42 -00:00: INF [client_net] add client vif {domid=23;device_id=0} with IP 10.137.0.21 2019-05-02 09:46:42 -00:00: INF [client_net] Client 23 (IP: 10.137.0.21) ready 2019-05-02 09:46:42 -00:00: INF [ethernet] Connected Ethernet interface fe:ff:ff:ff:ff:ff 2019-05-02 09:46:43 -00:00: INF [client_net] add client vif {domid=22;device_id=0} with IP 10.137.0.21 2019-05-02 09:46:43 -00:00: INF [qubes.db] got rm "/qubes-iptables-domainrules/" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/qubes-iptables-header" = "# Generated by Qubes Core on Thu May 2 12:46:43 2019\nfilter\n:INPUT DROP [0:0]\n:FORWARD DROP [0:0]\n:OUTPUT ACCEPT [0:0]\n-A INPUT -i vif+ -p udp -m udp --dport 68 -j DROP\n-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A INPUT -p icmp -j ACCEPT\n-A INPUT -i lo -j ACCEPT\n-A INPUT -j REJECT --reject-with icmp-host-prohibited\n-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A FORWARD -i vif+ -o vif+ -j DROP\nCOMMIT\n" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/20" = "filter\n-A FORWARD -s 10.137.0.18 -j ACCEPT\n-A FORWARD -s 10.137.0.18 -j DROP\nCOMMIT\n" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/22" = "filter\n-A FORWARD -s 10.137.0.21 -j ACCEPT\n-A FORWARD -s 10.137.0.21 -j DROP\nCOMMIT\n" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/8" = "filter\n-A FORWARD -s 10.137.0.8 -j ACCEPT\n-A FORWARD -s 10.137.0.8 -j DROP\nCOMMIT\n" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/qubes-iptables" = "reload" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-ip" = "10.137.0.21" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-gateway" = "10.137.0.23" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/0000" = "action=accept" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/policy" = "drop" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21" = "" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got rm "/qubes-firewall/10.137.0.21/" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/0000" = "action=accept" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/policy" = "drop" 2019-05-02 09:46:43 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21" = "" 2019-05-02 09:46:55 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-02 09:46:55 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-02 09:46:55 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id d692, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-02 09:46:55 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.21 (dropping) 2019-05-02 09:46:55 -00:00: WRN [client_net] Incorrect source IP 10.137.0.1 in IP packet from 10.137.0.21 (dropping) 2019-05-02 09:46:55 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 8cca, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-02 09:46:55 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 8ccb, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-02 09:46:55 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-02 09:46:55 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id d693, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-02 09:46:55 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-02 09:46:55 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id d694, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-02 09:46:55 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id d695, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-02 09:46:55 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 8ccc, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-02 09:46:55 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id abac, off 0 proto 17, ttl 1, options UDP port 59337 -> 5355) 2019-05-02 09:46:55 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 8ccd, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-02 09:46:55 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-02 09:46:55 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id d696, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-02 09:46:55 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id d697, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-02 09:46:55 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id d698, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-02 09:46:55 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 8cce, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-02 09:46:55 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id abad, off 0 proto 17, ttl 1, options UDP port 60496 -> 5355) 2019-05-02 09:46:55 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 8ccf, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-02 09:46:55 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-02 09:46:55 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id d699, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-02 09:46:55 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.21 (dropping) 2019-05-02 09:46:55 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-02 09:46:57 -00:00: WRN [client_net] Incorrect source IP 10.137.0.1 in IP packet from 10.137.0.21 (dropping) 2019-05-02 09:46:58 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.21 (dropping) 2019-05-02 09:46:58 -00:00: WRN [client_net] Incorrect source IP 10.137.0.1 in IP packet from 10.137.0.21 (dropping) 2019-05-02 09:46:58 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 408b, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-02 09:46:58 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 408c, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-02 09:46:58 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id ca2e, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-02 09:46:58 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id ff79, off 0 proto 17, ttl 1, options UDP port 63386 -> 5355) 2019-05-02 09:46:58 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id ca2f, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-02 09:46:58 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 408d, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-02 09:46:58 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-02 09:46:58 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 408e, off 0 proto 2, ttl 1, options

talex5 commented 5 years ago

In this trace, Windows keeps asking for 10.137.0.1, which doesn't sound right. Did the Windows IP configuration get reset? In the previous logs, Windows was asking for 10.137.0.23, which makes more sense. Check the gateway setting, etc.

cgchinicz commented 5 years ago

Hi Thomas,

I forgot to set DHCP.. once I configured DHCP I was able to connect to the web but could not print on corporate network printers under Microsoft/AD nor see any MS/DNS resources.

I've then returned to my previous setting with sys-firewall with auto DHCP and noted that it was using IPv4 DNS pointing to internal/corporate servers. I wrote down the IPs, set DHCP with these servers (and, of course, my IP, mask and gateway/mirage) and tested mirage firewall again, this time fully functional.

Thank you VERY much for your time and patience. Now Mirage for Qubes can be used by Windows 10 Pro users who happen to be connected to a domain for accessing corporate resources. I think this is an achievement the Qubes team should celebrate as they also target business users.

Best Regards, Claudio

talex5 commented 5 years ago

Great - thanks for testing!

I said above that DHCP wouldn't work because mirage-firewall doesn't run a DHCP server, but in fact it appears that Qubes runs a DHCP server in the stub domain. And OpenBSD does get the correct IP address from it. However, the DHCP server is configured to return as the "router" the client's IP, but with .1 on the end, which is the address of the DHCP server:

https://github.com/QubesOS/qubes-vmm-xen-stubdom-linux/blob/master/rootfs/init#L26

When the guest tries to use that route, it makes an ARP request for 10.137.0.1, which mirage-firewall ignores as an unknown address. I don't know how it's supposed to work.

cgchinicz commented 5 years ago

That explains all the "[client_eth] who-has 10.137.0.1? unknown address; not responding" messages.

But why it worked with the Linux Mint HVM before introducing the change on Mirage? was it because, as you wrote above, the Linux kernel has PV drivers while Windows does not?

talex5 commented 5 years ago

If you use PV drivers then you're not going via the stub-domain and so there's no bridge. Bridges are confused by having two devices with the same MAC address, whereas simple point-to-point links aren't.

I've asked about the strange 10.137.0.1 route on qubes-devel: https://groups.google.com/forum/#!topic/qubes-devel/2GzoiAIIG1c

talex5 commented 5 years ago

Looks like it's a Qubes bug (https://github.com/QubesOS/qubes-issues/issues/5022). I've added a work-around for it in #61.

If you want to test it with Windows:

cd qubes-mirage-firewall
git reset --hard origin/master
git pull origin pull/61/head
sudo ./build-with-docker.sh

The redeploy as before.

cgchinicz commented 5 years ago

Hi Thomas,

But isn't this fix #61 the one I'm currently using? or did you update it yesterday after the answer from Marek? If this is new, what is the difference from the previous fix?

The one I'm using is working just fine with Windows 10 Pro HVM, it's just that it requires that I setup my IP, mask and the gateway's IP instead of using DHCP.

Regards, Claudio

talex5 commented 5 years ago

Yes, I updated it. The new version should work with DHCP too.

cgchinicz commented 5 years ago

Hi Thomas,

I've tested the new fix but it did not work with DHCP on my Windows HVM. See below the mirage logs.

Weird that when now wireshark shows USB only but no network, so I did not saved any file from it.

Regards, Claudio

2019-05-06 07:25:03 -00:00: INF [client_eth] who-has 10.137.0.23? responding with fe:ff:ff:ff:ff:ff 
2019-05-06 07:25:03 -00:00: WRN [command] << Unknown command "QUBESRPC qubes.VMShell dom0"
2019-05-06 07:25:09 -00:00: INF [client_eth] who-has 10.137.0.23? responding with fe:ff:ff:ff:ff:ff
2019-05-06 07:25:13 -00:00: INF [client_eth] who-has 10.137.0.23? responding with fe:ff:ff:ff:ff:ff
2019-05-06 07:25:20 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.13 -> 239.255.255.250: id bb71, off 16384 proto 17, ttl 1, options
 UDP port 34800 -> 1900)
2019-05-06 07:25:21 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.13 -> 239.255.255.250: id bbc4, off 16384 proto 17, ttl 1, options
 UDP port 34800 -> 1900)
2019-05-06 07:25:22 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.13 -> 239.255.255.250: id bf96, off 16384 proto 17, ttl 1, options
 UDP port 34800 -> 1900)
2019-05-06 07:25:23 -00:00: INF [client_eth] who-has 10.137.0.23? responding with fe:ff:ff:ff:ff:ff
@@@                                                                             
-- VISUAL --                                        2         2,90          Top

talex5 commented 5 years ago

Could you try rebooting Windows and record the mirage logs from there? If Windows doesn't have a network device, that sounds like a different problem, because the emulated Realtek device is provided by the stubdom, not directly by the firewall, so I don't think the firewall should be able to make it disappear.

cgchinicz commented 5 years ago

Yes, I've rebooted and Windows does see the Realtek emulated driver but Wireshark doesn't (it is configured to show "all interfaces"), it sees just the external USB.

cgchinicz commented 5 years ago

So, the Windows HVM is working with Mirage through manually configuring IP entries and not through DHCP.

Shall we continue investigating why still does not work with DHCP? If yes, what else can I do to move forward?

talex5 commented 5 years ago

Could you try rebooting Windows and report the mirage logs from there?

cgchinicz commented 5 years ago

Hi, pls see below:

2019-05-06 10:23:12 -00:00: INF [client_net] add client vif {domid=22;device_id=0} with IP 10.137.0.21 2019-05-06 10:23:13 -00:00: INF [client_net] Client 22 (IP: 10.137.0.21) ready 2019-05-06 10:23:13 -00:00: INF [ethernet] Connected Ethernet interface fe:ff:ff:ff:ff:ff 2019-05-06 10:23:13 -00:00: INF [client_net] add client vif {domid=21;device_id=0} with IP 10.137.0.21 2019-05-06 10:23:13 -00:00: INF [qubes.db] got rm "/qubes-iptables-domainrules/" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-iptables-header" = "# Generated by Qubes Core on Mon May 6 13:23:13 2019\nfilter\n:INPUT DROP [0:0]\n:FORWARD DROP [0:0]\n:OUTPUT ACCEPT [0:0]\n-A INPUT -i vif+ -p udp -m udp --dport 68 -j DROP\n-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A INPUT -p icmp -j ACCEPT\n-A INPUT -i lo -j ACCEPT\n-A INPUT -j REJECT --reject-with icmp-host-prohibited\n-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A FORWARD -i vif+ -o vif+ -j DROP\nCOMMIT\n" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/14" = "filter\n-A FORWARD -s 10.137.0.18 -j ACCEPT\n-A FORWARD -s 10.137.0.18 -j DROP\nCOMMIT\n" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/21" = "filter\n-A FORWARD -s 10.137.0.21 -j ACCEPT\n-A FORWARD -s 10.137.0.21 -j DROP\nCOMMIT\n" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/11" = "filter\n-A FORWARD -s 10.137.0.25 -j ACCEPT\n-A FORWARD -s 10.137.0.25 -j DROP\nCOMMIT\n" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/18" = "filter\n-A FORWARD -s 10.137.0.13 -j ACCEPT\n-A FORWARD -s 10.137.0.13 -j DROP\nCOMMIT\n" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/8" = "filter\n-A FORWARD -s 10.137.0.8 -j ACCEPT\n-A FORWARD -s 10.137.0.8 -j DROP\nCOMMIT\n" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-iptables" = "reload" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-ip" = "10.137.0.21" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-gateway" = "10.137.0.23" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got rm "/qubes-firewall/10.137.0.21/" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/0000" = "action=accept" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/policy" = "drop" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21" = "" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got rm "/qubes-firewall/10.137.0.21/" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/0000" = "action=accept" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/policy" = "drop" 2019-05-06 10:23:13 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21" = "" 2019-05-06 10:23:23 -00:00: INF [client_eth] who-has 10.137.0.23? responding with fe:ff:ff:ff:ff:ff 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id e2b3, off 0 proto 17, ttl 1, options UDP port 58447 -> 5355) 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id e2b4, off 0 proto 17, ttl 1, options UDP port 59013 -> 5355) 2019-05-06 10:23:25 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id e2b5, off 0 proto 17, ttl 1, options UDP port 53088 -> 5355) 2019-05-06 10:23:25 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.21 (dropping) 2019-05-06 10:23:25 -00:00: WRN [client_net] Incorrect source IP 10.137.0.1 in IP packet from 10.137.0.21 (dropping) 2019-05-06 10:23:25 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-06 10:23:25 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 1ab5, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-06 10:23:25 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id aba0, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id aba1, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id aba2, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:25 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 10:23:25 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 10:23:25 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 1ab6, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id aba3, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id aba4, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id aba5, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id aba6, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id e2b6, off 0 proto 17, ttl 1, options UDP port 62106 -> 5355) 2019-05-06 10:23:25 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.21 (dropping) 2019-05-06 10:23:25 -00:00: WRN [client_net] Incorrect source IP 10.137.0.1 in IP packet from 10.137.0.21 (dropping) 2019-05-06 10:23:25 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id aba7, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id e2b7, off 0 proto 17, ttl 1, options UDP port 56356 -> 5355) 2019-05-06 10:23:25 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id aba8, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:25 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 10:23:25 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 10:23:25 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-06 10:23:25 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.21 (dropping) 2019-05-06 10:23:25 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-06 10:23:27 -00:00: WRN [client_net] Incorrect source IP 10.137.0.1 in IP packet from 10.137.0.21 (dropping) 2019-05-06 10:23:28 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.21 (dropping) 2019-05-06 10:23:28 -00:00: WRN [client_net] Incorrect source IP 10.137.0.1 in IP packet from 10.137.0.21 (dropping) 2019-05-06 10:23:29 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 23a4, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:29 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id 7159, off 0 proto 17, ttl 1, options UDP port 52076 -> 5355) 2019-05-06 10:23:29 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 23a5, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:29 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 10:23:29 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 23a6, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:29 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 23a7, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:29 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id 715a, off 0 proto 17, ttl 1, options UDP port 64877 -> 5355) 2019-05-06 10:23:29 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-06 10:23:29 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 10:23:29 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 239.255.255.250: id 9764, off 0 proto 17, ttl 4, options UDP port 50079 -> 1900) 2019-05-06 10:23:30 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-06 10:23:30 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 10:23:30 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 10:23:30 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 23a8, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 10:23:30 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 23a9, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353)

talex5 commented 5 years ago

who-has 10.137.0.1? unknown address; not responding

This log message suggests it's an older build of the firewall. The new version is supposed to detect IP addresses ending in .1 and respond. Check that your copy of client_eth.ml contains this at line 71:

  let lookup t ip =
    if ip = t.net.client_gw then Some t.client_link#my_mac
    else if (Ipaddr.V4.to_bytes ip).[3] = '\x01' then (
      Log.info (fun f -> f ~header:t.client_link#log_header
                   "Request for %a is invalid, but pretending it's me (see Qubes issue #5022)" Ipaddr.V4.pp ip);
      Some t.client_link#my_mac
    ) else None

If not, update with:

git fetch origin
git reset --hard origin/master
rm -rf _build
sudo ./build-with-docker.sh

The final build hash (printed at the end of the build) should be dbf7460fa628bea5d132a96fe7ba2cd832e3d9da7005ae74f6a124957f4848ea.

cgchinicz commented 5 years ago

Hi, I've recreated the build and verified that The final build hash (printed at the end of the build) is dbf7460fa628bea5d132a96fe7ba2cd832e3d9da7005ae74f6a124957f4848ea

I've resetted the firewall and it did not work. Then I resarted Qubes and tried both Windows and Linux Mint with DHCP and neither worked. Please see below logs for both.

Logs for Linux Mint

2019-05-06 11:48:00 -00:00: INF [client_net] add client vif {domid=12;device_id=0} with IP 10.137.0.18 2019-05-06 11:48:00 -00:00: INF [client_net] Client 12 (IP: 10.137.0.18) ready 2019-05-06 11:48:00 -00:00: INF [ethernet] Connected Ethernet interface fe:ff:ff:ff:ff:ff 2019-05-06 11:48:01 -00:00: INF [client_net] add client vif {domid=11;device_id=0} with IP 10.137.0.18 2019-05-06 11:48:01 -00:00: INF [qubes.db] got rm "/qubes-iptables-domainrules/" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/qubes-iptables-header" = "# Generated by Qubes Core on Mon May 6 14:48:01 2019\nfilter\n:INPUT DROP [0:0]\n:FORWARD DROP [0:0]\n:OUTPUT ACCEPT [0:0]\n-A INPUT -i vif+ -p udp -m udp --dport 68 -j DROP\n-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A INPUT -p icmp -j ACCEPT\n-A INPUT -i lo -j ACCEPT\n-A INPUT -j REJECT --reject-with icmp-host-prohibited\n-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A FORWARD -i vif+ -o vif+ -j DROP\nCOMMIT\n" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/11" = "filter\n-A FORWARD -s 10.137.0.18 -j ACCEPT\n-A FORWARD -s 10.137.0.18 -j DROP\nCOMMIT\n" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/8" = "filter\n-A FORWARD -s 10.137.0.21 -j ACCEPT\n-A FORWARD -s 10.137.0.21 -j DROP\nCOMMIT\n" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/10" = "filter\n-A FORWARD -s 10.137.0.8 -j ACCEPT\n-A FORWARD -s 10.137.0.8 -j DROP\nCOMMIT\n" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/qubes-iptables" = "reload" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.18/visible-ip" = "10.137.0.18" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.18/visible-gateway" = "10.137.0.23" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got rm "/qubes-firewall/10.137.0.18/" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.18/0000" = "action=accept" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.18/policy" = "drop" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.18" = "" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got rm "/qubes-firewall/10.137.0.18/" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.18/0000" = "action=accept" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.18/policy" = "drop" 2019-05-06 11:48:01 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.18" = "" 2019-05-06 11:48:04 -00:00: INF [net-xen:backend] Frontend asked to close network device dom:12/vif:0 2019-05-06 11:48:04 -00:00: INF [client_net] client {domid=12;device_id=0} has gone 2019-05-06 11:48:05 -00:00: INF [client_net] Client 11 (IP: 10.137.0.18) ready 2019-05-06 11:48:05 -00:00: INF [ethernet] Connected Ethernet interface fe:ff:ff:ff:ff:ff 2019-05-06 11:48:09 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.18 (dropping) 2019-05-06 11:48:13 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.18 (dropping) 2019-05-06 11:48:18 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.18 (dropping) 2019-05-06 11:48:24 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.18 (dropping) 2019-05-06 11:48:39 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.18 (dropping) 2019-05-06 11:48:48 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.18 (dropping)

Logs for Windows

2019-05-06 11:52:45 -00:00: INF [client_net] add client vif {domid=15;device_id=0} with IP 10.137.0.21 2019-05-06 11:52:45 -00:00: INF [client_eth] who-has 10.137.0.23? responding with fe:ff:ff:ff:ff:ff 2019-05-06 11:52:45 -00:00: INF [client_net] Client 15 (IP: 10.137.0.21) ready 2019-05-06 11:52:45 -00:00: INF [ethernet] Connected Ethernet interface fe:ff:ff:ff:ff:ff 2019-05-06 11:52:46 -00:00: INF [client_net] add client vif {domid=14;device_id=0} with IP 10.137.0.21 2019-05-06 11:52:46 -00:00: INF [qubes.db] got rm "/qubes-iptables-domainrules/" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/qubes-iptables-header" = "# Generated by Qubes Core on Mon May 6 14:52:46 2019\nfilter\n:INPUT DROP [0:0]\n:FORWARD DROP [0:0]\n:OUTPUT ACCEPT [0:0]\n-A INPUT -i vif+ -p udp -m udp --dport 68 -j DROP\n-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A INPUT -p icmp -j ACCEPT\n-A INPUT -i lo -j ACCEPT\n-A INPUT -j REJECT --reject-with icmp-host-prohibited\n-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT\n-A FORWARD -i vif+ -o vif+ -j DROP\nCOMMIT\n" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/11" = "filter\n-A FORWARD -s 10.137.0.18 -j ACCEPT\n-A FORWARD -s 10.137.0.18 -j DROP\nCOMMIT\n" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/14" = "filter\n-A FORWARD -s 10.137.0.21 -j ACCEPT\n-A FORWARD -s 10.137.0.21 -j DROP\nCOMMIT\n" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/13" = "filter\n-A FORWARD -s 10.137.0.13 -j ACCEPT\n-A FORWARD -s 10.137.0.13 -j DROP\nCOMMIT\n" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/qubes-iptables-domainrules/10" = "*filter\n-A FORWARD -s 10.137.0.8 -j ACCEPT\n-A FORWARD -s 10.137.0.8 -j DROP\nCOMMIT\n" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/qubes-iptables" = "reload" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-ip" = "10.137.0.21" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/mapped-ip/10.137.0.21/visible-gateway" = "10.137.0.23" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got rm "/qubes-firewall/10.137.0.21/" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/0000" = "action=accept" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/policy" = "drop" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21" = "" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got rm "/qubes-firewall/10.137.0.21/" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/0000" = "action=accept" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21/policy" = "drop" 2019-05-06 11:52:46 -00:00: INF [qubes.db] got update: "/qubes-firewall/10.137.0.21" = "" 2019-05-06 11:53:02 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 98f1, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 11:53:02 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 98f2, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 11:53:02 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 98f3, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 11:53:02 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:02 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id 368d, off 0 proto 17, ttl 1, options UDP port 63574 -> 5355) 2019-05-06 11:53:02 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id 368e, off 0 proto 17, ttl 1, options UDP port 64272 -> 5355) 2019-05-06 11:53:02 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id 368f, off 0 proto 17, ttl 1, options UDP port 61397 -> 5355) 2019-05-06 11:53:02 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id 3690, off 0 proto 17, ttl 1, options UDP port 63095 -> 5355) 2019-05-06 11:53:02 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:02 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.21 (dropping) 2019-05-06 11:53:02 -00:00: WRN [client_net] Incorrect source IP 10.137.0.1 in IP packet from 10.137.0.21 (dropping) 2019-05-06 11:53:03 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-06 11:53:03 -00:00: WRN [client_net] Ignored unknown IPv4 message: Ignoring non-TCP/UDP packet: IPv4 packet 10.137.0.21 -> 224.0.0.22: id 012b, off 0 proto 2, ttl 1, options 94 04 00 00 2019-05-06 11:53:03 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 98f4, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 11:53:03 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id 3691, off 0 proto 17, ttl 1, options UDP port 59783 -> 5355) 2019-05-06 11:53:03 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.21 (dropping) 2019-05-06 11:53:03 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-06 11:53:05 -00:00: WRN [client_net] Incorrect source IP 10.137.0.1 in IP packet from 10.137.0.21 (dropping) 2019-05-06 11:53:06 -00:00: WRN [client_net] Incorrect source IP 0.0.0.0 in IP packet from 10.137.0.21 (dropping) 2019-05-06 11:53:06 -00:00: WRN [client_net] Incorrect source IP 10.137.0.1 in IP packet from 10.137.0.21 (dropping) 2019-05-06 11:53:06 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 2a2e, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 11:53:06 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.252: id c266, off 0 proto 17, ttl 1, options UDP port 49154 -> 5355) 2019-05-06 11:53:06 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 2a2f, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 11:53:06 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:06 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 2a30, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 11:53:06 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 2a31, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 11:53:06 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:06 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:06 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-06 11:53:06 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:07 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 224.0.0.251: id 2a32, off 0 proto 17, ttl 1, options UDP port 5353 -> 5353) 2019-05-06 11:53:07 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:07 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-06 11:53:07 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:07 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:08 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-06 11:53:08 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:08 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 239.255.255.250: id 50ac, off 0 proto 17, ttl 4, options UDP port 49158 -> 1900) 2019-05-06 11:53:09 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:09 -00:00: INF [client_eth] who-has 10.137.0.21? ignoring request for client's own IP 2019-05-06 11:53:09 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:09 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:10 -00:00: INF [client_eth] who-has 10.137.0.1? unknown address; not responding 2019-05-06 11:53:10 -00:00: WRN [firewall] Failed to add NAT rewrite rule: Cannot NAT this packet (IPv4 packet 10.137.0.21 -> 239.255.255.250: id 50ad, off 0 proto 17, ttl 4, options UDP port 49158 -> 1900)

talex5 commented 5 years ago

What do you get if you run this command in dom0?

[...@dom0 ~]$ sha256sum /var/lib/qubes/vm-kernels/mirage-firewall/vmlinuz

If it doesn't start with dbf746 then you're still using an older version. Try copying it over again.

If it does, try changing the log statement in client_eth.ml from:

        Log.info (fun f -> pf f "unknown address; not responding");

to:

        Log.info (fun f -> pf f "unknown address; not responding (%d)" (int_of_char (Ipaddr.V4.to_bytes req_ipv4).[3]));

That will log the last byte of the IP address when printing the unknown address; not responding message. I'd be a bit surprised if the current code isn't working, though.

Mint is using the PV driver, so DHCP isn't expected to work there (I guess it won't work with sys-firewall either). You can tell it's using PV because of the Frontend asked to close network device dom:12/vif:0 message just after it boots up; this is the PV driver closing down the HVM device before connecting itself directly.

cgchinicz commented 5 years ago

Hi,

The hash was different so I removed directory mirage-firewall, recompiled and now Windows 10 HVM is working with DHCP !!!

Thanks again, Regards,

Claudio

talex5 commented 5 years ago

Great - thanks for testing!