(Linux) Firewall prevents Flatpak apps from launching

[jordan-ivpn]

Bug report

Describe your environment

• Device: _____
• OS name and version: __Fedora Workstation 37___
• IVPN app version: __3.10.14/3.10.15___

Describe the problem

When the IVPN App's firewall is enabled, Internet-based Flatpak apps (Firefox, Thunderbird, librewolf) never launch. Disabling the app's firewall for one to three seconds allows whatever the Flatpak apps require to satisfy their startup requirements and launch.

https://flathub.org/apps/details/org.mozilla.Thunderbird https://flathub.org/apps/details/io.gitlab.librewolf-community

Steps to reproduce:

1. Setup repo and install:

   flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
   flatpak remote-modify --enable flathub
   flatpak install flathub org.mozilla.Thunderbird
   flatpak install flathub io.gitlab.librewolf-community

2. Run the app:

   
   flatpak run org.mozilla.Thunderbird
   (gtk messages can be ignored)

flatpak run io.gitlab.librewolf-community

3. Wait for a minute or more and nothing happens.  After launching a Flatpak app, toggle the IVPN App's firewall off for one to three seconds, then toggle it back on and the Flatpak app launches.  Close the Flatpak app and launch it again _with the IVPN App's firewall enabled_ and it launches successfully.  Reboot the system and launch the app and it fails.

### Observed Results:

There is a burst of network traffic when the IVPN App's firewall is disabled.  It is IPv4 and IPv6.  Even when IPv6 is disabled on the system, there is still IPv6 traffic in the burst.  This burst is not present when toggling the IVPN App's firewall off and not launching the Flatpak app.

To capture traffic on the local system, find the network interface (usually `1`):

sudo tcpdump -D

Capture packets where `a.b.c.d` is the incoming address for the VPN server (`dig +short XX.wg.ivpn.net`):

sudo tcpdump -n -i 1 not host a.b.c.d 2>&1 | tee tcpdump.log

I have checked and unchecked the box in the IVPN App for WireGuard + IPv6, but the issue persists.  I have tried disabling IPv6 on the system, but there is no change.

It might be related to some internal network-based access requirement, like accessing some internal process or socket or other via an IP address.  Flatpak app permissions include `network` and `ipc` as shared/accessible and access to a `Session Bus` exists as well:

$ flatpak info --show-permissions org.mozilla.Thunderbird [Context] shared=network;ipc; sockets=x11;wayland;pulseaudio;pcsc;cups; devices=dri; filesystems=xdg-download;xdg-run/gnupg:ro;/run/.heim_org.h5l.kcm-socket;~/.gnupg; persistent=.thunderbird;

[Session Bus Policy] org.freedesktop.Notifications=talk org.a11y.Bus=talk org.mozilla.thunderbird_default.*=own

### Expected Results:

The Flatpak app launches when the IVPN App's firewall is enabled.

--

Thanks.

[ruckerzerg]

This might be related to systemd-resolved as I have the exact same issue with Firefox (native install, no Flatpak) but only when I have enabled systemd-resolved service and active IVPN firewall. Firefox won't start or open new windows. The already opened window works fine though if you activate the firewall afterwards...

[stenya]

@jordan-ivpn I was not able to reproduce the issue (VM#139 Fedora37). You didn't mention if the VPN should be connected or not for reproducing the issue. Anyway, I tried both variants. Did I miss something? Would you be able to try to reproduce it on VM#139?

[jordan-ivpn]

@stenya I cannot reproduce it on (VM#141 Jordan-Fedora37, pw=username).

You didn't mention if the VPN should be connected or not for reproducing the issue.

VPN connected (OpenVPN, WireGuard IPv6 or not), firewall enabled. AntiTracker enabled.

While I was setting up the VM#141, I launched the Thunderbird flatpak on my local VirtualBox VM and left it alone for 10 minutes. Upon returning to the local VM, Thunderbird had launched. I rebooted the local VM and launched librewolf. It took 5 minutes, but it eventually launched. I rebooted the local VM and repeated the process (launch flatpak + wait 5 minutes) and Thunderbird launched. The thunderbird process is running when the flatpak is launched, though it took the GUI 5 minutes to appear. System resources are similar on both VMs, so it is unclear why there is a 5-minute timeout on my local VM.

This issue can probably be closed, though it is unclear what communication is occurring or not occurring between the flatpak instance and whatever it needs to connect to out on the Internet.

[stenya]

This might be related to systemd-resolved as I have the exact same issue with Firefox (native install, no Flatpak) but only when I have enabled systemd-resolved service and active IVPN firewall. Firefox won't start or open new windows. The already opened window works fine though if you activate the firewall afterwards...

Did it happen on Linux running under any VM (e.g. VirtualBox)?

[ruckerzerg]

This might be related to systemd-resolved as I have the exact same issue with Firefox (native install, no Flatpak) but only when I have enabled systemd-resolved service and active IVPN firewall. Firefox won't start or open new windows. The already opened window works fine though if you activate the firewall afterwards...

Did it happen on Linux running under any VM (e.g. VirtualBox)?

No this was on a native fresh install of Arch Linux. Also starting a VM vith libvirt does not work with active IVPN firewall in combination with using systemd-resolved.

Edit: Could also reproduce it on another machine with native Arch by enabling systemd-resolved.

@jordan-ivpn: Same experience for me with Firefox and Thunderbird (native, not Flatpak) - after about 5 minutes it starts, I guess the request that needs to be done on startup then runs into a timeout.

[ghost]

Hello. I can also confirm that I'm affected by this issue with Fedora 38 Workstation on a real system. It's at some point annoying that it takes 5 minutes to open a program really through Flatpak. But I must also say that not all Flatpaks apps are affected by this problem (one example Geary - a mail client). This are probably apps which don't need a connection at the launch process / don't need a connection after launching it. A workaround would be nice until we know better what produces this problem.

[EKolyshkin]

Still able to reproduce the issue with Flatpak 1.15.4, LibreWolf 114.0.2-1 on Fedora Linux 38.20230630.0 (Silverblue), Turning off Firewall option in IVPN (3.10.23) instantly resolves it. Haven't had any similar issue with other Flatpak apps otherwise. Issue doesn't happen with integrated Firefox browser. Could it be LibreWolf-specific?

[sith-on-mars]

Was able to reproduce the same issue on Silverblue 38. Firefox Flatpak took a few minutes to launch. Brave wasn't affected by the issue.

[EKolyshkin]

Finally found the culprit: in Firefox and its forks, about:config has a network.notify.IPv6 option. Set it to false, and the flatpak app opens instantly with IVPN firewall on. That's probably what was causing the burst of IPv6 traffic, even if IPv6 is turned off in the app.

[EKolyshkin]

Nevermind. Still hangs after adjusting all these settings. I think the app was able to reopen with firewall on due to some background process that hangs for a while after quitting the app GUI.

[jordan-ivpn]

Fedora 40, Firefox (not a Flatpak), same experience with a 4+ minute delay, though might be a different issue.

$ strace -t firefox 2>strace.txt
18:07:19 execve("/usr/bin/firefox", ["firefox"], 0x7ffdd11a31c8 /* 45 vars */) = 0

This is the first halt point:

18:07:19 openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 20
18:07:19 fstat(20, {st_mode=S_IFREG|0644, st_size=74107, ...}) = 0
18:07:19 mmap(NULL, 74107, PROT_READ, MAP_PRIVATE, 20, 0) = 0x7f6f7ed3e000
18:07:19 close(20)                      = 0
18:07:19 openat(AT_FDCWD, "/lib64/libnss_resolve.so.2", O_RDONLY|O_CLOEXEC) = 20
18:07:19 read(20, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
18:07:19 fstat(20, {st_mode=S_IFREG|0755, st_size=190952, ...}) = 0
18:07:19 mmap(NULL, 186696, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 20, 0) = 0x7f6f7e804000
18:07:19 mmap(0x7f6f7e807000, 131072, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 20, 0x3000) = 0x7f6f7e807000
18:07:19 mmap(0x7f6f7e827000, 36864, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 20, 0x23000) = 0x7f6f7e827000
18:07:19 mmap(0x7f6f7e830000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 20, 0x2c000) = 0x7f6f7e830000
18:07:19 close(20)                      = 0
18:07:19 mprotect(0x7f6f7e830000, 4096, PROT_READ) = 0
18:07:19 munmap(0x7f6f7ed3e000, 74107)  = 0
18:07:19 rt_sigprocmask(SIG_BLOCK, [HUP USR1 USR2 PIPE ALRM CHLD TSTP URG VTALRM PROF WINCH IO], [], 8) = 0
18:07:19 futex(0x7f6f7e831900, FUTEX_WAKE_PRIVATE, 2147483647) = 0
18:07:19 socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 20
18:07:19 connect(20, {sa_family=AF_UNIX, sun_path="/run/systemd/resolve/io.systemd.Resolve"}, 42) = 0
18:07:19 sendto(20, "{\"method\":\"io.systemd.Resolve.Re"..., 100, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 100
18:07:19 recvfrom(20, 0x7f6f70b5b000, 131072, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
18:07:19 ppoll([{fd=20, events=POLLIN}], 1, {tv_sec=119, tv_nsec=999928000}, NULL

This is the second halt point:

18:09:20 newfstatat(AT_FDCWD, "/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=953, ...}, 0) = 0
18:09:20 newfstatat(AT_FDCWD, "/etc/nsswitch.conf", {st_mode=S_IFREG|0644, st_size=671, ...}, 0) = 0
18:09:20 openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 22
18:09:20 fstat(22, {st_mode=S_IFREG|0644, st_size=384, ...}) = 0
18:09:20 lseek(22, 0, SEEK_SET)         = 0
18:09:20 read(22, "# Loopback entries; do not chang"..., 4096) = 384
18:09:20 read(22, "", 4096)             = 0
18:09:20 close(22)                      = 0
18:09:20 rt_sigprocmask(SIG_BLOCK, [HUP USR1 USR2 PIPE ALRM CHLD TSTP URG VTALRM PROF WINCH IO], [], 8) = 0
18:09:20 socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 22
18:09:20 connect(22, {sa_family=AF_UNIX, sun_path="/run/systemd/resolve/io.systemd.Resolve"}, 42) = 0
18:09:20 sendto(22, "{\"method\":\"io.systemd.Resolve.Re"..., 100, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 100
18:09:20 recvfrom(22, 0x7f6f70b5b000, 131072, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
18:09:20 ppoll([{fd=22, events=POLLIN}], 1, {tv_sec=119, tv_nsec=999897000}, NULL, 8j

Firefox launches at around 18:11:29.

Unlike the original issue, closing Firefox, then re-opening it results in another 4+ minute delay.

[stenya]

Observation: The DNS configuration appears to be related to the issue. When the DNS search domains are reset for the default interface, the issue disappears.

Example: To check the DNS configuration, run: resolvectl status

Global
         Protocols: LLMNR=resolve -mDNS -DNSOverTLS DNSSEC=no/unsupported
  resolv.conf mode: stub
Current DNS Server: 10.0.254.4
       DNS Servers: 10.0.254.4

Link 2 (enp0s3)
    Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
         Protocols: +DefaultRoute LLMNR=resolve -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.2.1
       DNS Servers: 192.168.2.1
        DNS Domain: google.com

To reset the DNS search domain on the main interface (changing from "google.com" to an empty value): sudo resolvectl domain enp0s3 ""

After this change, the issue is no longer reproducible on my test machine (until the next reboot).

---

Additionally, the issue is immediately resolved when all DNS communication is allowed on the test system. Currently, the IVPN firewall only permits DNS packets to the IVPN DNS server or a custom server defined in the settings, which may be at the core of the issue.

---

This is still under investigation.

[Condobloke]

Linux Mint 22 Cinnamon

'Action' on desktop: send as attachment

Thunderbird (flathub/flatpak...recent download !~2 days) Thunderbird (flatpak) 128.3.2esr

send as attachment opens email, ....but with no attachment

---

System package Thunderbird : 1:128.3.1esr+linuxmint1+wilma, ...works as expected