Flatpak apps can't launch the default browser

cizra commented 2 years ago

Describe the bug

Flatpak apps (like Thunderbird) can't launch the default browser

Steps To Reproduce

Steps to reproduce the behavior:

flatpak --user install org.mozilla.Thunderbird
Verify you have a default browser: xdg-settings get default-web-browser - I'm getting firefox.desktop as output
Open Thunderbird. Click on any link in a mail or somewhere in UI (there's the "Donate" link on one of the tabs shown by Thunderbird on a clean install)

Expected behavior

Default browser opens.

Screenshots

Additional context

  programs.sway.enable = true;
  services.flatpak.enable = true;
  xdg.portal = {
    wlr.enable = true;
    extraPortals = [ pkgs.xdg-desktop-portal-gtk ];
    enable = true;
  };

To make sure my user settings aren't messed up, I created a new test user in my system. It too exhibited the buggy behavior.

Notify maintainers

@jtojnar

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.126, NixOS, 21.11 (Porcupine)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.16`
 - channels(root): `"nixos-21.11.337975.eabc3821918, unstable-22.11pre405560.2da64a81275"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

jtojnar commented 2 years ago

That should be handles by the xdg-desktop-portal’s OpenURI portal.

Do you see anything relevant in the systemd journal when you click the link?

xdg-settings is not relevant here, you should check gio mime x-scheme-handler/https.

And you will need to check that the portal systemd service has access to installed desktop files in XDG_DATA_DIRS. Running systemd-run --user --pipe gio mime x-scheme-handler/https will probably be the easiest.

cizra commented 2 years ago

Reading the source you provided, I see that OpenURI portal calls xdp_dbus_impl_app_chooser_call_choose_application. Where's that implemented? It seems this is the failing point.

Here's an exceprt from dbus-monitor:

method call time=1662371825.457775 sender=:1.4 -> destination=:1.7 serial=151 path=/org/freedesktop/portal/desktop; interface=org.freedesktop.impl.portal.AppChooser; member=ChooseApplication
   object path "/org/freedesktop/portal/desktop/request/1_31/t"
   string "org.mozilla.Thunderbird"
   string ""
   array [
   ]
   array [
      dict entry(
         string "content_type"
         variant             string "x-scheme-handler/https"
      )
      dict entry(
         string "uri"
         variant             string "https://jbdpbthere's a long URL here"
      )
   ]
method return time=1662371826.791107 sender=:1.7 -> destination=:1.4 serial=71 reply_serial=151
   uint32 1
   array [
      dict entry(
         string "choice"
         variant             string ""
      )
   ]

Indeed, logs show that it's the xdg-desktop-portal-gtk failing (looked it up by PID).

Sep 06 10:46:59 hostname .xdg-desktop-po[1978]: Unhandled parent window type
Sep 06 10:46:59 hostname .xdg-desktop-po[1978]: Failed to associate portal window with parent window

Is it being confused because I'm on Wayland?

Regardless, we've narrowed it down the likely suspect to be xdg-desktop-portal-gtk, let's see how the -kde version would fare. I uninstalled -gtk, installed -kde (though I'm still using Sway), rebooted, and now I'm getting an app picker. After choosing any browser in it, I'm getting no browsers to open. Logs show this:

Sep 06 13:49:55 hostname org.freedesktop.impl.portal.desktop.kde[2031]: xdp-kde-app-chooser: ChooseApplication called with parameters:
Sep 06 13:49:55 hostname org.freedesktop.impl.portal.desktop.kde[2031]: xdp-kde-app-chooser:     handle:  "/org/freedesktop/portal/desktop/request/1_25/t"
Sep 06 13:49:55 hostname org.freedesktop.impl.portal.desktop.kde[2031]: xdp-kde-app-chooser:     app_id:  "org.mozilla.Thunderbird"
Sep 06 13:49:55 hostname org.freedesktop.impl.portal.desktop.kde[2031]: xdp-kde-app-chooser:     parent_window:  ""
Sep 06 13:49:55 hostname org.freedesktop.impl.portal.desktop.kde[2031]: xdp-kde-app-chooser:     choices:  ()
Sep 06 13:49:55 hostname org.freedesktop.impl.portal.desktop.kde[2031]: xdp-kde-app-chooser:     options:  QMap(("content_type", QVariant(QString, "x-scheme-handler/https"))("uri", QVariant(QString, "https://vectorbelly.com/electrical312.html")))
Sep 06 13:49:55 hostname org.freedesktop.impl.portal.desktop.kde[2031]: kf.kirigami: Failed to find a Kirigami platform plugin

cizra commented 2 years ago

^ This is what xdg-desktop-portal-pantheon shows.

How do I get it registered? The .desktop file sure thinks it's supported:

$ grep x-scheme-handler/https .local/share/flatpak/exports/share/applications/org.mozilla.firefox.desktop
MimeType=text/html;text/xml;application/xhtml+xml;application/vnd.mozilla.xul+xml;text/mml;x-scheme-handler/http;x-scheme-handler/https;

Also:

$ gio mime x-scheme-handler/https
Default application for “x-scheme-handler/https”: org.mozilla.firefox.desktop
No registered applications
No recommended applications

$ systemd-run --user --pipe gio mime x-scheme-handler/https
Running as unit: run-u33.service
No default applications for “x-scheme-handler/https”

It seems something's missing... Though what? the XDG_DATA_DIRS seems to be set, according to /proc/pidof ... the portal/environ. Also it can access the file itself:

$ systemd-run --user --pipe cat /home/user/.local/share/flatpak/exports/share/applications/org.mozilla.firefox.desktop

jtojnar commented 2 years ago

Reading the source you provided, I see that OpenURI portal calls xdp_dbus_impl_app_chooser_call_choose_application. Where's that implemented? It seems this is the failing point.

That would be portal implementations like x-d-p-gtk, all of which should support Wayland (sometimes X is more of an afterthought):

https://github.com/flatpak/xdg-desktop-portal-gtk/blob/347fcd1f61e126ae77950b5d4b66ad9522aec1f2/src/externalwindow.c#L78

For https URLs, the app chooser portal should be only triggered when no default app is found, IIRC so you should start looking there.

$ systemd-run --user --pipe gio mime x-scheme-handler/https
Running as unit: run-u33.service
No default applications for “x-scheme-handler/https”

This is critical, without it no portal will work. Maybe the issue is that Flatpak exports are not in the systemd service’s XDG_DATA_DIRS environment variable. Try checking systemd-run --user --pipe env.

cizra commented 2 years ago

$ systemd-run --user --pipe env | grep XDG_
Running as unit: run-u94.service
XDG_CONFIG_DIRS=/home/user/.local/share/flatpak/exports/etc/xdg:/var/lib/flatpak/exports/etc/xdg:/home/user/.nix-profile/etc/xdg:/etc/profiles/per-user/user/etc/xdg:/nix/var/nix/profiles/default/etc/xdg:/run/current-system/sw/etc/xdg
XDG_DATA_DIRS=/nix/store/3bcmfxgns1a143m9lmsmn7qv29wkfilq-desktops/share:/home/user/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/home/user/.nix-profile/share:/etc/profiles/per-user/user/share:/nix/var/nix/profiles/default/share:/run/current-system/sw/share
XDG_DESKTOP_PORTAL_DIR=/nix/store/1lvg8hqlbi5ii45klplpxjlbqadsfh5r-xdg-portals/share/xdg-desktop-portal/portals
XDG_RUNTIME_DIR=/run/user/1000
XDG_CURRENT_DESKTOP=sway

$ systemd-run --user --pipe grep Mime /home/user/.local/share/flatpak/exports/share/applications/org.mozilla.firefox.desktop
Running as unit: run-u97.service
MimeType=text/html;text/xml;application/xhtml+xml;application/vnd.mozilla.xul+xml;text/mml;x-scheme-handler/http;x-scheme-handler/https;

Seems to be present... What else could I check?

jtojnar commented 2 years ago

Can you share the whole contents of ~/.local/share/flatpak/exports/share/applications/org.mozilla.firefox.desktop? Maybe it is broken (https://github.com/NixOS/nixpkgs/issues/156664 comes to mind).

Does gio open https://jbdpbthere run from your session work?

It is weird that gio mime returns no registered applications. I would expect that to only happen when there are no valid desktop files. But then why the difference? The enly explanation that comes to mind is that the desktop file’s Exec key could depend on PATH, which is different.

cizra commented 2 years ago

gio open https://jbdpbthere - this works fine, when launched outside of systemd-run:

$ nix-shell -p glib
[nix-shell:~]$ gio open https://jbdpbthere
[nix-shell:~]$
(firefox:2): Gdk-WARNING **: 14:05:03.864: Failed to read portal settings: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.portal.Settings” on object at path /org/freedesktop/portal/desktop
// something's happening async here - the prompt is printed before the warning. The browser opens, though.
[nix-shell:~]$ systemd-run --user --pipe gio open https://jbdpbthere
Running as unit: run-u184.service
gio: https://jbdpbthere: Operation not supported
// Browser does not open.

Since I can't figure out how to upload the file into GitHub (drag/drop doesn't work), I'll just paste it here (I edited it to remove localized strings):

$ cat org.mozilla.firefox.desktop
[Desktop Entry]
Version=1.0
Name=Firefox
GenericName=Web Browser
Comment=Browse the Web
Exec=flatpak run --branch=stable --arch=x86_64 --command=firefox --file-forwarding org.mozilla.firefox @@u %u @@
Icon=org.mozilla.firefox
Terminal=false
Type=Application
MimeType=text/html;text/xml;application/xhtml+xml;application/vnd.mozilla.xul+xml;text/mml;x-scheme-handler/http;x-scheme-handler/https;
StartupNotify=true
Categories=Network;WebBrowser;
Keywords=web;browser;internet;
Actions=new-window;new-private-window;profile-manager-window;
StartupWMClass=firefox
X-Flatpak=org.mozilla.firefox

[Desktop Action new-window]
Name=Open a New Window
Exec=flatpak run --branch=stable --arch=x86_64 --command=firefox --file-forwarding org.mozilla.firefox --new-window @@u %u @@

[Desktop Action new-private-window]
Name=Open a New Private Window
Exec=flatpak run --branch=stable --arch=x86_64 --command=firefox --file-forwarding org.mozilla.firefox --private-window @@u %u @@

[Desktop Action profile-manager-window]
Name=Open the Profile Manager
Exec=flatpak run --branch=stable --arch=x86_64 --command=firefox org.mozilla.firefox --ProfileManager

jtojnar commented 2 years ago

Can you also check if flatpak is on systemd-run --user --pipe env | grep PATH?

cizra commented 2 years ago

Ahh, that's it!

$ systemd-run --user --pipe env | grep ^PATH
Running as unit: run-u100.service
PATH=/nix/store/fij51i0dx84p8g94fyc3r4dhdjbg8rp4-systemd-249.7/bin/
21:02:05 sutra ~ (master) $ ls /nix/store/fij51i0dx84p8g94fyc3r4dhdjbg8rp4-systemd-249.7/bin/
bootctl      halt         journalctl  machinectl  reboot      shutdown         systemd-ask-password  systemd-cgtop        systemd-detect-virt  systemd-hwdb     systemd-machine-id-setup  systemd-nspawn  systemd-resolve          systemd-stdio-bridge  systemd-tty-ask-password-agent  udevadm
busctl       hostnamectl  localectl   networkctl  resolvconf  systemctl        systemd-cat           systemd-cryptenroll  systemd-dissect      systemd-id128    systemd-mount             systemd-path    systemd-run              systemd-sysext        systemd-umount                  userdbctl
coredumpctl  init         loginctl    poweroff    resolvectl  systemd-analyze  systemd-cgls          systemd-delta        systemd-escape       systemd-inhibit  systemd-notify            systemd-repart  systemd-socket-activate  systemd-tmpfiles      timedatectl

Hence two questions:

How do I fix it?
Why am I apparently the only one affected by this?

jtojnar commented 2 years ago

How environment variables are set is complicated, might depend your shell and activation environment. Here’s mine, for the record (using fish on GNOME Shell):

$ systemd-run --user --pipe env | grep ^PATH
Running as unit: run-u3853.service
PATH=/run/wrappers/bin:/home/jtojnar/.local/share/flatpak/exports/bin:/var/lib/flatpak/exports/bin:/home/jtojnar/.nix-profile/bin:/etc/profiles/per-user/jtojnar/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin

cizra commented 2 years ago

Running

systemctl --user import-environment PATH

fixes the PATH for any new processes. Restarting the affected services makes it work.

As per https://discourse.nixos.org/t/systemd-user-units-and-no-such-path/8399/2

{
  systemd.user.extraConfig = ''
    DefaultEnvironment="PATH=/run/current-system/sw/bin"
  '';
}

... also fixes the issue. How can we fix it for other users? Perhaps document in manual, if nothing else?

jtojnar commented 2 years ago

Right, we should have “Building your own desktop environment from scratch” section which lists all the assumptions you need for a working DE.

First, we should probably figure out what component normally does the environment import and replicate is as a module.

cizra commented 2 years ago

:grin: I'm afraid it'll go beyond the scope of this bug. Should we close this one, as a workaround is found?

Thank you so much for helping investigate this! I learned a bunch about how things work.

jtojnar commented 2 years ago

I would keep it open at least until we do:

we should probably figure out what component normally does the environment import and replicate is as a module.

nixos-discourse commented 1 year ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/open-links-from-flatpak-via-host-firefox/15465/5

nixos-discourse commented 11 months ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/clicked-links-in-desktop-apps-not-opening-browers/29114/26

liff commented 11 months ago

FWIW, since systemd 254 it’s possible to use %u replacement in DefaultEnvironment, so you can use /etc/profiles/per-user/%u/bin in the `PATH:

{
  systemd.user.extraConfig = ''
    DefaultEnvironment="PATH=/run/wrappers/bin:/etc/profiles/per-user/%u/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin"
  '';
}

hydragyrum32 commented 10 months ago

I'm experiencing the same symptoms on sway, nixos 23.05 and unstable, the environment fix to PATH causes systemd-run --user --pipe gio open https://github.com to fail silently (appears to succeed, but no browser tab or window is opened, gio open without being wrapped in systemd-run works fine

cizra commented 10 months ago

@hydragyrum32 Did you already try


cat > ~/.config/xdg-desktop-portal/portals.conf <<EOF
[preferred]
# use xdg-desktop-portal-gtk for every portal interface
default=gtk
# except for the xdg-desktop-portal-wlr supplied interfaces
org.freedesktop.impl.portal.Screencast=wlr
org.freedesktop.impl.portal.Screenshot=wlr
EOF

hydragyrum32 commented 10 months ago

@cizra Thanks, doing that and a quick systemctl restart xdg-desktop-portal.service fixed it for me

jtojnar commented 10 months ago

That configuration requires newer version of x-d-p that is only part of #247766 (not yet merged) so it cannot have an effect.

nonetrix commented 9 months ago

Having this issue as well, I also noticed in in Warframe which is not using Flatpak The suggested fix worked though

LunNova commented 7 months ago

For i3 users: make sure services.xserver.windowManager.i3.updateSessionEnvironment is on so your portals get env vars.

If that still doesn't work your portal might be starting before the environment gets updated due to a timing issue. I have a bodge for it here you can apply https://github.com/LunNova/nixos-configs/blob/abb5e7e8f124fd2680e05b3bd74f73983b4dcd2d/nixpkgs-patches/graphical-session-delay.patch but it feels too jank to make into a PR.

uninsane commented 7 months ago

for anyone managing their own mime associations, make sure share/applications/mimeinfo.cache exists in whichever of the XDG_DATA_DIRS paths that xdg-desktop-portal would find its mime associations. it's easy to get by without this file, and only mimeapps.list, until you have a program which specifically wants to query all handlers for a filetype (which the portal does) instead of just the default (which the gio open example does).

update-desktop-database ~/.local/share/applications (from desktop-file-utils package) can generate mimeinfo.cache when pointed to a directory of .desktop files.

nonetrix commented 5 months ago

Having another likely unrelated issue, but for some reason xdg-open asks me what I want to open something in sometimes despite the fact it's defined in mimeapps.list :/

uninsane commented 5 months ago

Having another likely unrelated issue, but for some reason xdg-open asks me what I want to open something in sometimes despite the fact it's defined in mimeapps.list :/

you'll have to narrow down that "sometimes" more precisely to make progress on this one. even with mimeapps.list, xdg-desktop-portal won't respect those associations as actual defaults until you exercise them 3 times. it's only special schemes like http:// which can actually be defaulted in the way we think of that in nix.

i believe there's the possibility of programmatically populating the "xdg permission store" to bypass that "defaults only become defaults after 3 launches" behavior, but i haven't seen anyone do that in practice yet.

nonetrix commented 5 months ago

i believe there's the possibility of programmatically populating the "xdg permission store" to bypass that "defaults only become defaults after 3 launches" behavior, but i haven't seen anyone do that in practice yet.

I think this is my issue, so I guess they are changing how it works. Annoying, I just want it to just work, why is this so hard? Reinventing the wheel for no reason seemingly

symphorien commented 4 months ago

For me (xfce+i3) services.xserver.windowManager.i3.updateSessionEnvironment was not enough and I needed to add delay this way:

    systemd.user.services."wait-for-full-path" = {
      description = "wait for systemd units to have full PATH";
      wantedBy = [ "xdg-desktop-portal.service" ];
      before = [ "xdg-desktop-portal.service" ];
      path = with pkgs; [ systemd coreutils gnugrep ];
      script = ''
        ispresent () {
          systemctl --user show-environment | grep -E '^PATH=.*/.nix-profile/bin'
        }
        while ! ispresent; do
          sleep 0.1;
        done
      '';
      serviceConfig = {
        Type = "oneshot";
        TimeoutStartSec = "60";
      };
    };

This waits for ~/.nix-profile/bin to appear in PATH

NixOS / nixpkgs