Improving Proxy support

[andrunko]

While working on implementing proxy support to respect system settings for the Chromium flatpak, I've noticed a few things that are currently missing/not covered by the portal APIs and that could be improved.

Afaics (please correct me if I'm wrong), the current way to use system proxy settings from a portal is through the ProxyResolver portal interface. The current version of the ProxyResolver interface (v1) only supports one method (GetProxyForURL) and has some limitations:

• No way to check if proxy is enabled in the system, which could be used to avoid invoking the portal to resolve proxies altogether if disabled
• No change notification signal, making it hard to implement a proper cache mechanism client side
• No access to actual settings (enabled/disabled, automatic/manual, pac_url, manual urls, ignore-hosts), forcing the client to use the portal for proxy resolution
• No support to launch the system settings UI and no write access to settings ^

Not having access to the actual settings could arguably be fine, as long as we have a way to reliably implement client caching and to skip the (D-Bus) calls to ProxyResolver.GetProxyForURL altogether when proxy is disabled in the system.

Not being able to launch the system settings UI is also Ok, the client could always rely on the user using the system settings UI directly and simply not support launching it, but that lowers the user experience. For example regular non-flatpak chromium linux builds have a link in its settings to launch the system settings UI, but that just spawns a command directly (e.g. gnome-control-center ... when running on GNOME). There are possible workarounds to make this work, but maybe we should consider an API for it?

With that in mind, I have a few ideas on how to improve here:

1. Add API to access actual settings, with change notification
   • This could be done either as part of the ProxyResolver interface or as a new interface
2. Add a method/property to the ProxyResolver interface to check if proxy is enabled and also a change notification signal trigerred when any of the settings change (including enabled/disabled)

In the chromium case, ideally we would have access to 1) so chromium could use its internal proxy mechanism based on the system settings retrieved using the portal, but 2) would already be a great improvement and allow for a proper cache mechanism and to avoid unnecessary D-Bus calls when the proxy is disabled. We could also provide both worlds, but that may be overkill...

Also worth considering possible privacy concerns if we end up implementing something along the lines of 1) above.

Thoughts?

[wjt]

• No access to actual settings (enabled/disabled, automatic/manual, pac_url, manual urls, ignore-hosts), forcing the client to use the portal for proxy resolution

For those who, like me, are not intimately familiar with how proxies work, I think these would look like this:

• state: enum of {automatic, manual, disabled}
• pac_url: string, meaningful only if 'state' is 'automatic'
• five other properties, meaningful only if 'state' is 'manual':
  • http proxy: (hostname: string, port: uint16)
  • https proxy: (s, u16)
  • ftp proxy: (s, u16)
  • socks proxy: (s, u16)
  • ignore hosts: array of string

I guess the question is whether allowing applications to access this information directly represents an unacceptable information leak. I'm not too worried about the 4 manual hosts/ports because the application can probe for these by requesting the appropriate proxy to connect to a series of URLs with the appropriate protocol.

Perhaps the Proxy Auto-Config URL and the ignored hosts are more problematic, since these allow an application to determine "interesting" URLs by inspecting the list (or the JavaScript code from the PAC URL) to find, e.g., intranet services for which no proxy is needed. (These can't be probed for with the current API without exhaustively searching the entire space of URLs.)

and no write access to settings ^

I don't think write access to the settings is viable. Is this something Chromium expects?

Not being able to launch the system settings UI is also Ok, the client could always rely on the user using the system settings UI directly and simply not support launching it, but that lowers the user experience. For example regular non-flatpak chromium linux builds have a link in its settings to launch the system settings UI, but that just spawns a command directly (e.g. gnome-control-center ... when running on GNOME). There are possible workarounds to make this work, but maybe we should consider an API for it?

Adding this seems reasonable to me (and orthogonal to the other questions). It's no different from the ability of a Flatpak'd app to pop up an email compose window (xdg-email --subject=Test --body=Hi wjt@endlessos.org for example) or screenshot window. If abused (i.e. called without user interaction) it is annoying but not destructive.

With that in mind, I have a few ideas on how to improve here:

1. Add API to access actual settings, with change notification

   • This could be done either as part of the ProxyResolver interface or as a new interface
2. Add a method/property to the ProxyResolver interface to check if proxy is enabled and also a change notification signal triggered when any of the settings change (including enabled/disabled)

This seems uncontroversial to me, if the full information above is not something we want to expose to apps.

For the time being, @andrunko has patched the Chromium Flatpak to use the existing portal and cache the results for a short period of time, 120 seconds IIRC. Since this change landed there has been at least one report that proxy extensions that run within Chromium itself don't work – https://github.com/flathub/org.chromium.Chromium/issues/50 – which might be related to this change to tell Chromium to use the system's proxy resolver rather than its internal logic.

[wjt]

FWIW I had a very brief look at the Android and iOS APIs for proxies.

• Android provides a number of methods (eg ConnectivityManager#getDefaultProxy()) which return a ProxyInfo object which includes the information above. There does not appear to be any restriction on which apps can access this from getDefaultProxy. Getting information about a particular network, as opposed to the default, requires https://developer.android.com/reference/android/Manifest.permission#ACCESS_NETWORK_STATE.
• iOS has a similar API and I can't see any reference to it being privileged

[wjt]

@matthiasclasen @alexlarsson Do you recall any discussion that led to https://github.com/flatpak/xdg-desktop-portal/pull/4 adding the current API, rather than exposing the proxy configuration?

[mcatanzaro]

Do you recall any discussion that led to #4 adding the current API, rather than exposing the proxy configuration?

• The portal cannot expose proxy configuration because it doesn't know how proxies are configured. It just uses the default GProxyResolver. In practice, it will either use libproxy, or will use GNOME proxy settings, since those are the two GProxyResolvers provided by glib-networking. GNOME settings get used if "GNOME" is present in $XDG_CURRENT_DESKTOP. Otherwise, libproxy is used.
• The current API is designed to implement GProxyResolverPortal in gio/gproxyresolverportal.c. That is doing sync D-Bus calls to the portal for every URL lookup, which is awful, but also impossible to fix without changing the GProxyResolver API.

IMO having a property to indicate whether some proxy is configured would be good. The property might be named, say, "may-use-proxy." Then Chromium (and GProxyResolverPortal) could skip proxy resolving in the general case where there is no proxy. (Exposing more than that would be tricky. See below.) Problem is that implementing may-use-proxy would require new API in GProxyResolver, so you would need to first add the API in GLib. Then you would need to implement it twice in glib-networking: once for GProxyResolverGnome, and once for GLibproxyResolver. The flow is: GProxyResolverPortal -> xdg-desktop-portal -> GProxyResolverGnome or GLibproxyResolver.

Libproxy doesn't provide enough information to implement this, so it would have to always set may-use-proxy to TRUE, so the optimization would never work except when running in GNOME. Only when running in GNOME, then we know our own settings and can know for sure whether a proxy might be configured, and we can set it accordingly. If you want to skip checking proxy settings when not running under GNOME, then you have to add new API in libproxy as well. We have a nice debate ongoing regarding whether libproxy should continue to exist here.

Another point to consider: the underlying GNOME settings might change. I can't tell you whether the GNOME proxy settings will even still exist next year: in the issue I linked to above, we talk of moving them to NetworkManager, so they can pick up proxy autoconfig from DHCP. Then we would have settings associated with network interfaces instead of desktop sessions. Lots of uncertainty there. Plus, nobody knows how the existing GNOME settings are supposed to work. If you're unfortunate enough to be working with proxies, perhaps you could help improve things.

[refi64]

Could we make these settings dependent on a portal implementation? From what I can tell, at least within each of GNOME and KDE, having methods to, say, open settings or read proxy configuration is relatively consistent, maybe there could be a new ProxySettings API and apps can fall back to ProxyResolver if ProxySettings is unavailable?

[mcatanzaro]

Could we make these settings dependent on a portal implementation? From what I can tell, at least within each of GNOME and KDE, having methods to, say, open settings or read proxy configuration is relatively consistent, maybe there could be a new ProxySettings API and apps can fall back to ProxyResolver if ProxySettings is unavailable?

It won't solve the underlying problem, which is that you still need API changes in libproxy to make the setting actually work properly, unless you are OK with this working only for GNOME. (The only way KDE settings will ever possibly get read is via libproxy.) And GLib API changes are required no matter what.

If we're going to modify the libproxy API, then let's please add async functions at the same time so that normal proxy lookup doesn't have to block the main thread on HTTP requests for web proxy autoconfig or whatever else libproxy needs to do.

[mcatanzaro]

BTW the technical problems here are pretty simple. Yes, it requires new APIs in multiple components, but there is no major problem that would stop somebody from making this all happen if sufficiently-motivated to spend the time and effort on it. (I have no such motivation, sorry!)

Edit: except exposing access to the underlying proxy settings is not possible, not unless Chromium wants to force use of one particular underlying GProxyResolver, e.g. GProxyResolverGnome, which would be really questionable and weird.

[mcatanzaro]

• No way to check if proxy is enabled in the system, which could be used to avoid invoking the portal to resolve proxies altogether if disabled

I think we all agree we should do this. It requires new API in GProxyResolver, and we'll have to update the GProxyResolvers that we maintained (in glib and glib-networking), plus the portals themselves, of course.

• No change notification signal, making it hard to implement a proper cache mechanism client side

Are you planning to implement a per-URL cache? Keep a hash table of previously-resolved URLs?

Thing is, it's doable, but it requires adding a changed signal to GProxyResolver, and that's going to be useless except for this very specific use-case.

• No access to actual settings (enabled/disabled, automatic/manual, pac_url, manual urls, ignore-hosts), forcing the client to use the portal for proxy resolution

Probably not practical, for the reasons discussed above.

• No support to launch the system settings UI and no write access to settings ^

Well write access cannot happen. Applications should not be able to write system settings.

Launching the settings UI seems theoretically OK, but there's no precedent for how such an API would look.

[GarikTester]

Both Ungoogled and ordinary Chromium, installed with flatpak, can't use proxy (through extensions, because natively this is done through the as*shole historically - Proxy SwitchyOmega or Proxy Switcher (https://chrome.google.com/webstore/detail/proxy-switcher/iejkjpdckomcjdhmkemlfdapjodcpgih). It can be configured without any errors (I use automatic PAC URL), but on the blocked sites I always get ERR_CONNECTION_RESET.

Ungoogled 96.0, Debian Bullseye, flatpak 1.12.2, xdg-desktop-portal 1.10.1, xdg-desktop-portal-gtk 1.10.0. The native Chrome and Chromium (+ Ungoogled) packages work as they should with the same settings.

[mcatanzaro]

Libproxy doesn't provide enough information to implement this, so it would have to always set may-use-proxy to TRUE,

Update from 2023: we control libproxy now, so we should just add new libproxy API for this.

Nothing here is difficult. It just requires a bit of time because there are so many different places that need updated to plumb this all together.