Intercept / modify network traffic

[muodov]

Intercept / modify network traffic

Submitter(s)

Maxim Tsoy, DuckDuckGo

Motivation

Some possible applications were described in this comment by @Token-LiMing. I'm listing them here without details:

• Parental control
• Access control
• Security firewall
• Web caching
• Transcoding data
• Privacy filter

From the DDG perspective, the Privacy angle is the most important. In our apps, we use this data to strip potentially sensitive data from HTTP traffic:

• we need to block tracking requests
• we need to inject headers, such as Sec-GPC for Global Privacy Control
• sometimes we need to remove or rewrite headers, e.g. User-Agent, Referer and Client Hints
• some more complex protections may require modifying/replacing the HTTP responses

A separate issue is the ability to control DNS resolution. We would like to implement DNS-over-HTTPS, which is difficult in the current implementations.

Stakeholders

WebView vendors, app developers

Analysis

By now, we've looked at Android WebView, Apple's WKWebView, and MSFT's WebView2. There are currently various limitations that block . For example:

• Apple WKWebView and Android WebView:
  • do not allow injecting headers in existing requests (instead you need to cancel the request and create a new one with the header). This causes a few usability problems.
  • subrequests (resources) cannot be modified
• In WKWebView it's not currently possible to monitor response headers for all requests. This in particular prevents us from proper handling of business's response to GPC opt-out signals
• MSFT WebView2 provides a handy WebResourceRequested that allows injecting headers

Related W3C deliverables and/or work items

How is the issue solved in the Browser, and what’s more is needed?

Web extensions have the ability to modify/block the requests: webRequest API, declarativeNetRequest API

[QingAn]

As discussed in 2022-08-23 meeting, please help to add summary for Android and iOS @NiklasMerz, and complete for WebView2 @aluhrs13

[NiklasMerz]

I'm not sure if there are other ways of intercepting webview request but on Android I used an API called WebViewAssetLoader and on iOS there is WKURLSchemeHandler.

WKURLSchemeHandler is quite powerful as you can declare a custom scheme like myapp://proxy. Every requests that gets sent to this scheme can be handled in native code and therefore you can make a new HTTP request and set headers etc. as you want. You can even sync cookies between the native request and the WebView.

WebViewAssetLoader is intended for loading local assets as the name suggests. You can use it to make native requests as well but you only have limited access to the request from the WebView. I only managed to make GET requests in some experiments.

Those two APIs offer some capabilities that you can use to intercept requests to certain origins. But they are not really intended for that and it's more of a workaround.

Maybe someone knows other existing APIs?