Support appending to User-Agent header in declarativeNetRequest

[oliverdunk]

In Chrome's implementation of declarativeNetRequest, we have an explicit list of headers that we allow the "append" HeaderOperation to be applied to.

We're considering adding "User-Agent" to this list following a developer request. We discussed that in the last meeting and there was general support, but some discussion around what separator character to use.

Looking at the examples on MDN, it seems like a space is the most common character here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent

This also matches the spec, which says it should follow the form User-Agent = product *( RWS ( product / comment ) ). Based on my understanding this is basically saying you can have as many products as you like and they are separated by whitespace.

Based on this, I propose allowing this header and making whitespace the character automatically added internally between each appended value.

[oliverdunk]

@Rob--W, @xeenon, can you share your thoughts here?

[Rob--W]

Is the use case behind the request publicly visible? I'd be interested in learning more about the use case.

The requested behavior sounds reasonable, including the request to concatenate additions with a space.

During the meeting (meeting notes pending to be merged at https://github.com/w3c/webextensions/pull/397), I mentioned that additions to the User-Agent are separated by semicolons. This was referring to historical behavior in Internet Explorer. Plugins (external software in general) could insert parts in the User-Agent header by setting the Post Platform or Pre Platform keys in the Windows Registry. This was eventually dropped in IE9 because "this led to performance issues for customers of certain tools and add-ons". More details on that in https://msdn.microsoft.com/en-us/library/Hh869301#user-agent-changes-in-earlier-versions.

P.S. This ticket here is about the "append" operation. It is already possibly to completely overwrite the User-Agent header with the "set" operation. If Chrome ends up supporting "append" for "user-agent", then we should add that to the list of headers at https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/declarativeNetRequest/ModifyHeaderInfo#header_limits

[oliverdunk]

The requested behavior sounds reasonable, including the request to concatenate additions with a space.

Perfect, thanks for taking a look and updating the label :)

Is the use case behind the request publicly visible? I'd be interested in learning more about the use case.

Sure, you can see the CL here. That said, I'm not advocating for the merits of that specific use case. This simply prompted us to re-consider if this should have been on the list and we can't see a reason it shouldn't be.

[mnoorenberghe]

Is the use case behind the request publicly visible? I'd be interested in learning more about the use case.

I had a similar use case but had to put in a workaround a few years ago due to lack of support for setting User-Agent at all in Chrome at that time: load balancer and web server logs usually log the User-Agent header by default which can be useful for debugging the cause of errors with our API (e.g. a specific extension version is the cause of all the HTTP 503 errors for a specific path). The extension I'm referring to would only append the extension version to the header on requests to our own API server, not to other origins.

[hanguokai]

I know it's not contradictory, but Chrome has been pushing for User-Agent reduction. https://developer.chrome.com/blog/user-agent-reduction-android-model-and-version/ https://developer.chrome.com/docs/privacy-sandbox/user-agent/

[bershanskiy]

Chromium 116 will likely support this feature: https://github.com/chromium/chromium/commit/b58ff2ec674efe7dbe663124b6beaa3feeb5ea49

[oliverdunk]

Thanks for the callout @bershanskiy, as you say this will be in M116. Adding the implemented label.