formatting of HTTP user agent string

[mavgit]

From @jpiesing on March 7, 2017 16:8

Web video providers have found it necessary to make requirements on the formatting of the HTTP user agent string to enable the device to be identified. This may be for white-listing, black-listing or identifying certain characteristics of the device.

Should WAVE say something about this?

Copied from original issue: w3c/webmediaapi#52

[mavgit]

From @jpiesing on March 7, 2017 16:16

For example, HbbTV chose to specify the following & this is referenced by at least one web video provider.

The User-Agent header shall include: HbbTV/1.4.1 (<capabilities> ; <vendorName> ; <modelName> ; <softwareVersion> ; [<hardwareVersion> ]; <familyName> ; <reserved>) Where:

• The <capabilities> field consists of zero or more concatenated HbbTV® option strings as defined in clause 10.2.4. ...
• The <vendorName> field shall reflect the consumer-facing make / brand of the terminal.
• The <vendorName>, <softwareVersion>, <familyName> combination, along with <hardwareVersion> where included, shall differ between terminals with significantly different implementations or performance.
• The <modelName> field should be representative of the consumer-facing model name to allow log messages to be matched up with user reported problems.
• The <familyName> field shall have the semantics defined in clause 7.3.3.2 of the OIPF DAE specification [1] but shall additionally be chosen so as to be globally unique. This shall be achieved either by prefixing with a reverse domain name of the organisation allocating the remaining portion of the familyName, or by using a version 4 UUID as the familyName, formatted as a simple string (i.e. without any urn:uuid prefix) [64].

[mavgit]

From @jeremypoulter on March 7, 2017 16:23

I would be cautious about encoding so much info in to the User-Agent header. Overloading in this way is hard to parse makes the string really long and can be user editable and hence unreliable. It may be better to add specific HTTP headers to carry more than just the spec identifier/version number.

[mavgit]

From @bbcrddave on March 17, 2017 11:26

This is something that the BBC would like to see in the WAVE specification. Whilst clearly not secure, user-agent strings are extremely useful for diagnostic and other uses.

I propose something very similar to the above:

WAVE/\<version> (\<vendorName>; \<modelName>; \<familyName>; \<softwareVersion>; \<reserved>)

where \<version> is the version of the specification (eg 2017), and the other fields are as described in the comment above. I guess we cannot reference OIPF here, so we will need to come up with our own definition of \<familyName>, but I think that is worthwhile for the benefits it offers in terms of grouping devices with equivalent features.

[mavgit]

From @tidoust on March 17, 2017 12:23

One problem with grouping is that it may also exclude. If you add a clause that the User-Agent string shall include tokens to flag compliance with this baseline spec, you will end up excluding (as in "they would no longer comply with the baseline spec) user agents that go beyond the baseline but are not necessarily aware of this spec or willing to adjust the User-Agent string they send. Regular desktop browsers come to mind.

Also, various user agents may actually want to prune the User-Agent string from too specific identifiers to reduce the fingerprinting surface. The Mitigating Browser Fingerprinting in Web Specifications document, developed by the Privacy Interest Group at W3C, highlights the User-Agent string in its definition of passive fingerprinting and includes a note that says:

[...] variation in the User-Agent string that reveals additional information about the user or device has been shown to provide substantial fingerprinting surface [...] https://w3c.github.io/fingerprinting-guidance/#a_standardized_profile

[mavgit]

From @bbcrddave on March 17, 2017 13:49

Regular desktop browsers come to mind.

Yes, fair point - I was only really thinking about CE devices at the time.

[mavgit]

From @bbcrddave on March 31, 2017 12:36

Given the comments in https://github.com/w3c/webmediaapi/issues/52#issuecomment-287340559, and lack of other positive opinion supporting this kind of addition, I propose we close this issue with no further action.

Call for consensus: Close issue with no further action.

[mavgit]

From @jpiesing on April 1, 2017 19:50

I don't agree to closing this issue with no further action until our thoughts on device capabilities are more advanced. Yes additions to the user agent have limitations and would not be a whole solution but they may be a useful piece of a solution for some categories of devices and some market segments.

[mavgit]

From @joeyparrish on April 5, 2017 15:57

User agent parsing and basing decisions on user agent strings has caused lots of grief for media projects I've worked on. It should be avoided whenever possible, IMO. I agree with closing this issue.

[mavgit]

I also agree with closing this issue. Additional issues with making decisions based, even partly, on user agent header:

• WAVE startup issue: When we publish this spec, there will be millions of devices that support all our required APIs, since we're only including APIs supported on all code bases. But these devices will all fail the user agent header test on day one, resulting in zero WAVE platforms. Alternatively, if we encourage apps to test for APIs, as recommended by W3C, these can all be WAVE-compatible platforms on day one.
• Spoofing: The user agent header is easily spoofed. For example, all user agents today spoof that they are Mozilla! Given the spoofing, a smart application will still need to test for APIs, making the user agent header test duplicative at best.
• W3C Recommendation path: It's one of the stated goals of this CG to try to get this spec to W3C Recommendation path. There is such W3C consensus against testing on user agent header, that this requirement won't survive transition to Recommendation path.

[mavgit]

From @jpiesing on April 6, 2017 6:52

There have been many discussions in other groups in WAVE about device capabilities. It's clear that there is no magic bullet and there will probably have to be a mixture of solutions for different market/ecosystem structures, different types of devices and over different timescales.

Most people hope the media capabilities API ( https://github.com/WICG/media-capabilities ) will be the long term solution but intermediate solutions will be needed until some future descendant of that is implemented in all 4 code bases. The nice thing about the user agent string being one element of a set of intermediate solutions is that it can often be changed by a browser integrator without changing the browser codebase itself.

If (part of) the user agent string is either used as a key into a database of device capabilities or even directly indicates some device capabilities then that might still be implementable for some products entering the market in 2018.

IMHO all intermediate solutions for indicating device capabilities have significant drawbacks but IMHO not being able to indicate them at all would be worse.

So what if it can be spoofed? If it's used to indicate device capabilities then all that happens is that the user experience for the spoofer isn't as good as it could be.

[mavgit]

From @jeremypoulter on April 6, 2017 10:56

The main issue I have with this is that I do not see a clear problem definition that this is trying to solve. I agree with the comments about why the User-Agent should not be used, but I think that some of the things (I believe) the change is trying to accomplish may be valuable.

I think we should close this issue after extracting a problem to solve and then look in to the alternatives, eg HTTP headers, JavaScript API, etc.

[mavgit]

From @jpiesing on April 7, 2017 10:8

I can only agree to closing this issue people agree that closing the issue here does not exclude HTTP user agents being one part of an overall WAVE solution to device capabilities.

[mavgit]

From @joeyparrish on April 7, 2017 14:58

User agent parsing is not about to go away. This is the current reality of the web, and device capability detection is still on the way. I have managed to treat ua-based decisions as a last resort in my projects, but not completely eliminate them.

That said, I think the original request to "make requirements on the formatting of the HTTP user agent string" is not something we should do. Whatever the problems you would like to solve with ua parsing, let's discuss those specifically in their own issues. Otherwise, ua formatting and parsing feels like a solution looking for a problem.

[mavgit]

From @joelkorpi on April 27, 2017 20:27

This was discussed in the NAB FTF meeting today and it was proposed that we include a WAVE user agent string as part of the specification which is to be implemented by WAVE-compatible hardware and software. However, because the agent string cannot be relied upon to be available immediately or universally across WAVE-enabled devices they will not be recommended as the primary way to indicate WAVE compatibility.

[mavgit]

@joelkorpi

it was proposed that we include a WAVE user agent string as part of the specification which is to be implemented by WAVE-compatible hardware and software. However, because the agent string cannot be relied upon to be available immediately or universally across WAVE-enabled devices they will not be recommended as the primary way to indicate WAVE compatibility.

I don't understand the proposal:

1. What is the WAVE user agent string requirement?
2. Is the new WAVE user agent string a MUST requirement or a SHOULD? If "it cannot be relied upon to be available immediately or universally across WAVE-enabled devices", it sounds like a SHOULD.
3. The phrase "...will not be recommended as the primary way to indicate WAVE compatibility." sounds like a recommendation for app writers (i.e. for the Developer Guidelines spec) not a recommendation for device implementors (i.e. not for the API spec).