Add a unique identifier for a PWA

[adewale]

If one is building a PWA Directory or App Store or search engine that detects PWAs one needs a way to uniquely identify a PWA from just the manifest.

Currently the spec doesn't explicitly say what that identifier or tuple of identifiers should be which leads to issues like: https://github.com/GoogleChrome/gulliver/issues/323

[dmurph]

I'm also ok with manifest_url. I'm fine with taking on the work to migrate to manifest url in chrome desktop. I don't think this is on the roadmap until at least q4.

[marcoscaceres]

During the f2f/TPAC, @dmurph agreed to provide us a summary of where we are at an our options here.

[aarongustafson]

I don’t think this was addressed above—forgive me if it was—but what is to stop a bad actor from spoofing the id of another app? Do you think there should be an additional check in place? Maybe something like origin + id? It would complicate things like domain migrations, but I feel like we could probably come up with a best practice for that as well.

[dmurph]

The id is intended to be only unique when combined with the domain - so the unique ID is technically origin+id. So a bad actor wouldn't be able to steal an app unless they can host a manifest on the same origin, which I think the group has considered an appropriate security boundary (and I believe is used for other specs).

If we were to talk about origin migration (which is out of scope of this discussion), I'm assuming, as a bare minimum, we would need 2 way authentication (old manifest points to new one, new one points to old one).

[aarongustafson]

The id is intended to be only unique when combined with the domain - so the unique ID is technically origin+id. So a bad actor wouldn't be able to steal an app unless they can host a manifest on the same origin, which I think the group has considered an appropriate security boundary (and I believe is used for other specs).

Perfect. Thanks for the clarification. Like I mentioned, I didn’t see this particular abuse vector addressed above, but it’s a long thread.

If we were to talk about origin migration (which is out of scope of this discussion), I'm assuming, as a bare minimum, we would need 2 way authentication (old manifest points to new one, new one points to old one).

Agreed. I’ll start a separate issue to discuss best practices for domain migration.

[aarongustafson]

I just published a Ratings & Reviews Prompt Explainer which makes use of the unique ID (implicit or explicit). Feedback encouraged.

[benfrancis]

During the f2f/TPAC, @dmurph agreed to provide us a summary of where we are at an our options here.

Is there a written record of this summary, and whether a decision was made? (Sorry I wasn't able to attend TPAC due to being on parental leave).

[dmurph]

https://www.w3.org/2020/10/26-webapps-minutes.html

[ralphch0]

+1 for allowing for the modification of the start_url, and the ability to set an identifier.

As developers of large apps, we have two use cases this would help solve: 1) We would like to be able to change our start_url, so we can add a tracking parameter. This also helps with a potential future migration of one of our app's start page. 2) We also have apps that don't have a stable manifest url, where an explicit identifier field would be needed. Some of our apps use a versioning model for the manifest url. We are also considering using XSRF tokens in the manifest url, to allow apps to generate per-user manifests securely (eg: dogfooding or A/B experimenting certain manifest changes).

[tomayac]

Something to think about when discussing "manifest_url" (see this comment above) are data URLs: <link rel="manifest" href="data:application/manifest+json;charset=utf-8,…">. I have seen PWAs repeatedly create manifests dynamically based on a JSON object.

[dmurph]

oh jeez wow not sure how we would deal with that..... if we are OK with complicating things even more, we could say that data urls are uniquely identified by the start_url....

I'd prefer to ban that, and say people should serve a manifest from their serviceworker if they really want it to be locally generated.

[dominickng]

Data URLs are perfectly valid URLs and have been supported for web app manifests for >5 years. I would be very uneasy about banning them - needing to do so suggests that the solution which requires banning them may not be a sufficient one.

[ralphch0]

Should we consider decoupling the two issues? 1) Introducing an id field that that takes precedence over the default id scheme 2) Consolidating the default id scheme across platforms.

[alancutter]

We should also spec a serialisation format for app IDs so sites can refer to each other.

[benfrancis]

@alancutter wrote:

We should also spec a serialisation format for app IDs so sites can refer to each other.

Or alternatively, if the manifest URL is used as the stable identifier for a web application, there's no need to invent a new ID serialisation format which can be used to link to other web apps.

Having the identifier dereference to the manifest could also have added benefits in use cases like the pwa-url-handler one linked above, because its metadata could directly be used to generate UI elements representing the handling apps (e.g. using the app title and icon in a permission prompt).

[dmurph]

@philloooo is doing an in-depth analysis here that we should be able to publish here by the end of the week.

So far manifest_url seems like it might be a security issue -

pretend Bing now has a music service, Bing Music, which is a PWA.

Along comes malware.com.... malware.com/manifest.json:

{
  ...
  id: "https://music.bing.com/manifest.json",
  name: "Bing Music!",
  start_url: "https://music.bing.malware.com",
  ...
}

And now malware.com has taken over bing music!

Anyways, explainer coming soon, and we're excited for feedback!

[benfrancis]

@dmurph wrote:

So far manifest_url seems like it might be a security issue ...

{
  ...
  id: "https://music.bing.com/manifest.json",
  name: "Bing Music!",
  start_url: "https://music.bing.malware.com",
  ...
}

To be clear, this is not what I mean when I say use the manifest URL as the identifier for the app.

Don't parse the manifest URL from an id member inside the manifest, use the URL from which you fetched the manifest as its identifier. That way only music.bing.com can host a manifest with the identifier of https://music.bing.com/manifest.json

Also, IIRC start_url currently has to be same-origin as document URL, so if this manifest was linked from a document hosted at malware.com then this start_url would not be allowed and would fall back to the document URL the app was installed from.

[dominickng]

It is a legitimate use case to host a manifest on a different origin to the start URL (e.g. a CDN). music.bing.com could have an identifier of cdn.bing.com in that case.

And while start_url currently has to be same-origin as document URL, https://github.com/w3c/manifest/pull/670 proposes to change that to the parent path of the manifest URL.

[dmurph]

Great - I think we can have a really good discussion soon once our explainer is out and we can talk about this stuff and consider all options :)

Sorry to jump ahead a bit!

[benfrancis]

@dominickng wrote:

It is a legitimate use case to host a manifest on a different origin to the start URL (e.g. a CDN). music.bing.com could have an identifier of cdn.bing.com in that case.

Yes that's fine, by using the URL the manifest was served from as the ID, by definition an ID of https://cdn.bing.com/manifest.json can only be served from https://cdn.bing.com/manifest.json. There still can't be a manifest at https://malware.com/manifest.json with an ID of https://cdn.bing.com/manifest.json.

And while start_url currently has to be same-origin as document URL, #670 proposes to change that to the parent path of the manifest URL.

Yes, that would be awesome. But as I understand it, even if that change finally lands, it just means that if https://malware.com/manifest.json doesn't provide a start_url then its start_url defaults to https://malware.com/. It still can't provide a start_url of https://music.bing.malware.com because if both the document URL and start_url are provided they still have to be same-origin.

Even if #668 succeeds in detaching manifest processing from the document URL entirely (which would be even more awesome), there still couldn't be a manifest at https://malware.com/manifest.json with an ID of https://cdn.bing.com/manifest.json.

I'm looking forward to reading Google's latest analysis, but so far from this 3.5 year thread my personal conclusion is that it really boils down to whether enabling URL versioning of manifests without needing to set up HTTP redirects is worth all the added complexity of:

1. Inventing a new ID/serialisation/linking system for web apps, parallel to web URLs, making web apps less directly linkable and discoverable
2. Making manifest updates a more complex multi-step process for operating systems, which can only take place when a user happens to visit a page which links to a manifest with a matching ID, using the same user agent they used to install the application
3. Dealing with the case of different manifests from different URLs claiming the same ID, which could itself be a security issue if not handled carefully

I think you could argue this risks compromising four key attributes of Progressive Web Apps: linkable, discoverable, safe and fresh.

I realise using manifest URL as the identifier would mean changing the current behaviour of Chrome desktop, but that seems like a comparatively small price to pay (which @dmurph has already volunteered to take on 😉).

[alancutter]

I'm quite compelled by preserving the linkability of manifest URLs as IDs. We could support manifest versioning via imports instead. E.g.

https://app.com/manifest.json:
{
  "import": "manifest-r31.8.json"
}

[philloooo]

Good point that using manifest_url is more linkable. Can you give an example use case that having the ID being a resolvable URL is needed/better?

Re: manifest_url versioning, If I understand it correctly, doing 301 redirect, or add an "import" syntax means that: the document will always still reference the original manifest, so user agents will keep the same previous ID for the app, and always do another round trip to fetch the latest content?

Note: we also still have the use case of specifying manifest as data url.

[benfrancis]

@philloooo wrote:

Good point that using manifest_url is more linkable. Can you give an example use case that having the ID being a resolvable URL is needed/better?

I have a few key use cases in mind:

1. Updates - If user agents can index web apps by their manifest URL, all they need to do to get the latest version of the manifest is to fetch that URL. This would help fill a key hole in the manifest specification regarding how to update a manifest, where the alternative would be much more complex. This is particularly useful where the web runtime is separate from the operating system (i.e. not something like Chrome OS or Firefox OS), because the operating system can fetch updates itself (e.g. to update icons on an app launcher). If the manifest URL does need to change for some reason, an HTTP redirect can tell the user agent to replace the current app rather than install a new one.
2. Remote installation
   1. App stores - We don't really talk about the fact that the way Microsoft uses manifests in the Microsoft Store is not compliant with the specification, because you can only really install an app from a document belonging to the app itself. Having the app ID resolve to its manifest is a step towards allowing installation of web apps from app stores/directories, if all you need to link to and install an app is its manifest URL. (See also: #668).
   2. App provisioning - Similarly, it could be possible in future to remotely provision a web app to a collection of user agents (e.g. a classroom full of Chromebooks), just by sending the manifest URL.
   3. Digital signage - Similar to the above, I'm currently working on a use case in digital signage where I'd like to be able to remotely install a web app to a kiosk runtime, which doesn't have its own user agent UI to install a web app. The kiosk runtime runs a single web app at a time full screen. It's quite common in digital signage to remotely provision APKs to an Android runtime, but the same is not currently possible with web apps.
3. Link relations - There was an example linked above where one web app manifest might want to refer to another web app manifest, to denote some kind of relationship between to the two apps. By using a simple web URL as an identifier it's possible to use link relations which describe all kinds of relationships between apps. Having the identifier resolve to metadata about the app is even more convenient (e.g. in the linked use case, for building the UI for a permissions prompt using the app's icon and title).

Re: manifest_url versioning, If I understand it correctly, doing 301 redirect, or add an "import" syntax means that: the document will always still reference the original manifest, so user agents will keep the same previous ID for the app, and always do another round trip to fetch the latest content?

As described above, if a user agent receives a redirect response when fetching a manifest URL, it can choose to replace the app (including its new ID), rather than install the new manifest as a separate application. That way the redirect only has to happen once on the user agent (but the server would need to maintain redirects for old manifest URLs for user agents which haven't updated yet).

The import approach could also work and might be more CDN friendly, but has the downside of always requiring an additional round trip, as you say.

There are obviously also other ways of telling a user agent that a manifest has changed which don't involve changing its URL, or re-fetching the whole manifest, (e.g. ETag or Last-Modified headers with less aggressive caching).

Note: we also still have the use case of specifying manifest as data url.

That would still work, a data URL can be used as an ID (I've done this in the past for storing icons). Obviously the limitation of choosing to use a data URL for a manifest is that if the manifest content changes, the manifest URL also changes and it becomes a new web app. But to me that feels like a reasonable compromise for such an edge case.

[dmurph]

@philloooo wrote:

Good point that using manifest_url is more linkable. Can you give an example use case that having the ID being a resolvable URL is needed/better?

I have a few key use cases in mind:

1. Updates - If user agents can index web apps by their manifest URL, all they need to do to get the latest version of the manifest is to fetch that URL. This would help fill a key hole in the manifest specification regarding how to update a manifest, where the alternative would be much more complex. This is particularly useful where the web runtime is separate from the operating system (i.e. not something like Chrome OS or Firefox OS), because the operating system can fetch updates itself (e.g. to update icons on an app launcher). If the manifest URL does need to change for some reason, an HTTP redirect can tell the user agent to replace the current app rather than install a new one.

Updates are definitely easiest in the manifest_url=global_id case. They are still possible using other schemes, but less nice.

2. Remote installation

   1. App stores - We don't really talk about the fact that the way Microsoft uses manifests in the Microsoft Store is not compliant with the specification, because you can only really install an app from a document belonging to the app itself. Having the app ID resolve to its manifest is a step towards allowing installation of web apps from app stores/directories, if all you need to link to and install an app is its manifest URL. (See also: #668).

I would expect that there would have to be a little more of a verification step here, either by the user agent or(and) by the store, to make sure that the document at start_url has a manifest link that matches the manifest_url. This step would probably be similar with other id schemes as well.

2. App provisioning - Similarly, it could be possible in future to remotely provision a web app to a collection of user agents (e.g. a classroom full of Chromebooks), just by sending the manifest URL.

This functionality, in theory, already exists in Chromium, as you can policy-install apps. It definitely has more hoops to jump through (loads the given URL, looks for manifest link, then uses that), but is possible.

3. Digital signage - Similar to the above, I'm currently working on a use case in digital signage where I'd like to be able to remotely install a web app to a kiosk runtime, which doesn't have its own user agent UI to install a web app. The kiosk runtime runs a single web app at a time full screen. It's quite common in digital signage to remotely provision APKs to an Android runtime, but the same is not currently possible with web apps.

Do you mind expanding on this? Would web packaging solve this use case? I could see us sending a web package to the kiosk as an option here too. I can also see the above policy-install functionality work too (more round-about, yes, but it works)

3. Link relations - There was an example linked above where one web app manifest might want to refer to another web app manifest, to denote some kind of relationship between to the two apps. By using a simple web URL as an identifier it's possible to use link relations which describe all kinds of relationships between apps. Having the identifier resolve to metadata about the app is even more convenient (e.g. in the linked use case, for building the UI for a permissions prompt using the app's icon and title).

I find this interesting! This is a nice feature. I think this is the first case here where it more on the impossible-side (or too-klunky-to-be-workable) for other schemes vs this scheme. But maybe I haven't thought of a way - any ideas how this could work in a different id scheme?

Re: manifest_url versioning, If I understand it correctly, doing 301 redirect, or add an "import" syntax means that: the document will always still reference the original manifest, so user agents will keep the same previous ID for the app, and always do another round trip to fetch the latest content?

As described above, if a user agent receives a redirect response when fetching a manifest URL, it can choose to replace the app (including its new ID), rather than install the new manifest as a separate application. That way the redirect only has to happen once on the user agent (but the server would need to maintain redirects for old manifest URLs for user agents which haven't updated yet).

The import approach could also work and might be more CDN friendly, but has the downside of always requiring an additional round trip, as you say.

There are obviously also other ways of telling a user agent that a manifest has changed which don't involve changing its URL, or re-fetching the whole manifest, (e.g. ETag or Last-Modified headers with less aggressive caching).

Note: we also still have the use case of specifying manifest as data url.

That would still work, a data URL can be used as an ID (I've done this in the past for storing icons). Obviously the limitation of choosing to use a data URL for a manifest is that if the manifest content changes, the manifest URL also changes and it becomes a new web app. But to me that feels like a reasonable compromise for such an edge case.

We could also do something weird like - say that the id = scope + "/manifest.webmanifest". Probably a bad idea. Any ideas about how to allow these to update & maybe migrate to a manifest file?

I think the main other issue with manifest_url = global_id is that you are STUCK with the url - if that was on a CDN, and that CDN goes out of business... or goes down, etc. You are stuck forever. Probably would mean, if we choose this option, we encourage devs to make sure they control the URL & 301 (is there a dns redirect you can do?) or something to a CDN if they need to. But I could see this biting people in the butt.

[aarongustafson]

One other use case would be some means of informing users they already have an app installed if they go to install the PWA from a different browser. Not so much to block it, but to inform the user that they will have 2 copies of the app (which could be confusing).

[benfrancis]

@dmurph wrote:

Digital signage - Similar to the above, I'm currently working on a use case in digital signage where I'd like to be able to remotely install a web app to a kiosk runtime, which doesn't have its own user agent UI to install a web app. The kiosk runtime runs a single web app at a time full screen. It's quite common in digital signage to remotely provision APKs to an Android runtime, but the same is not currently possible with web apps.

Do you mind expanding on this?

Basically I have a kiosk runtime which acts as a kind of remote control web browser. It hosts a web interface (and WoT API) into which you can enter a URL to tell the kiosk to load that web page on its screen and act as a digital sign. In addition to just loading a web page I'd like to be able to remotely install a web app to the kiosk, including installing a service worker so that content can be cached for offline use in the case of network outages. The kiosk can also be interactive, so the runtime may want to restrict users to the navigation scope of the installed app. The kiosk runtime does not have a user interface for navigating to and installing web apps itself, that functionality is only provided remotely via a separate UI which doesn't have access to documents inside the browsing context. Ideally I would like to be able to remotely install a web app onto the kiosk from that UI using its manifest URL.

Would web packaging solve this use case? I could see us sending a web package to the kiosk as an option here too.

I'm not sure, I'm afraid I've lost track of the latest attempts to bring packaging to the web since packaged apps were such a disaster for Firefox OS. But ideally I'd like to be able to just give the kiosk any manifest URL and have it pull down a PWA itself.

I can also see the above policy-install functionality work too (more round-about, yes, but it works)

I agree it could be possible to implement a workaround for this use case using another approach, but it would be much more complicated and unreliable to have to download and parse HTML files in order to extract manifest link relations and find manifest URLs (I'd rather leave the tricky task of parsing HTML to the browser engine, rather than my application).

I think the main other issue with manifest_url = global_id is that you are STUCK with the url

I understand why this might be tricky in some production environments, but I personally happen to think it's a good thing.

[dmurph]

@benfrancis wrote:

I can also see the above policy-install functionality work too (more round-about, yes, but it works)

I agree it could be possible to implement a workaround for this use case using another approach, but it would be much more complicated and unreliable to have to download and parse HTML files in order to extract manifest link relations and find manifest URLs (I'd rather leave the tricky task of parsing HTML to the browser engine, rather than my application).

As a side note, if you're doing this with a build of chromium, you can use the kWebAppForceInstall pref today - that will force-install apps as described above today, with some override options as well.

[dmurph]

I thought of a use case for Chromium browsers that makes manifest_url difficult - "Create Shortcut..." apps. These currently create a fake manifest if there is none, and then will 'upgrade' to a real manifest if one is ever linked. Unless I'm missing something easy here, these would have to be handled specially if we end up going with the manifest_url approach.

[alancutter]

Manifest-less Create Shortcut apps should use the start_url as their ID. Once a manifest is detected we can upgrade the app to use the manifest URL. This may require a user prompt to avoid cross-device sync conflicts.

[ralphch0]

@benfrancis

As described above, if a user agent receives a redirect response when fetching a manifest URL, it can choose to replace the app (including its new ID), rather than install the new manifest as a separate application. That way the redirect only has to happen once on the user agent (but the server would need to maintain redirects for old manifest URLs for user agents which haven't updated yet).

This means though that the site can never switch to linking to the new manifest url. Since it would not know if the client still has the old version of the app keyed to the old manifest url. The user agent would probably need to refetch all the existing manifests on the domain first to make sure the IDs are up to date, a process that's likely to be brittle (defunct urls, temporary server errors, etc..).

[benfrancis]

@ralphch0 wrote:

This means though that the site can never switch to linking to the new manifest url. Since it would not know if the client still has the old version of the app keyed to the old manifest url. The user agent would probably need to refetch all the existing manifests on the domain first to make sure the IDs are up to date, a process that's likely to be brittle (defunct urls, temporary server errors, etc..).

What are you worried will happen if the user agent visits a page linking to the new manifest before it has updated to that new manifest? And whatever it is, wouldn't that always happen in the alternative case of an id which can only be updated after following a link from such a page? The manifest could even theoretically never get updated if the user only ever visits pages which were within the navigation scope of the previous manifest but outside the navigation scope of the new manifest and/or no longer link to the manifest?

At least with a fixed manifest URL the user agent always has the opportunity to update.

[philloooo]

In the alternative case, If a manifest is updated , the document that's within the previous navigation scope will update their manifest link to the latest one or the document url will redirect to the new document url, but still keep the stable ID if the apps' origin is not changed.

I think either options, the user agents always have ways to update. The difference is with your proposal of how to do the redirect, the ID will be changed and we could hit a race condition for user agents, that loading the latest page finds the manfiest_v2, before existing installed app v1 is updated to v2, and result in duplicated apps being installed. Or we need to follow what @ralphch0 said to go through all existing apps to update their manifests whenever you browse to a link.

This also gets trickier with user agents that support syncing apps across devices.

[benfrancis]

@philloooo wrote:

In the alternative case, If a manifest is updated , the document that's within the previous navigation scope will update their manifest link to the latest one or the document url will redirect to the new document url, but still keep the stable ID

That's an interesting solution, I hadn't considered the possibility of pages which are no longer within the navigation scope of the application continuing to link to the manifest. Could that cause problems for new users who come along and install the web app from those old pages though? That might be OK, as long as the page doesn't become part of a new navigation scope (e.g. if a scope is split off into two separate apps or two apps are combined into one scope).

if the apps' origin is not changed.

This is of course another limitation of "an opaque token that uniquely identifies the app within the origin's namespace", it may not be safe to move the app/manifest to a different origin, including switching CDN.

I think either options, the user agents always have ways to update. The difference is with your proposal of how to do the redirect, the ID will be changed and we could hit a race condition for user agents, that loading the latest page finds the manfiest_v2, before existing installed app v1 is updated to v2, and result in duplicated apps being installed. Or we need to follow what @ralphch0 said to go through all existing apps to update their manifests whenever you browse to a link. This also gets trickier with user agents that support syncing apps across devices.

I agree that could be a problem. It seems recoverable though, by merging app installations once a redirect to the new ID is detected.

Ultimately I think if you want the start URL, scope and manifest URL to all be changeable, then there are going to be edge cases where things can get into a broken state. Introducing a new non-web ID namespace into the equation can solve some of those problems, but also create others (e.g. getting stuck in an update loop where two different pages point to two different manifests claiming the same ID).

I posit that there is no perfect solution, which is why after about a decade of discussing this topic (I'm not joking), we still haven't landed on a solution upon which everyone agrees.

I still personally think manifest URL is the best solution. The second best solution I've come across is origin + scope, but that has problems too. Bad experiences from packaged apps in Firefox OS make me very wary of introducing a new non-web ID namespace for web apps. If there absolutely has to be an ID inside the manifest, I would suggest it should at least be a URL which defaults to manifest URL, but I'm concerned that will paint us into a corner where #668 (and therefore some of the use cases described above) become difficult or impossible.

[philloooo]

Hi All, I created an explainer doc in my personal repo. Please take a look and create issues if you think there are missing use cases/incorrect information or any suggestion for edits!
We can still keep the ongoing conversations about which way is better here.

[dmurph]

And to report current conclusions on the doc - the current best solution, given all the use cases that we found, is the "global_id = start_url_origin + manifest_specified_id", where the default value of "global_id" is the start_url.

We always could have missed something though, so eyeballs & feedback appreciated on that explainer :)

[ralphch0]

Thanks @philloooo & @dmurph for the explainer. Personally in agreement with the conclusions of the doc.

One comment I had was regarding the default ID when the "id" field is missing:

Feels to me like if we were building this from scratch, we would likely go with scope? It's the same as service workers, and less likely to change. Sounds like the main reason start_url is preferred is because we would only be changing expectations for one set of devs (Android), but I wonder if long term scope will make more sense for future developers. Is start_url even that intuitive/expected by Desktop developers today?

I assume here that regardless of what we choose, already installed apps will be migrated to the new ID scheme? (i.e. on the first chrome version that has the new code, the ID'ing will be migrated to the new scheme, and updates to the app will continue working by matching using the new ID scheme). IOW, there should be no technical diff (since I assume almost all apps out there don't use a dynamically changing scope), just a change in developers mental model that they may need to be aware of.

[benfrancis]

CC @fabricedesre @tantek @mounirlamouri

@philloooo wrote:

Hi All, I created an explainer doc in my personal repo. Please take a look and create issues if you think there are missing use cases/incorrect information or any suggestion for edits!
We can still keep the ongoing conversations about which way is better here.

Thank you for sharing that analysis, @philloooo.

When discussing behaviors on Android, this doc is exclusively only covering Chromium-based browsers on Android

...

When discussing behaviors on desktop, this doc covers Chromium-based browsers and Firefox (desktop).

As I understand it Firefox desktop does not support installing PWAs, so this analysis actually only covers Chrome desktop and Chrome for Android.

Taken for what it is (an analysis of what works best for the Chrome team, given their particular set of requirements and technical debt), I think the conclusions are completely reasonable.

Overall, if we use manifest_url as the global_id, we need to not support the use case of making manifest_url updatable or deal with non stable primary keys for managing apps across devices... If there weren't existing apps & infrastructure that relied on changing manifest_url, and this was being designed from the beginning, this option is a lot more attractive.

My reading of this is "If we'd have thought about this earlier we may have concluded that manifest URL was the best identifier for a web app, but since Chrome used start URL as the identifier and we never told developers that manifest URLs shouldn't change, we're now going to have to specify what Chrome desktop already does, but with a hack to work around the fact that start URL was always a bad choice as it's the URL which is most likely to change."

The explainer hasn't done anything to change my own conclusions about the best solution (manifest URL as ID, using HTTP redirects for discouraged URL changes), but it would be great if we could hear from other implementors.

To the best of my knowledge, the following user agents all support install/add to home using a web app manifest:

• Android
  • Firefox
  • Chrome
  • Samsung Internet
  • Edge
  • Opera
  • Brave
  • UC Browser
  • QQ Browser
  • Huawei Browser
  • Baidu Browser (caniuse.com says manifest is supported, but I can't find any information)
• iOS & iPadOS
  • Safari
• Windows
  • Chrome
  • Edge
• MacOS
  • Chrome
  • Edge
• Linux
  • Chrome/Chromium
• Chrome OS
• KaiOS
• Webian (for completeness, but probably only of interest to me, Webian Shell uses manifest URL as the ID)

As the explainer notes, user agents are not the only consumers of manifests. What about app stores? E.g.

• Microsoft Store
• Play Store
• AppStore (App-Bound Domains)
• Kai Store
• PWA Rocks
• PWA Store
• ProgressiveApp.Store

[philloooo]

Feels to me like if we were building this from scratch, we would likely go with scope? It's the same as service workers, and less likely to change. Sounds like the main reason start_url is preferred is because we would only be changing expectations for one set of devs (Android), but I wonder if long term scope will make more sense for future developers. Is start_url even that intuitive/expected by Desktop developers today?

I agree. The scope is a better choice if not considering change of expecations from one set of devs. A default value is supported mostly to allow smooth transition from existing web apps that don't have IDs. That's why we are favoring start_url as that's expected by existing desktop users.

I assume here that regardless of what we choose, already installed apps will be migrated to the new ID scheme? (i.e. on the first chrome version that has the new code, the ID'ing will be migrated to the new scheme, and updates to the app will continue working by matching using the new ID scheme).

They will be migrated to the new ID scheme and if we are choosing the ID scheme that has a new id field, the default value will be used if the apps don't have it specified.

As I understand it Firefox desktop does not support installing PWAs, so this analysis actually only covers Chrome desktop and Chrome for Android.

Oh thanks for pointing out, I just found out the announcement of them dropping the support recently. I will remove firefox desktop from the doc.

My reading of this is "If we'd have thought about this earlier we may have concluded that manifest URL was the best identifier for a web app, but since Chrome used start URL as the identifier and we never told developers that manifest URLs shouldn't change, we're now going to have to specify what Chrome desktop already does, but with a hack to work around the fact that start URL was always a bad choice as it's the URL which is most likely to change."

I think this is misrepresenting what I was saying in the doc. The doc was comparing manifest_url vs a id field. With id not specified, start_url was preferred as the default value. But that's totally different from using start_url as the id. And apparently id is the most stable option. I also tried to explain in detail that doing 301 redirect for use case of changing manifest_url either doesn't really support the use case, or ends up with unreliable behaviors for user agents.

Thanks for listing out the current user agents and PWA stores. Most of the user agents you are listed are chromium based. As I stated in the doc, safari doesn't support updating/looking up installed PWAs, so it doesn't add any use cases relevant to the unique ID problem. I'd be interested to know any use cases that I am missing in this doc.

PWA stores are being considered when comparing the pros and cons of the options. I can elaborate more on specific examples of PWA stores in the beginning of the doc if that's helpful.

[dmurph]

It seems like the main sticking point here is the 4th scenario - changing the manifest url. We know many sites do this, and crawling the data from https://pwa-directory.appspot.com shows us about 8.4% of manifests seem to have versions. Even with false positives, and that directory not being authoritative, that's a really high percentage, and a higher percentage than 2018: https://github.com/w3c/manifest/issues/586#issuecomment-386820382.

If chromium never supported a changing manifest url, then this scenario wouldn't be as important, but unfortunately it was supported and thus people now rely on it. Seems to be the story of the web platform, for better or worse :/

Command:

curl -sSL 'https://pwa-directory.appspot.com/api/pwa/?limit=4000' | jq -r '.[] | .manifestUrl' | sort | perl -ne 'print if /[0-9a-fA-F]{7}/ || /v[0-9]+/ || /v=/'

Total PWAs & percentage:

last valid page: https://pwa-directory.appspot.com/?page=130
130 * 32 = 4160 total

353 PWAs in the below list
353 / 4160 = ~8.4%

Details

``` http://res.cloudinary.com/dyyjph6kx/raw/upload/v1502969304/webui/eng/icons/manifest.json https://13tv.co.il/wp-content/themes/reshet_tv/assets/images/favicon/manifest.json?v=1.3.12.0 https://15f7fy16numklnwbn3m52e15-wpengine.netdna-ssl.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=303798283697 https://3235253.ucoz.ru/appmanifest/manifest.json https://3235253.ucoz.ru/manifest.json https://3235253.ucoz.ru/manifest_u.json https://3235253.ucoz.ru/manifest_v.json https://3235253.ucoz.ru/rss/manifest.json https://7352323.ucoz.ru/7manifest.json https://7352323.ucoz.ru/manif9est.json https://7352323.ucoz.ru/manifest.json https://ae.almosafer.com/mweb/manifest_almosafer.json?1539868957 https://agen.bukalapak.com/_nuxt/manifest.387a3d55.json https://alleideen.com/manifest.0d8eb347138956bc50bca91b31fb6204.json https://app.milanote.com/manifest-20201221.json https://app.truewealth.ch/assets/favicons-d5c20aa/manifest.json https://app.uplancer.io/_nuxt/manifest.dacc6f01.json https://aqi.breezo.in/manifest.json?v=0.2 https://arquivosapk.com/wp-json/pwa-for-wp/v2/pwa-manifest-json https://assets.slideslive.com/assets/favicon/manifest-1145378c0ec8e607c095f2fe43a6a16e7c22f86161165c502d836eebe2452832.json?v=vMr5gnj6OR https://assets.vita4you.gr/static/version1575339893/frontend/Fixit-mobile/vita4you/el_GR/manifest.json https://assets.vodspy.de/img/favicons/manifest.json?v=rM3LLEeJYB https://atmocast.com/static/weather_main/www/static/weather_main/dist/manifest/manifest.eea759a01fcb.json https://auskunft.avv.de/manifest.json?v=2.3.4 https://auth.magicleap.com/manifest.json?v=20171020 https://b2v8w6eq1p1erh.cdn.jp.idcfcloud.com/sp/top/json/manifest.json https://badoo.com/badoo/manifest-en.json?v101 https://bael-theme.jake101.com/_nuxt/manifest.a2e69f0a.json https://banca.oimparcial.com.br/app/themes/banca/assets/imgs/icons/manifest.json?v=518 https://bber.unm.cool/_nuxt/manifest.6ac3b74e.json https://bitsofco.de/assets/manifest.json?v=25a0d663d4 https://bitsofco.de/assets/manifest.json?v=aa20601b17 https://brex.com/icons-e860f2dd2739fb15edb772012ada22b9/manifest.json https://buddy.works/manifest.json?v=800 https://burdie.co/wp-json/app/v1/pwp-manifest https://catalogue.accasoftware.com/mn-catalogo-en.json?v=110 https://cdn.framebridge.com/assets/favicons/manifest-33209ed395f245282c3a56d5cad485e1.json?v=BGBGy60NMJ https://cdn.launchforth.io/7bf59d7/images/forth/favicons/manifest.json?v=OmyQRN88Nf https://cdn.quiqup.com/webapp/qa/2.27.2/develop/f315cdaa3883ac2a24a2fb7ca2f8cec33c7d1d74/manifest.5c43e6283efbdbaa8fd7d567201c273b.json https://cdn.shopify.com/s/files/1/0086/4865/4895/t/2/assets/site.webmanifest?v=10225873222616858641 https://cdn.shopify.com/s/files/1/0196/9616/t/72/assets/manifest.json?8865377658911702158 https://cdn.shopify.com/s/files/1/0199/9492/t/112/assets/manifest.json?15459288187714632786 https://cdn.shopify.com/s/files/1/1346/5473/t/60/assets/manifest.json?9199080155338096461 https://cdn.shopify.com/s/files/1/1366/8315/t/7/assets/manifest.json?8987383188635618396 https://cdn.shopify.com/s/files/1/1425/6082/t/97/assets/manifest.json?6342006907692054657 https://cdn.shopify.com/s/files/1/1751/8993/t/60/assets/manifest.json?3304484036080071639 https://cdn.shopify.com/s/files/1/2546/6304/t/3/assets/manifest.json?13114392473843896000 https://cdn.shopify.com/s/files/1/2964/9158/t/27/assets/manifest.json?14159899360358630437 https://celebrities.pictures/wp-json/wp/v2/web-app-manifest https://chastnik-m.ru/manifest.json?v=0.2 https://club.tarifhaus.de/manifest.json?v=v1.17.0 https://competitiondigest.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=467456738029 https://connery.dk/graphics/manifest.json?version=1532614009 https://d2zcl7c6767qct.cloudfront.net/web-static-08/web_weathertab/favicon/manifest.2dee4bb9ca57.json https://d30hza86imyv6n.cloudfront.net/statics/js/fcm/manifest.json?v=20180717122826 https://dbushell.com/manifest.webmanifest?v=10.2.0 https://de.tonybet.com/assets/theme15/images/favicons/manifest-449749a474a645bd5d1b8ada7264e2c5.json https://decider.com/wp-content/themes/nypost-2016/static/decider-manifest.json?ver=1ea14db262b1746d91d4 https://deliverydudes.com/manifest.json?v=3.4.2-fbbd0343066d0de6d6899ebfdc26a8a2e6bcc035-desktop https://deliverydudes.com/manifest.json?v=3.4.3-22b7c67ac3ede49412caab9ba33a59f88fa6929b-desktop https://demo.scandipwa.com/static/version1573566737020/frontend/Scandiweb/pwa/en_US/Magento_Theme/manifest.a848d42b8c744af528977d4408ceb768.json https://designwebkit.com/manifest.json?v1 https://dev.bber.unm.edu/_nuxt/manifest.6ac3b74e.json https://dj-extensions.com/templates/dj-exetensions-v3/icons/manifest.json https://dl.myqiwei.com/jdb-gamehallv3/pool/manifest.29b173a7.json https://do.getcompliant.com/manifest.json?v=20180927 https://dresslily.com/manifest.json?20180117 https://dwbxi9io9o7ce.cloudfront.net/firebase-cloud-messaging/manifest.83370ce1333a.json https://dwtricks.net/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=862555086377 https://earningkart.in/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=511885697223 https://ecourse.el-css.edu.om/manifest.8d9136a424a4e9fe28c4540e560b2eb4.json https://edugorilla.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=482472720033 https://ehazi.hu/manifest.json?v=2 https://emojibombs.com/manifest.a73d7e65.webmanifest https://emojibombs.com/manifest.d0fb4354.webmanifest https://ethiopian-calendar.netlify.com/assets/icons-b23883c8/manifest.json https://eurekamag.com/favicons/manifest.json?v=rM3ed7neBK https://face2faceafrica.com/wp-content/themes/f2fa-v3/manifest.json https://facerepo.com/app/images/favicons/manifest.json?v=8190 https://farlofacile.com/s/tmpl/on/manifest/manifest.json?v=dLBXnGO3e6 https://findpizza.space/_nuxt/manifest.99f3e3ec.json https://fotobiz.pro/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=151586796557 https://gazeta-licey.ru/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=338097788326 https://geekpublicitario.com.br/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=45077066116 https://generative.fm/c58bca5e9803118f86e9c9afcf9bdb1f.webmanifest https://gruzovichkof.ru/site.webmanifest?v=69BqJv4KQN https://hackr.io/manifest.json?v=1.1 https://hardwaresfera.com/wp-json/pwa-for-wp/v2/pwa-manifest-json https://health.cloudtcm.com/_nuxt/manifest.cb7a46f8.json https://hindi.indiansutras.com/browser.json?v=1.0.2 https://horacekeung.github.io/scrum-poker/_nuxt/manifest.3fb4b0a0.json https://huren.flixbus.nl/manifest.json?dcea861a57 https://hydrobuilder.com/skin/frontend/rwd/hydrobuilder/manifest.json?v=NmP8E8oJLa https://infohookah.ru/wp-json/app/v1/pwp-manifest https://intriper.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=10954234324 https://invidio.us/site.webmanifest?v=a3045a3 https://is-nb.a8e.net.br/images/mobile/favicon/manifest.json?v=92aaaa https://jcl.cz/favicons/manifest.json?v=8j6XJMAvAp https://job-draft.jp/assets/favicon/manifest-5e26f226e3746a244b9991698dd9f76d0d2a8498f2f0e221ad4bffb2aeeb067e.json https://journalisticapp.com/_nuxt/manifest.452c72d4.json https://jovemaprendizbr.com.br/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=878054065872 https://kannada.indiansutras.com/browser.json?v=1.0.2 https://lavito.pl/images/favicon/manifest.json?v=20160928100049 https://leifheit.de/manifest.5cdb6c2deddae2a7fde58d6f1d9ed298.json https://lifeacademy.ru/Media/favicons/manifest.json https://liveindex.org/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=247458917002 https://livetoday.online/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=46304322746 https://liwli.ru/s/i/manifest.json?v=5A5ydyp6Jy https://lowerspendings.com/_nuxt/manifest.070d2d8c.json https://lunarworks.co.uk/site.webmanifest?v=XBzAaWkGGw https://m-loi-pinel.trouver-un-logement-neuf.com/manifest.json?v=8jeqgYk97l https://m.ithome.com/json/manifest.json?v=20190214 https://m.ithome.com/json/manifest.json?v=2019042801 https://m.livingathome.de/r1531314069712/manifest.json https://m.met.hu/site.webmanifest?v=2bBRvozOrO https://m.visitjeju.net/_nuxt/manifest.b93ff49c.json https://malayalam.indiansutras.com/browser.json?v=1.0.2 https://market24hclock.com/manifest.json?v=2016.01 https://matexperience.com/site.webmanifest?v=694olv6vGq https://meapaixonei.com.br/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=766161200110 https://melaku.ml/assets/icons-b23883c8/manifest.json https://mmm.dk/graphics/manifest.json?version=1532529712 https://mobile.48365365.com/WebAppManifestV2.json https://msk.gruzovichkof.ru/site.webmanifest?v=69BqJv4KQN https://mundomulheres.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=703126565085 https://net4game.com/manifest.json?v=3 https://new.reddit.guide/_nuxt/manifest.3fd2d7ad.json https://nico.news/_nuxt/manifest.e3e59dbe.json https://nocorruption.in/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=548458404239 https://number-place-puzzle.net/manifest.json?20180620 https://numerics.info/_nuxt/manifest.5c7e420c.json https://oimparcial.com.br/app/themes/impar/assets/imgs/icons/manifest.json?v=3 https://optinium-apps.co.uk/Assets/Icons/site.webmanifest?v=PYePN5lxOa https://parismatch.be/app/themes/parismatch-v1/assets/images/favicons/manifest.json https://prod-static.disney-plus.net/builds/a823eb64c7e453be430322b0d8a71a8e4ae624e3_1579906393987/images/favicons/manifest.json https://psd.download.land/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=1040297877182 https://push.nextincareer.com/app/v3/manifest.json.php?gcm_sender_id= https://pymex.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=34581308698 https://r34.app/_nuxt/manifest.02b4b57d.json https://r34.app/_nuxt/manifest.6dc733da.json https://r34.app/_nuxt/manifest.b0237dbf.json https://r34.app/_nuxt/manifest.cd2b7647.json https://r34.app/_nuxt/manifest.d10eabcf.json https://radio.codemonkey-spb.ru/res/js/manifest.json?1ea25136 https://rainu.github.io/dev-notes/_nuxt/manifest.1a0b3aa1.json https://reittiopas.hameenlinna.fi/icons-hameenlinna-dac763a78f4a5fda6d72934070bb5c33//manifest.json https://resource.wakelet.com/v2/4.4.3/manifest.json https://reteteculinare.ro/assets/push/manifest.json?t=1532538296 https://s1.hdslb.com/bfs/static/jinkela/mstation-h5/manifest.01d8c0c950f8b24714dd3e8afc8d88f8de18494b.json https://sanieattivi.it/s/tmpl/on/manifest/manifest.json?v=dLBXnGO3e6 https://sayhello.ch/wp-json/app/v1/pwp-manifest https://seekingalpha.com/samw/manifest.b3b95f524139044321637a3833ada808.json https://sexshop69.pl/favicons/manifest.json?v=wAAwRrg2LW https://siamoption.com/manifest.json?v=5 https://sizeer.cz/common/images/favicon/manifest.json?v= https://sizeer.de/common/images/favicon/manifest.json?v= https://sizeer.lt/common/images/favicon/manifest.json?v= https://sizeer.sk/common/images/favicon/manifest.json?v= https://smartelement.netsons.org/index.php?rest_route=/wp/v2/web-app-manifest https://socialcdn.i-say.com/assets/data/manifest.json?t=1532528798881 https://startsat60.com/app/themes/startsat60v7/favicons/manifest.webmanifest https://static-cache.md.uaprom.net/image/portal/icons/manifest_prom.json?r=31b0755eb4f7b6cae4042261a24a38ce https://static.glampinghub.com/img/pets-manifest.95617630.json https://static5.mnlcdn.com/assets/favicons/manifest.json?v=kPgJaJEYB0 https://studitemps.de/wp-content/themes/limitless/manifest.json?v=2 https://suzuki-club.ru/manifest.json?v=m2dX696Pkj https://szybkagotowka.pl/manifest.json?v=pgrXrjp8JB https://tamethebots.com/_nuxt/manifest.a71cb8c6.json https://tcg.loke.dev/_nuxt/manifest.2ca00a5a.json https://techmaniacs.gr/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=811528213463 https://tehnot.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=662102224587 https://teknobolge.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=372304022646 https://terahertz.fr/wp-json/pwa-for-wp/v2/pwa-manifest-json https://thebroodle.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=304973626749 https://thriva.co/_nuxt/manifest.7b37346d.json https://tonybet.com/assets/theme15/images/favicons/manifest-449749a474a645bd5d1b8ada7264e2c5.json https://topwords.me/site!7e0a6c72546e9794ac2477c41d73a550.webmanifest https://tribunaonline.com.br/theme/tol/assets/favicon/manifest.json?5bf6fec5b9 https://tsuriho.com/wp-content/themes/tsuriho_f0ca56f/icons/manifest.json https://unsplash.com/site-v2.webmanifest https://v8.dev/.webmanifest https://web.sylinghim.com/wp-content/plugins/OneSignal-WordPress-Plugin-a176bc7f4bfe19d4deca899c0aa0085dad14c7b5/sdk_files/manifest.json.php?gcm_sender_id=22356042089 https://webimpng.web.app/manifest.json?h=c7a5441a53a565aa8f9f7497c1126f52 https://wiiu.hacks.guide/images/manifest.json?v=PYEmwKvQAx https://wmarket.com.ua/manifest.json?v=2 https://worldissmall.fr/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=2397340514 https://www.4588888.ru/manifest.json https://www.aquasana.com/manifest.json?v=xQwx0zQO2d https://www.arteblog.net/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=349712862781 https://www.autabuy.com/manifest.json?v=4 https://www.b-mall.ro/assets/v-20191205160838/vendor/onesignal/manifest.json https://www.babygames.com/manifest.json?2017022102 https://www.belk.com/on/demandware.static/Sites-Belk-Site/-/default/dwa7de5e17/manifest.json https://www.birminghampost.co.uk/manifest.json?v=00277067608714089d14c9532ff9fa55 https://www.bonmarche.co.uk/on/demandware.static/Sites-BONMARCHE-GB-Site/-/default/dwd11febf9/images/webclipicons/manifest.json https://www.bons-plans-astuces.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=495554646901 https://www.bridgemedia.ru/manifest.json?v=3 https://www.browsersync.io/manifest.json?v=qAqkxQaJm0 https://www.buddy.works/manifest.json?v=800 https://www.burtonmail.co.uk/manifest.json?v=d6678a8591dc5e45e3a5d07875099abe https://www.bustimes.org.uk/static/manifest.7111e27983b6.webmanifest https://www.carethy.net/imgs/4/favicons/site.webmanifest?v=2 https://www.chesterchronicle.co.uk/manifest.json?v=246d5c7d7c46fe9606e71016395ab784 https://www.chroniclelive.co.uk/news/manifest.json?v=2d9b353308db652f99447f552b69d088 https://www.cityplug.be/favicon/manifest.json?v200 https://www.cizgiroman.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=7072932560 https://www.club-vulkan-club.com/uploads/pwa/manifest.json?v=1491403533 https://www.club-vulkan-slots.org/uploads/pwa/manifest.json?v=1491403533 https://www.codecourse.com/_nuxt/manifest.bf25d7b2.json https://www.coininvest.com/uploads/site_settings/manifest/manifest.json?v=22 https://www.concealednation.org/manifest.json?v=2bb3QegpEf https://www.connery.dk/graphics/manifest.json?version=1532614013 https://www.connexion-emploi.com/manifest.json?1532617102 https://www.croydonadvertiser.co.uk/manifest.json?v=9bb6f5c11e7bb1905bc12840740df6f2 https://www.culinar.ro/assets/push/manifest.json?t=1532532470 https://www.culinar.ro/assets/push/manifest.json?t=1532533689 https://www.cybrhome.com/favicons/manifest.json?20180702184436 https://www.dahaboo.com/manifest.json?v=20180709 https://www.dbushell.com/manifest.webmanifest?v=10.2.0 https://www.debaser.it/manifest.json?v=3 https://www.designwebkit.com/manifest.json?v1 https://www.diana.bg/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=392133639149 https://www.dubaijobs77.com/manifest.json?sndid=103953800507 https://www.dublinlive.ie/manifest.json?v=5651aec8749f353696bdd3bc2190b30d https://www.e-lens.com.br/_nuxt/manifest.d05cec94.json https://www.elk.co/favicon/manifest.json?f194cabf7e92c09271e5cc21f5f08876 https://www.elkedagietsleuks.nl/manifest.json?_=1532556244 https://www.essexlive.news/manifest.json?v=8f5dbf5344afe4d4ebd21eb7a51ac322 https://www.facerepo.com/app/images/favicons/manifest.json?v=8190 https://www.fastfoodmenuprices.com/wp-content/uploads/fbrfg/site.webmanifest?v=lk2j03rYvj https://www.feedough.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=507835320803 https://www.findafishingboat.com/manifest.json?v=vMMo8wbo3G https://www.findspecials.co.za/Images/Icons/manifest.json?v=JynK8GNLGe https://www.finevent.top/_nuxt/manifest.3023ac05.json https://www.finevent.top/_nuxt/manifest.b9d56fcd.json https://www.finevent.top/_nuxt/manifest.e29fcaba.json https://www.fontsquirrel.com/manifest.json?v=2 https://www.foodservicedirect.com/static/version1540372735/frontend/BrewInteractive/FsdTheme/en_US/assets/images/favicon.ico/manifest.json https://www.football.london/chelsea-fc/manifest.json?v=4177f5a241e402b6a81f7603bc1b2701 https://www.freedym.com/main/wp-content/uploads/fbrfg/manifest.json?v=gAa4em0oyd https://www.getsurrey.co.uk/manifest.json?v=6e38aa1ad34fbdb9bab04d45692696cc https://www.gloucestershirelive.co.uk/manifest.json?v=8cfb8836786918f731c4c5838b3f9d13 https://www.goandroid.co.in/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=199824241014 https://www.goodsrepublic.com/media/manifest/manifest.json?20170919-213446 https://www.graphiccompetitions.com/site.webmanifest?v=Gvkqe429dM https://www.grimsbytelegraph.co.uk/manifest.json?v=c8c7793858b56b2ea6a0db7b65941fe7 https://www.gruzovichkof.ru/site.webmanifest?v=69BqJv4KQN https://www.gutezitate.com/manifest.json?v=2016 https://www.hertfordshiremercury.co.uk/manifest.json?v=2c9b01885310c626d32d56ac44cb795f https://www.hpbn.co/7a58c37113db4464699ec4f4646b5566.json https://www.hqroom.ru/manifest.json?v=777 https://www.icover.ru/manifest.json?v=A0RbE3GQBj https://www.iheart.com/v8.24.1/2720e41/bundles/manifest.json https://www.indianbodybuilding.co.in/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=813727397953 https://www.indiansutras.com/browser.json?v=1.0.2 https://www.inmyarea.com/manifest.json?v=1.0 https://www.inshared.nl/manifest.json?v3 https://www.jcl.cz/favicons/manifest.json?v=8j6XJMAvAp https://www.jibo.com/wp-content/themes/jibo-v2/jibo.webmanifest https://www.justwebworld.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=585128775620 https://www.koala.com.au/pwa/manifest.3b4de2b4.json https://www.kosmetik4less.de/manifest.json?1532366103 https://www.leifheitsklep.pl/manifest.5cdb6c2deddae2a7fde58d6f1d9ed298.json https://www.lifeacademy.ru/Media/favicons/manifest.json https://www.lift.co/manifest.0279069221784098277312156ea5edc4.json https://www.lincolnshirelive.co.uk/manifest.json?v=f5a577e6287fd6c05db173ddc72b10de https://www.liverpoolecho.co.uk/all-about/liverpool-fc/manifest.json?v=6f25c85eff7278914c6d2b2e2ef98830 https://www.liwli.ru/s/i/manifest.json?v=5A5ydyp6Jy https://www.magicleap.com/static/manifest.json?v=20171114 https://www.mahiroffice.com/wp-json/wp/v2/web-app-manifest https://www.marktguru.at/assets/manifest.at.json?v3 https://www.minhatatuagem.com/wp-content/uploads/fbrfg/manifest.json?v=qAqvK0vEev https://www.mirror.co.uk/manifest.json?v=e3a19310c81adecc1db07d28d9f6877a https://www.mivoto.app/static/manifest.835ebd007499.json https://www.mmm.dk/graphics/manifest.json?version=1532529717 https://www.monito.com/_sponse/manifest.142e82b8.json https://www.motocms.com/manifests/manifest-en.json?v3 https://www.movieclock.com/manifestmovie.json?v=Misia https://www.mrjmpl3-official.es/_nuxt/manifest.2ac1f5d0.json https://www.mygadget.su/manifest.json?v=lkgnRnWjAx https://www.nature.com/static/manifest.1a481c42b1.json https://www.net4game.com/manifest.json?v=3 https://www.newsheadlines.com.ng/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=376386989667 https://www.numerics.info/_nuxt/manifest.5c7e420c.json https://www.nur.kz/9497ff436/manifest.json https://www.nydj.com/on/demandware.static/Sites-NYDJ-Site/-/default/dw13e5aa58/images/site.webmanifest https://www.optimize.me/wp-content/themes/philosopher/images/favicons/manifest.json?v=X5rbgYn6zg https://www.pescanova.pt/manifest.json?v=2.3.0 https://www.pinnacleperformanceandnutrition.com.au/a/sc/manifest.json?v=2 https://www.pkjobs77.com/manifest.json?sndid=103953800507 https://www.pooltracker.com/manifest.json?v=YAK4a4y4Rgc https://www.presto-pizza.ro/bundles/mountfrontend/favico/manifest.json?version=$Id:%20c1488 https://www.qefira.com/assets/qe-site/favicon/site.a90bed74.webmanifest https://www.qqcitations.com/manifest.json?v=2016 https://www.resto.be/across/resources/static/87c90c49965f12e0060453bbffe74b086c2f8b39/site/en.resto.be.json https://www.retailnews.asia/wp-content/uploads/fbrfg/site.webmanifest?v=NmPGez5wvx https://www.reteteculinare.ro/assets/push/manifest.json?t=1532538298 https://www.revealedrecordings.com/uploads/assets/1530694500.1528097602/static/manifest.json https://www.rockantenne.de/assets/icons/rockantenne-de/manifest.json?v=2 https://www.scunthorpetelegraph.co.uk/manifest.json?v=c8c7793858b56b2ea6a0db7b65941fe7 https://www.sexshop69.pl/favicons/manifest.json?v=wAAwRrg2LW https://www.share4you.com/en/wp-content/themes/share4youv2/manifest/en/manifest.json https://www.siamoption.com/manifest.json?v=5 https://www.sizeer.cz/common/images/favicon/manifest.json?v= https://www.sizeer.de/common/images/favicon/manifest.json?v= https://www.sizeer.sk/common/images/favicon/manifest.json?v= https://www.southportvisiter.co.uk/manifest.json?v=548e74556b39b6b25a2b7a4828f7783e https://www.sozler.im/_nuxt/manifest.ffa97656.json https://www.sparkk.tv/site.webmanifest?v=yyQK3y337b https://www.speedtest-ptcl.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=238445687875 https://www.standoutbooks.com/manifest.json?v=PYY4W494ra https://www.stavros.io/static/images/favicons/manifest.json?h=facfed0f https://www.sumissura.com/en-us/manifest_json?v=1531917562 https://www.suzuki-club.ru/manifest.json?v=m2dX696Pkj https://www.tajawal.com/mweb/manifest_tajawal.json?1539868353 https://www.tappytoon.com/v2/manifest.json?id=72de668d3ea21fc516b0 https://www.technodom.kz/static/version1573576417415/frontend/Technodom/pwa/en_US/Magento_Theme/manifest.472a41db5f3a18af69a553705dfa3fc6.json https://www.tefenua.gov.pf/manifest.db4f1fcd181b13fb4f3a9158c895007c.json https://www.theatlantic.com/assets/static/a/frontend/dist/theatlantic/manifest/lacroix/manifest.523d8b9c9b01.json https://www.thehappening.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=237543628831 https://www.thesmackdownhotel.com/manifest.json?v=2017 https://www.thriva.co/_nuxt/manifest.7b37346d.json https://www.tipsport.org/images/chance/favicon/manifest.json?v=2 https://www.todamujeresbella.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=628438311256 https://www.tonybet.com/assets/theme15/images/favicons/manifest-449749a474a645bd5d1b8ada7264e2c5.json https://www.tribunaonline.com.br/theme/tol/assets/favicon/manifest.json?5bf6fec5b9 https://www.trulygeeky.com/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=749330624120 https://www.uninterrupted.com/d8182a65c526ea82b18d9b220253532e.json https://www.vcclub-vulkan.com/uploads/pwa/manifest.json?v=1491403533 https://www.verkehrsmittelvergleich.de/columbus-assets/favicons/manifest-8ccb0648b58c03d33b070004605eacdf015ff1e132f4a6968ee7560afa57fa4d.json https://www.visitjeju.net/_nuxt/manifest.1b1bbe02.json https://www.vivus.com.ar/manifest.json?v=1.0.0 https://www.vivus.com.mx/manifest.json?v=1.0.0 https://www.vivus.dk/assets/favicon/manifest.json?v=1 https://www.vivus.lt/assets/favicon/manifest.json?v=1 https://www.vivus.lv/assets/favicon/manifest.json?v=1 https://www.vivus.pl/assets/favicon/manifest.json?v=1 https://www.viz.com/favicon/manifest.json?v=47MPqANpyj https://www.volcano1.net/uploads/pwa/manifest.json?v=1491403533 https://www.vooks.net/img/fbrfg/manifest.json?v=GvJxPypopN https://www.vulcan-klub.online/uploads/pwa/manifest.json?v=1491403533 https://www.vulclub.org/uploads/pwa/manifest.json?v=1491403533 https://www.we-are.travel/manifest.json?v=wAA9BoEKk8 https://www.webtures.com.tr/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=530367467346 https://www.westland.ru/favicons/manifest.json?v=Ryok6lQ6d3 https://www.whatsnewonnetflix.com/assets/favicon/manifest-ae3f7e8a4cb2298a9194b985101464e2348753f0f7a5015e174040de72de5e7d.json https://www.winni.in/icons/manifest.json?v=m2leeBq7Gw https://www.xn--kndigen-n2a.de/assets/manifest-77b841cae31bf680c8decc4641f376c14861b0537025485bb42d4897c780fb89.json https://www.yaallah.in/wp-content/plugins/onesignal-free-web-push-notifications/sdk_files/manifest.json.php?gcm_sender_id=499381340670 https://www.zoomcatalog.com/manifest.json?v6.2 https://www2.elkedagietsleuks.nl/manifest.json?_=1532556244 https://wynajem.flixbus.pl/manifest.json?dcea861a57 https://zellbury.com/static/version1595174166/frontend/Scandiweb/pwa/en_US/Magento_Theme/assets/manifest_v2.json ```

[benfrancis]

@philloooo wrote:

I agree. The scope is a better choice if not considering change of expecations from one set of devs.

I also agree scope makes more sense conceptually than start URL. One other concern I have about scope though is that web apps may eventually need to support an array of scope URLs, so it could end up being a compound key which is less neat.

Oh thanks for pointing out, I just found out the announcement of them dropping the support recently. I will remove firefox desktop from the doc.

Firefox desktop has never supported installing web apps from a web app manifest. The recent decision to end the single site browser experiment doesn't change that, but it does remove one path to getting there.

I think this is misrepresenting what I was saying in the doc.

Sorry if my summary came across as a little cynical, I'm just a bit frustrated with how this has played out over the years.

I also tried to explain in detail that doing 301 redirect for use case of changing manifest_url either doesn't really support the use case, or ends up with unreliable behaviors for user agents.

I still think this could be workable. The worst case scenario is that users end up installing duplicate apps on the same device (if they don't remember they already installed it) or with different manifest URLs across different devices, which need de-duping as user agents follow redirects or synchronise with a server. I don't think it would be necessary for documents to always link to the original manifest URL, or for user agents to check all their manifests every time they encounter a new one.

Regarding the disadvantages you list for using processed manifest URL as a global ID...

• S2: installed apps on desktop, S3: Apps that are installed on both Android and desktop needs migration.

Why would Chrome for Android need migration if it already uses manifest URL?

• Change of implicit expectations for desktop WebApp developers

Whatever solution is chosen, the behaviour of some user agents (including at least one of Chrome desktop and Chrome for Android) will need to change

• S6: Apps that specify manifest as data URL will not be updatable

Agreed, but this already isn't possible on most user agents today (Chromium desktop being the exception). This seems like a reasonable tradeoff given it is inherent in the nature of data URLs. Developers can switch to an HTTP manifest URL if they want to start supporting updates, requiring a one-off re-installation.

• PWA stores will recognize different versions of manifest URLs as different web apps.

Depending on how PWA stores collect manifests (developer submission/user submission vs. web crawling) they may temporarily end up with duplicate records which need merging the next time a listing is updated. I agree this is the biggest disadvantage (and impacts sync services too), but I am of the view that developers should be discouraged from versioning their manifest URLs (like 92% of existing web apps), which would help reduce the impact.

• S4: Apps that need to update manifest URLs will need to keep supporting the previous manfiest_url, handle 301 to the latest manifest forever. The document will need to always use the original manifest_url. An extra roundtrip will always be needed for loading the app.

I agree with the first point (in theory), but disagree with the second two points. Documents should always be able to link to the latest manifest URL.

• If the manifest_url is hosted under a CDN, they will need to stick to that CDN provider.

I don't agree, they can redirect to a new one.

• Difficult to reconcile with S8: apps installed from A2HS, which does NOT require a manifest. ID creating here is weird, and instead would have to have custom upgrade handling for when a manifest is found on the page.

I think fake apps should be considered out of scope given they don't follow the specification anyway. Fake apps could use a generated chrome:// URL as a manifest URL. If a manifest is later found on the same page I would argue it is desirable for that to be treated as a separate application to a fake app.

Most of the user agents you are listed are chromium based.

Yes, only Firefox, Kai OS and Safari are still using something other than Blink 🙁.

Do you think it's safe to assume that all Chromium-based user agents behave the same way as Chrome on desktop and on Android? (Genuine question as outside of the implementations I've worked on myself I don't know). Due to the interplay between the browser engine, browser chrome, underlying operating system and sometimes app store and sync service, evaluating support for the Web App Manifest specification is more complicated than for other web specifications which are implemented entirely inside a browser engine. Chromium-based user agents could have different browser chrome which implements add to home functionality differently, or use their own sync service or app store (Webian Shell for example is built on a desktop build of Chromium but uses manifest URL to identify web apps). Operating systems may also try to update apps independently of the browser they were installed from (I'm not sure whether this is currently the case).

As I stated in the doc, safari doesn't support updating/looking up installed PWAs, so it doesn't add any use cases relevant to the unique ID problem.

I don't think it's fair to exclude Safari from the analysis on this basis, given the specification doesn't currently define how to update a manifest it can't be expected for user agents to already support this. As I understand it Safari on iOS does store metadata from web app manifests when adding web apps to the home screen, so what it uses as an identifier to store those data still seems relevant. It should be assumed that Safari on iOS may support updating web apps in the future, if and when that feature is defined in the specification.

As I understand it Firefox for Android doesn't currently support updating manifests either.

I've chatted with people who work on Firefox for Android and KaiOS and tried to encourage them to contribute to this discussion. From what I can tell, Firefox for Android currently uses start URL to identify web apps. The upcoming release of KaiOS uses manifest URL.

I think if you count the number of user agents (including all the Chromium based browsers on Android), there are probably more user agents which currently use manifest URL than start URL.

I'd be interested to know any use cases that I am missing in this doc.

The remote installation and link relation use cases I mentioned above aren't explicitly called out as use cases or requirements, but they are mentioned within a couple of the possible solutions.

Out of scope

• For a given id scheme, migrations between unique ids won’t be handled.
• Migrating between origins.

FWIW if these weren't out of scope, it might be possible to handle both use cases with manifest URL as ID, using redirects.

[benfrancis]

@dmurph wrote:

It seems like the main sticking point here is the 4th scenario - changing the manifest url. We know many sites do this, and crawling the data from https://pwa-directory.appspot.com shows us about 8.4% of manifests seem to have versions.

Thank you for sharing this list, it helps illustrate the problem well. Although this also means 92% of manifests don't version their manifest URLs, I acknowledge that it has created a real problem.

If chromium never supported a changing manifest url, then this scenario wouldn't be as important, but unfortunately it was supported and thus people now rely on it. Seems to be the story of the web platform, for better or worse :/

This is very frustrating to me, and a worrying sign of things to come for the web platform given Chromium's increasing dominance, but we are where we are.

The positives I see regarding Google's proposed solution are:

1. The id field in the manifest is optional
2. In practice all IDs are HTTP URLs, which is better than inventing a new non-web ID namespace (albeit URLs which don't resolve to a useful resource)

I'm wondering whether a better compromise between the best conceptual solution and the practical reality might be to add an id field as proposed, but make that id member an absolute URL and use processed manifest URL as the default. That would mean picking what Chrome for Android currently does as the default behaviour rather than what Chrome desktop does. It would allow start URL and scope to change without issues and allow developers who want to change the manifest URL to specify an ID inside the manifest in order to do so. Developers who don't want to change the manifest URL (but may want to change the start URL) can continue to use it as an identifier and don't need to use the optional id field at all.

A manifest with URL https://www.example.com/manifest.webmanifest but no id field would have the ID https://www.example.com/manifest.webmanifest

If the developer wants to change the manifest URL, they can add an id field to the manifest which matches the original manifest URL.

https://www.example.com/manifest.webmanifest?v=2

{
  "id": "https://www.example.com/manifest.webmanifest"
}

Apps would only be allowed to update using this approach if the origin of the manifest URL stays the same (switching origins could optionally be supported using HTTP redirects, which would happen far less often than the versioned URL use case).

Regarding the potential disadvantages:

S6: Manifests with data URL won’t be supported/updatable.

At first glance the approach above could work for data URLs too, by using the original data URL as an id inside the new data URL. Unfortunately because data URLs are treated as unique opaque origins that wouldn't be allowed, because otherwise any new manifest could claim to replace any existing manifest which uses a data URL.

This is the biggest downside, but seems like a reasonable tradeoff for that edge case given it is inherent to the nature of data URLs. Developers can always switch to an HTTP URL if they want to start supporting updates, requiring a one-off re-installation. Updating from a data URL is already not supported on any user agent except for Chromium desktop.

Change of implicit expectations for desktop WebApp developers

This is balanced out by the default behavior remaining the same for all those Chromium-based browsers on Android (and KaiOS).

S7: manifest is hosted under a different origin: could be a security concern to use a URL from different origin as the global key.

I would like to understand this better. What are the potential risks?

When S7: manifest is hosted under a different origin needs to change manifest_url, they can’t create an id field that matches previous manifest_url to keep the identity.

If the ID is an absolute URL rather than being resolved against the document URL, could this be possible (providing the new manifest URL has the same origin as the old one)?

S2: installed apps on desktop need to handle migration.

But all the Chromium-based browsers on Android don't.

S8: web apps that don’t have manifest URLs won’t be supported, they will have to use a different ID system and need to be migrated when they “upgrade”to have a manifest_url.

Is this about web pages which are added to the homescreen but don't have a manifest? It seems desirable to me that if a page starts linking to a manifest, the resultant app should be treated as separate to what is effectively a bookmark to a particular page within the app. Migrating from one to the other probably shouldn't be attempted.

[dmurph]

...

@benfrancis wrote: @philloooo wrote:

I also tried to explain in detail that doing 301 redirect for use case of changing manifest_url either doesn't really support the use case, or ends up with unreliable behaviors for user agents.

I still think this could be workable. The worst case scenario is that users end up installing duplicate apps on the same device (if they don't remember they already installed it) or with different manifest URLs across different devices, which need de-duping as user agents follow redirects or synchronise with a server. I don't think it would be necessary for documents to always link to the original manifest URL, or for user agents to check all their manifests every time they encounter a new one.

This is an edge case that is bad for users - we strongly want to avoid the case where duplicate apps would exist for a user, or the user would see an 'install' icon when that shouldn't be applicable (for this case & other cases). These types of issues are also really big for enterprise users who more often than not difficult-to-change infrastructure and need their user flows to go EXACTLY as expected. Weird things like install icons showing up or duplicate apps appearing.

@benfrancis wrote:

@philloooo wrote:

• Change of implicit expectations for desktop WebApp developers

Whatever solution is chosen, the behaviour of some user agents (including at least one of Chrome desktop and Chrome for Android) will need to change

Kind of - the no default global_id has the option of no migration, but then kind of specs an opt-in migration with the legacy ids.

@benfrancis

@philloooo write:

• S6: Apps that specify manifest as data URL will not be updatable

Agreed, but this already isn't possible on most user agents today (Chromium desktop being the exception). This seems like a reasonable tradeoff given it is inherent in the nature of data URLs. Developers can switch to an HTTP manifest URL if they want to start supporting updates, requiring a one-off re-installation.

It does break existing behavior that developers are relying upon. I would be interested in getting stats here too.

• PWA stores will recognize different versions of manifest URLs as different web apps.

Depending on how PWA stores collect manifests (developer submission/user submission vs. web crawling) they may temporarily end up with duplicate records which need merging the next time a listing is updated. I agree this is the biggest disadvantage (and impacts sync services too), but I am of the view that developers should be discouraged from versioning their manifest URLs (like 92% of existing web apps), which would help reduce the impact.

Same comment above - bad for users to have this happening, and it would happen, possibly with big properties.

@benfrancis wrote:

• If the manifest_url is hosted under a CDN, they will need to stick to that CDN provider.

I don't agree, they can redirect to a new one.

CDNs go out of business - it's dependency that people already rely on. Maybe it would never be an issue, but worth mentioning as a Con I think.

@benfrancis

As I stated in the doc, safari doesn't support updating/looking up installed PWAs, so it doesn't add any use cases relevant to the unique ID problem.

I don't think it's fair to exclude Safari from the analysis on this basis, given the specification doesn't currently define how to update a manifest it can't be expected for user agents to already support this. As I understand it Safari on iOS does store metadata from web app manifests when adding web apps to the home screen, so what it uses as an identifier to store those data still seems relevant. It should be assumed that Safari on iOS may support updating web apps in the future, if and when that feature is defined in the specification.

I'm guess I'm confused why you mentioned them then. Excited to help get that support prioritized though by figuring out this spec :)

@benfrancis wrote: I'm wondering whether a better compromise between the best conceptual solution and the practical reality might be to add an id field as proposed, but make that id member an absolute URL and use processed manifest URL as the default. That would mean picking what Chrome for Android currently does as the default behaviour rather than what Chrome desktop does. It would allow start URL and scope to change without issues and allow developers who want to change the manifest URL to specify an ID inside the manifest in order to do so. Developers who don't want to change the manifest URL (but may want to change the start URL) can continue to use it as an identifier and don't need to use the optional id field at all. ....skipping some text... If the ID is an absolute URL rather than being resolved against the document URL, could this be possible (providing the new manifest URL has the same origin as the old one)?

This is a good question - and I think this is actually the 2. global id = id (default processed manifest url). This would let someone make this manifest:

{
  ...
  id: 'https://www.google.com/manifest.json',
  start_url: "https://malware.com/step_3_profit",
  ...
}

I'm not sure how we would have this sort of malware takeover not happen. Let me know if you're thinking of something different here though.

I guess my perspective here is that, while not perfect, our proposed solution allows us to support all of the use cases, keeps all of the relevant information declaratively in the manifest & document (not needing to support redirects for edge cases, possibly forever), but sacrifices:

• explicit discoverability
• android webapp developer expectations (only option which doesn't change someone's expectations is the 'no default global id' one)
• Migration cost (which exists for someone for all options, but at least is a one-time migration).

which, since the expectations & migration cost would happen here no matter what somewhere, the sacrifice would be discoverability. I think this is solvable by indexes & crawling, so it seems like the best solution to me. I don't think it's acceptable for us to break data urls and manifest versioning, and with some of the 301 workarounds I don't see how we could support the use cases without nasty user agent edge cases (e.g. "should I show install icon" check would need to re-fetch all existing manifests for an origin to see if any of them now redirect to the manifest listed in the html page). But maybe I'm not seeing something here.

(Ironically, this still would mean that Chromium desktop would still need migration work due to our sync implementation not ignoring 'bad' ids - but that's not really relevant, for this doc, as we're mostly focused on the use cases here.)

[dmurph]

@ralphch0 wrote: One comment I had was regarding the default ID when the "id" field is missing:

Feels to me like if we were building this from scratch, we would likely go with scope? It's the same as service workers, and less likely to change. Sounds like the main reason start_url is preferred is because we would only be changing expectations for one set of devs (Android), but I wonder if long term scope will make more sense for future developers. Is start_url even that intuitive/expected by Desktop developers today?

start_url is how we currently allow updating, so I'm sure there is a lot of infrastructure that devs have in place that assume this case. But I don't have any concrete data here yet. @benfrancis noted that there is a request for having multiple scopes, which would make this probably a bad option if people need that.

@ralphch0 wrote: I assume here that regardless of what we choose, already installed apps will be migrated to the new ID scheme? (i.e. on the first chrome version that has the new code, the ID'ing will be migrated to the new scheme, and updates to the app will continue working by matching using the new ID scheme). IOW, there should be no technical diff (since I assume almost all apps out there don't use a dynamically changing scope), just a change in developers mental model that they may need to be aware of.

As much as possible, yes. Some solutions would not support existing installed apps, and IDK what we would do in those cases. I think you're right - developers on any platform where the behavior is changing would have to have a change in mental model, yes.

[wanderview]

@benfrancis noted that there is a request for having multiple scopes, which would make this probably a bad option if people need that.

The plan of record in service-worker-land is to default to scope when using the legacy single scope attribute and there is no id. If you are using the new-fangled-scope attribute that supports multiple scope values, etc, then you are required to set an explicit id.

[benfrancis]

This is a good question - and I think this is actually the 2. global id = id (default processed manifest url).

Yes sorry, that's the section I intended to link to. The difference being that id would be an absolute URL, rather than just a string which is concatenated with the start URL's origin.

This would let someone make this manifest:

{ ... id: 'https://www.google.com/manifest.json', start_url: "https://malware.com/step_3_profit", ... }

I'm not sure how we would have this sort of malware takeover not happen. Let me know if you're thinking of something different here though.

Ah, yes. That does look bad.

Just out of curiosity, and I'm sure I'm missing something here, but although this looks scary, what problems does it actually create? If the id URL is just an arbitrary URL used for identification purposes (which could be a manifest URL, or any other URL) and is never actually fetched or used for anything else, does it actually pose a security risk if a developer does something unexpected like use someone else's origin for their ID?

If a URL as an ID worked as I suggested and is only allowed to update an existing id-less app if the origin matches the original manifest URL, is it actually possible for malware.com to "take over" an app? I guess it is possible for the user to install malware.com's app first, which could then prevent them from later installing an app from google.com with a conflicting id? That does seem messy.

This does bring me back to an earlier point though - if id is a field inside the manifest what happens if two different web app manifests claim the same id? Is it not possible for user agents to get stuck in an update loop where they keep switching between two or more manifests linked from different web pages as the user happens to navigate to them? This is arguably another benefit of making the manifest URL the global identifier because that can't easily happen.

I guess my perspective here is that, while not perfect, our proposed solution allows us to support all of the use cases, keeps all of the relevant information declaratively in the manifest & document (not needing to support redirects for edge cases, possibly forever), but sacrifices:

• explicit discoverability
• android webapp developer expectations (only option which doesn't change someone's expectations is the 'no default global id' one)
• Migration cost (which exists for someone for all options, but at least is a one-time migration).

...

I don't think it's acceptable for us to break data urls and manifest versioning, and with some of the 301 workarounds I don't see how we could support the use cases without nasty user agent edge cases (e.g. "should I show install icon" check would need to re-fetch all existing manifests for an origin to see if any of them now redirect to the manifest listed in the html page).

Losing the linkability/explicit discoverability of web app manifests does feel like a missed opportunity, as does further tying manifest processing to a document URL which limits use cases for manifests outside of a web browser (app stores, app provisioning, kiosks etc. as discussed above).

But if we really have created a situation where it's no longer possible to use manifest URL as an identifier because identity was left undefined in the specification for so long, and it's not even possible to use manifest URL as a default, then I agree with @ralphch0 that scope URL might be the next least worst option.

I used the processed scope URL as an ID for storing web apps in a Firefox OS tablet prototype and it seemed to work OK, but it was just a prototype. The thing that concerned me about it was the potential for overlapping scopes between web apps, but that's probably fine as long as two web apps don't use the exact same scope (see the comment about multiple manifests claiming the same ID above).

@wanderview wrote:

The plan of record in service-worker-land is to default to scope when using the legacy single scope attribute and there is no id. If you are using the new-fangled-scope attribute that supports multiple scope values, etc, then you are required to set an explicit id.

That's interesting, and eases my concern about multiple scopes inside a manifest. Is there a draft that we can read anywhere? Aligning manifest with Service Worker in this respect seems attractive.

The advantages of using scope as a default listed in the explainer are also quite attractive and the disadvantages boil down to it being equally inconvenient for everyone!

[dmurph]

@benfrancis wrote:

@dmurph wrote: This would let someone make this manifest: { ... id: 'https://www.google.com/manifest.json', start_url: "https://malware.com/step_3_profit", ... } I'm not sure how we would have this sort of malware takeover not happen. Let me know if you're thinking of something different here though.

Ah, yes. That does look bad.

Just out of curiosity, and I'm sure I'm missing something here, but although this looks scary, what problems does it actually create? If the id URL is just an arbitrary URL used for identification purposes (which could be a manifest URL, or any other URL) and is never actually fetched or used for anything else, does it actually pose a security risk if a developer does something unexpected like use someone else's origin for their ID?

If a URL as an ID worked as I suggested and is only allowed to update an existing id-less app if the origin matches the original manifest URL, is it actually possible for malware.com to "take over" an app? I guess it is possible for the user to install malware.com's app first, which could then prevent them from later installing an app from google.com with a conflicting id? That does seem messy.

This would let malware 'take over' an app, as they would just set their id field to the manifest url. Even if we decided to 'verify' the start_url has a link to this new manifest, they could change the start_url in their bad manifest & it would look valid.

This does bring me back to an earlier point though - if id is a field inside the manifest what happens if two different web app manifests claim the same id? Is it not possible for user agents to get stuck in an update loop where they keep switching between two or more manifests linked from different web pages as the user happens to navigate to them? This is arguably another benefit of making the manifest URL the global identifier because that can't easily happen.

I think the real 'trusted' aspect here is the <link rel="manifest" href="...relative or absolute url here..."> part. This is, in a sense, the source of truth here, and I think we would treat the manifest linked from the start_url as the authoritative one. I think would have to be the same for all solutions here, even manifest_url - stores and browsers would verify that the start_url's manifest link actually links to the given manifest. If it doesn't match, then they would use the one in the rel-link.

@wanderview wrote:

The plan of record in service-worker-land is to default to scope when using the legacy single scope attribute and there is no id. If you are using the new-fangled-scope attribute that supports multiple scope values, etc, then you are required to set an explicit id.

That's interesting, and eases my concern about multiple scopes inside a manifest. Is there a draft that we can read anywhere? Aligning manifest with Service Worker in this respect seems attractive.

The advantages of using scope as a default listed in the explainer are also quite attractive and the disadvantages boil down to it being equally inconvenient for everyone!

Yeah, and most importantly here we would be breaking the expectations / infrastructure of all WebApp devs, as they are relying on either start_url or manifest_urlimplementations, instead of just one set of WebApp devs. Hard to say if it will continue to "just work" for most of them.

[dmurph]

Also, I see the id as becoming recommended, even to the point of us eventually requiring it for installability. Once everyone has an id set, then the world becomes a whole lot simpler. These defaults basically just help existing apps update to the new id world.

[philloooo]

This does bring me back to an earlier point though - if id is a field inside the manifest what happens if two different web app manifests claim the same id? Is it not possible for user agents to get stuck in an update loop where they keep switching between two or more manifests linked from different web pages as the user happens to navigate to them? This is arguably another benefit of making the manifest URL the global identifier because that can't easily happen.

That's the reason we force the id to prefix with the start_url's origin. So if within the same origin, they serve multiple manifests with the same id , yeah they will be recognized as the same app. But I think that should be expected by the app developers for that site as this is spelled out in the spec.

This does bring more work for PWA stores that do web scrawls without curation - they need to verify that the document of the start_url links to the manifest_url, if the manifest is at a different origin. I think this is the right thing to do though. Also I don't know which PWA store scrawl the web without any validation process right now.

Also, I see the id as becoming recommended, even to the point of us eventually requiring it for installability. Once everyone has an id set, then the world becomes a whole lot simpler. These defaults basically just help existing apps update to the new id world.

+1. I think with id added to manifest, this should be the recommended way. And the default is just a fallback mechanism that's optimized for backward compatibility. With scope it doesn't really give us backward compatibility benefit. With manifest_url, for cross-origin and data url cases, also doesn't allow backward compatibility.

[philloooo]

Do you think it's safe to assume that all Chromium-based user agents behave the same way as Chrome on desktop and on Android? (Genuine question as outside of the implementations I've worked on myself I don't know). Due to the interplay between the browser engine, browser chrome, underlying operating system and sometimes app store and sync service, evaluating support for the Web App Manifest specification is more complicated than for other web specifications which are implemented entirely inside a browser engine. Chromium-based user agents could have different browser chrome which implements add to home functionality differently, or use their own sync service or app store (Webian Shell for example is built on a desktop build of Chromium but uses manifest URL to identify web apps). Operating systems may also try to update apps independently of the browser they were installed from (I'm not sure whether this is currently the case).

The code for web app id is deeply embedded in Chromium in various places. Technically their forked version can change all the places to use a different id mechanism, but I will be surprised if they do that. Although I only know for sure Edge desktop doesn't deviate from Chromium for this feature.

I updated the explainer with Firefox Android behavior based on information from @NotWoods . They support updating manifest in newer firefox and android versions and use start_url as the id.

[benfrancis]

@dmurph wrote:

This would let malware 'take over' an app, as they would just set their id field to the manifest url.

Ah yes I see that's what I was missing, so an absolute URL wouldn't work.

@philloooo wrote:

That's the reason we force the id to prefix with the start_url's origin. So if within the same origin, they serve multiple manifests with the same id , yeah they will be recognized as the same app. But I think that should be expected by the app developers for that site as this is spelled out in the spec.

Yes I agree that at least contains the problem to a single origin.

The code for web app id is deeply embedded in Chromium in various places.

Which I imagine also makes changing the current behaviour to a different solution quite tricky.

I updated the explainer with Firefox Android behavior

Thank you! Don't forget KaiOS browser.

---

OK, well I came here to offer my feedback, based on experience of implementing the specification a few times, that there's no need to add an id member to the manifest because manifests already have a natural universal identifier on the web which makes them directly linkable and discoverable and which de-references to a useful resource. It's unfortunate that we've gone so many years with no identifier being defined in the specification that it appears the obvious solution is no longer practical.

If there's now no option but to define an id member inside the manifest, then my recommendation is that if manifest URL can not be a default then scope is the next least worst option in the long term. If that makes manifest consistent with service workers then that's even better.

I wouldn't expect using scope as a default identifier to cause much breakage for existing applications because it is probably the most stable of all the available options inside the manifest, but I don't have any data available to me to validate that hypothesis. Start URL has always been a dubious choice as an identifier for a web application as it's the most likely to change, so it's unfortunate in my opinion that the Chrome team chose that option in the absence of a recommendation in the specification.

I also wouldn't expect breaking "implicit expectations" of developers to be a big problem because in practice developers don't just target a single browser, they are already targeting a range of desktop and mobile browsers which behave inconsistently. It may even fix problems they were previously having with changing start URL or manifest URL, without any action on their part.

As an independent Invited Expert I can only offer my feedback, I don't have a large user base behind me to give that feedback much leverage. So as long as Mozilla, Apple and KaiOS Technologies are not participating in the discussion, I expect the Chrome team will do what they believe is best for their users, and developers.

Thank you @philloooo and @dmurph for taking the time to explain your rationale.