Proposal: Pollyfill hints

[guybedford]

This outlines a proposal for polyfill mode to work around the recent Chrome change that stops the fetch cache from sharing the module network cache.

To provide some background on how es-module-shims works in polyfill mode - when running in polyfill mode all module scripts in HTML are fetched and their sources analyzed for what modules features they use. Per the readme these feature detections include:

• import maps
• dynamic import
• import.meta.url
• import.meta.resolve
• JSON imports

If a feature is being used that is not supported in the current browser, then we know the module would not have exected correctly and we then initiate the graph reexecution.

Now ideally when running in modern browsers we could skip all analysis entirely and just allow native execution.

In order to achieve this, we could support feature attributes on module scripts. For example:

<script type="module" src="/app.js" polyfill="importmap,json"></script>

Where the above would read as: "if running in a browser that does not support import maps or JSON imports, run this module through the polyfill, otherwise skip processing it entirely and don't attempt to fetch the sources".

Polyfill attributes would likely include for now just "importmap" and "json". And soon "css" most likely.

In Chrome 91/92 that don't support the given feature (say when using "css"), there would still be a double network hit due to the lack of cache sharing existing in these later Chrome versions. But at least such behaviour would be restricted to the polyfill fallback path.

[dhh]

This seems reasonable to me. It would presumably also allow a faster exit of the polyfill, if you're just going to check for a specific feature or list of features, and if they're natively supported, you can abort any polyfill logic, yeah?

[sashafirsov]

Crossposting as hints could be used not as polyfill parameters but as part of transformation descriptors

the page and particular module should have ability to control ( or point to external descriptor ) the content-type driven transformation. The app developer would pick or define the way this transformation happen. And consumer JS or HTML would be written according to defined transformation chain.

https://github.com/WICG/webcomponents/issues/645#issuecomment-898968440

In the html imports thread proposed the concept of gradual transformation for modules of virtually any kind in unified way. Not just json or importmaps, css. The hints could serve the type of module transformation processor. As such it is better to reflect the content-type of given resource and perhaps declare the intent of returned type.

<!-- define transformation utility with input and output types -->
<script type="module" src="/imoprtmaps.polyfill.js" process="application/importmap+json,application/json" produce="module/importmaps"></script>

<!-- define what transformation steps entries to be applied -->
<script type="module" src="/app.js" transform="module/importmap,module/html,module/json"></script>

The difference is little in syntax and applicability. Rather declaring what polyfill features to be enforced by polifill, it would describe what processing steps to be done. The step would use the content-type as "feature". Transformer routine could be as polyfill, as external script, as embedded in browser routine.

Since the transfromers definition is separated from consuming and could be served in own descriptor, it could vary for different environments and over time substitute the coded with native implementation without changing the consuming script call.

Polyfills in such implementation have to be replaced with native implementation when it is available. The transform module( and polyfill) perhaps could be ignored when native implementation available.

That could be done by changing the type=module on transform script to type=transform or something similar. Of course we would need "module transformer" proposal and polyfill 😜

[guybedford]

@sashafirsov thanks for the input here. For what it is worth, es-module-shims is a module transformer. It even has the ability to directly hook source resolution or full JS rewriting as described in the readme via the fetch and resolve hooks - https://github.com/guybedford/es-module-shims#resolve-hook, https://github.com/guybedford/es-module-shims#fetch-hook.

For both of those hooks it is possible to define pipelines that selectively perform transforms based on different environment conditions. And I do plan for the future of this project to eg support polyfilling other features such as top-level await, module blocks and module fragments in future. The types of transforms done are still constrained to the set of transforms that work on top of the lexer with simple string concatenations of the source and without requiring a full parser due to the performance constraint of the project.

The polyfill attributes are thus more about relating feature detections to the boolean question of whether or not to run the whole pipeline for a given top-level module script. This requires users to know if they are eg relying on a given feature, but by also making it an opt-in that works without being used (albeit with worse performance), we hopefully maintain a good UX.

This makes me think - maybe we should provide a warning in the console when using the polyfill mode in development when a module script is missing polyfill hints that would improve performance.

All of that to say I think I'm going to go ahead and implement this proposal as-is. I hope that the pipeline hooks demonstrate the existing extensibility of this project to your use cases (and note that these hooks are "composable" in that you can transform a dependency and the parent modules automatically rewrite their references to the transformed blobs, so this is actually exactly the right base for transform extensions and isn't a hack. I've actually been surprised no one has explored transformations on top of this project further to be honest.).

Happy to continue discussion...

[guybedford]

I ended up implementing this as:

<script type="module" skip-shim="import-maps" src="/app.js"></script>

where in any browser that supports import maps, the script is then entirely ignored instead of analyzing the sources to check for CSS modules / JSON modules support.

PR merged at https://github.com/guybedford/es-module-shims/pull/171. I plan to release this soon, so any feedback before release is welcome.

[guybedford]

The approach in #171 felt a little too unintuitive, so instead I went with something closer to the concept of feature flags in https://github.com/guybedford/es-module-shims/pull/172.