Closed jonathantneal closed 1 year ago
I think the media
example is really compelling! Whether it fits into import reflection or not I think the use case should be strongly considered within the realm of imports in general, and it's probably a good idea to consider how it intersects with reflection and assertions and whether we want potentially three related import features: assertions, reflection, and something else (attributes?).
As CSS module scripts are a way for a JS module to depend directly on CSS, it somewhat takes the place of <link>
but embedded in the module graph rather than in the top-level HTML. However imports are missing some features of <link>
notably the media
attribute.
Adding a media
attribute would enable a form of conditional imports that close the gap with <link>
. I'm not sure where else conditional static imports might have been discussed, but I could see something like this:
import * as desktopStyles from './desktop.css' assert { type: 'css'} with { media: '(width > 640px)' };
import * as mobileStyles from './mobile.css' assert { type: 'css'} with { media: '(width <= 640px)' };
if (desktopStyles !== undefined) {
document.adoptesStyleSheets.push(desktopStyles.default);
}
if (mobileStyles !== undefined) {
document.adoptesStyleSheets.push(mobileStyles.default);
}
Other potential examples in some distant future might include:
import boldOpenSans from './OpenSans-Bold.woff' assert { type: 'font' } with { weight: 700 }
await boldOpenSans.load()
import bgm from './background-music.flac' assert { type: 'audio' } with { loop: true }
bgm.addEventListener('canplaythrough', () => {
bgm.play()
})
import myWorker from './my-worker.js' with { type: 'worker' }
myWorker.postMessage({ cmd: 'start', msg: 'Hi' })
I think it would be great if a more extensible syntax were used rather than requiring a parser change each time a new attribute is introduced. There are a lot of potential use cases for import attributes, and I don't think it always necessarily makes sense for each one to go through the TC39 process since they won't be implemented by all engines (e.g. some attributes might not make sense in browsers). I raised this back on the import assertions proposal as well, and a number of good use cases came up there: https://github.com/tc39/proposal-import-assertions/issues/99.
For example, one thing that would be cool is support for preload/prefetch attributes. For example, something like this:
import foo from "./foo.js" with { load: "prefetch" };
// some time later...
const exports = await foo.get()
// or maybe
const exports = await import(foo);
Somewhat related: #16.
@devongovett I wonder if a usecase like preload might not be better suited to a dedicated import.preload
function in due course over attributes. Are there specific reasons you'd want attributes over a dynamic mechanism here?
Yeah might also be useful for some cases, e.g. on-demand preloading when a user hovers over a link.
I think one benefit of attributes is that they are much more easily statically analyzed. This not only benefits build tools, but could potentially benefit browsers and other runtimes as well. A quick pass could be done to determine what to preload without parsing and evaluating the entire module.
Btw, this was actually @littledan's idea, and there was some prior discussion about it here.
Another potential use case is lazy loading, i.e. subsuming https://github.com/tc39/proposal-defer-import-eval.
import.preload(specifier)
would be just as statically analyzeable as import(specifier)
, as well as lintable (if you want to restrict it to a static specifier), no?
Yeah, but you have to evaluate the code to know when to preload, whereas a declarative import
statement allows immediate preloading without fully parsing or running any code. Browser engineers could tell us if that's useful, but seems like it could be (esp if it supported non-JS resources, i.e. subsuming asset references - #16).
I also like the idea that it could be done the same way either as an import statement or a dynamic import.
import foo from "./foo.js" with { load: "prefetch" };
import('./foo.js', {with: {load: 'prefetch' }});
Given that import.preload
would be syntactic, it would show up on a first parse (which engines have to do anyways to hoist import statements), so it doesn't seem like there'd be a need to run any code if the argument is a static string.
<link rel=modulepreload>
is the early hint for browsers, and is preferable to discovered preloads (which involves latency) for the static case.
Sure, for initial page loads but would be nice to have a way to preload things for later loaded things other than insert one of those into the document manually.
Anyway, this is all kinda off topic. I am mainly interested in having a syntax that is forward compatible with future additions, as well as extensible for build tools to innovate in this space without waiting for TC39.
Sure, for initial page loads but would be nice to have a way to preload things for later loaded things other than insert one of those into the document manually.
This is a great discussion and one I would love to have further.
Anyway, this is all kinda off topic. I am mainly interested in having a syntax that is forward compatible with future additions, as well as extensible for build tools to innovate in this space without waiting for TC39.
I don't think it's at all off-topic, we should discuss features in the context of use cases. The more use case discussions the better.
I don't think it's at all off-topic, we should discuss features in the context of use cases. The more use case discussions the better.
I appreciate the optimism, and what you write is true, but it’s also incomplete. For me, a single extensible API is important because 1. I’m concerned TC39 folks might not understand the needs of source file tooling where I want to innovate, and because 2. a single extensible API can be delivered in a more timely manner than several divergent APIs.
I see a not insignificant number of features that would require different keywords to accomplish what we want from one.
Still, here’s another 'transform' use case I was reviewing, borrowing Justin’s syntax:
import images from './profile.jpg' as 'image-set' with { size: [ 320, 480, 640 ], type: [ 'avif', 'webp' ] }
I think that there seem to be two related but distinct use-cases that are floating around in this discussion. I believe that they may receive better attention if they were explicitly called out.
In the spirit of the types-as-comments spec, perhaps it would be worthwhile to think about some explicit namespace or mechanism for code authors to pass expressive intent to the tooling ecosystem. If that was a first-class concept then we might find ourselves avoiding situations where semantic concepts like import specifiers or assertions are being used in unintended ways to achieve important and necessary goals.
I agree that work should go towards a general, object-based syntax rather than a single string. Many other use cases have been discussed already, in the import assertions repo, when folks were arguing that it shouldn't be limited to assertions. On the other hand, I am unconvinced that the functionality exposed in this proposal is all that high priority. Rather than as for the keyword, I would prefer something which implies that the module will be changed, e.g. with.
After seeing the presentation at TC39 and talking more with @guybedford , I'm convinced that this proposal meets an important use case; I've striked out part of my comment above.
While I would prefer a general syntax exposing other parameters, but there's a legitimate concern that this would be blocked in committee, as @ljharb previously did. I don't think it's worth stopping this proposal in its tracks for such a broader generalization, though the generalization would be my preferred outcome. Overall, I agree with @jridgewell 's comment that we probably made an error in import assertions using the "assert" keyword and should've been more general in the first place (however, this preference contradicts the arguments that @ljharb and @devsnek previously made, and does not have TC39 consensus; in any case, it's too late to make changes in that proposal).
If we don't go with a generalized key-value pair, I'd suggest that we use some other syntax besides as
, for all the reasons that as
was rejected in the import assertions case. If we're going for something specific, let's be fully specific with a keyword indicating the change in mode of importing the module. So, for example, import reflective
and import asset
.
For now, I'd like to focus on working out the details of this proposal, especially how it relates to module blocks, compartments, module fragments, Wasm/ESM integration, Wasm components, etc. It's good to have this syntax debate running in the background, but let's not get too bogged down on it; I am confident that we'll be able to find some syntax or other which is agreeable, and we have a lot of other details to work out.
Thanks @littledan for clarifying here, we are open to using an alternative syntax to as
and would like to explore these options further, in a way that can work with and leave the door open for evaluator attributes syntax. From our discussion I'm also confident we can find a suitable approach.
Agreed the primary proposal details to work out right now though are how a user-exposed JS module record gets specified between these proposals, and what other cross-cutting concerns apply. It definitely makes sense to continue to focus on that for now.
I'm disappointed to see the syntax has been changed to yet again use a static keyword in yet another position rather than an extensible syntax. Just like I raised for import assertions (https://github.com/tc39/proposal-import-assertions/issues/99), the syntax is not symmetric between dynamic and static imports. Dynamic imports use an extensible object syntax, whereas static imports do not. From the readme:
import asset x from "<specifier>";
await import("<specifier>", { reflect: "asset" });
If the import assertions proposal had used an extensible object syntax as raised then and in this issue, this proposal wouldn't even have been necessary. We could simply do this:
import x from '<specifier>' with { reflect: 'asset' };
This syntax would allow engines to add new attributes where it makes sense for them. Again, it's already the case with the second parameter to dynamic import, just not static import, and I really don't understand why. How long are we going to keep adding attributes to the language one by one? Why does each attribute need to be in a different part of the syntax?
Lots of tools and developers want to use import attributes of some kind, for purposes beyond just the ones specified. Many use cases are covered here and in the other issue linked above. Some tools have already started abusing import assertions for this, which is bad. In my strong opinion, this needs to be solved once and for all, and not by adding new attributes one by one every few years. Progress is too slow this way, and it doesn't leave enough room for tools and engines to innovate in their respective domains.
@devongovett tools that have abused import assertions for this sort of thing are likely violating the spec; while the spec has no enforcement power, it's very important that intentions be explicitly conveyed. "of some kind" is frighteningly vague, and I'd love to hear more concrete use cases if the existing proposals don't address them.
Here are two examples I saw recently where it happened. Not sure if either of them ended up shipping because it was called out, but still. The demand for more extensibility is there.
Further discussion on both threads shows that neither of those tools were brazen enough to blatantly violate the spec - it remains a critical gate for the ecosystem.
The spec restriction that forbids import assertions from altering the interpretation of the module is completely arbitrary and should be gleefully ignored. Allow the bundler ecosystem to experiment with the syntax space to better support their customizability.
@ljharb What is your opposition to extensibility at a syntax level? People have been raising this for years, and every time it just goes nowhere. Clear demand and use cases have been documented and discussed. Every time it's raised someone punts anything that isn't their exact problem to some other future proposal. When are we gonna solve this? What is the actual technical reason why we can't solve it once and for all rather than blocking all future features for module loading and evaluation on TC39 adding yet another new syntax?
The semantics of specific keys in an object syntax could be specified just as they are today for dynamic import. I see no difference between import asset x from '...'
and import x from '...' with { reflect: 'asset' }
from a semantic point of view, but the latter allows additional keys, as well as new values for the reflect
key to be added without changing the parser. How is this not better?
I feel like having a relaxed syntax for import statements could really help the community as a whole: A lot of different bundlers have different and custom syntaxes to be able to apply custom process pipelines to files:
loader!path
loader:path
./asset.js?url
Even though all of those don't have actual meaning during the runtime, having an extensible spec could allow for all the bundlers to give the possibility to follow a similar syntax and reduce the gap between all bundlers (and also allow to make bundlers feel like more align with the spec)
I opened in the past an issue to add the possibility in parcel to support this (see https://github.com/parcel-bundler/parcel/issues/7648) but @devongovett rejected this idea as it is out of scope (syntax assertions aren't supposed to be used for transformations).
The semantics of specific keys in an object syntax could be specified just as they are today for dynamic import. I see no difference between import asset x from '...' and import x from '...' with { reflect: 'asset' } from a semantic point of view, but the latter allows additional keys, as well as new values for the reflect key to be added without changing the parser.
If the concern is about future collisions between user-land metadata keys and keys having semantic meaning, then let's get ahead of the problem. Give us a metadata key or key prefix that is reserved for non-semantic purposes.
Tooling could then freely strip this syntax during optimization passes but runtimes would still be able to consume it as-is.
Reserved prefix:
import Article, { metadata } from './path/to/Article.mdx' with { 'x-loaders': ['mdx'] };
Would be symmetric to dynamic import:
const { default: Article, metadata } = await import('./path/to/Article.mdx', { 'x-loaders': ['mdx'] });
Reserved key:
import Article, { metadata } from './path/to/Article.mdx' with { extra: { loaders: ['mdx'] } };
Would also be symmetric to dynamic import:
const { default: Article, metadata } = await import('./path/to/Article.mdx', { extra: { loaders: ['mdx'] } });
@jridgewell it's the entire reason the feature is allowed to exist. If such things are "gleefully ignored", then that will just ensure that future dangerous features, including this one, never advance. It's also disheartening for a TC39 delegate to be publicly advocating willfully violating the spec, and comes across as very bad-faith behavior.
Things can experiment with syntax all they want - it just makes them noncompliant.
@devongovett You can think it's better all you want, but unrestrained syntax experimentation is a net detriment for the ecosystem. We all benefit from interoperability, and that means that semantics have to be tightly specified and not wildly host-defined.
Tools can do whatever they want within specifiers already - that's the space for innovation. You don't need permanent, never-removable, expensive-to-implement syntax to test things out.
You can think it's better all you want, but unrestrained syntax experimentation is a net detriment for the ecosystem. We all benefit from interoperability, and that means that semantics have to be tightly specified and not wildly host-defined.
That's incorrect, the entire motivation behind import.meta
is to be a random grab-bag of metadata which can be host or bundler defined. Having extensible metadata is a good and useful thing, as has been repeatedly proven both in the JS ecosystem and in other language's ecosystems.
@Pauan that's for metadata for a module author, which is quite distinct from a module importer, by design.
@ljharb And? That doesn't make any difference, it's useful in both cases, you're trying to make a distinction which doesn't exist. People have provided multiple use cases for extensible properties on imports.
And in both the case of import.meta
and import x from '...' with { foo: 'bar' }
the behavior is specified by the bundler or host. So all of your arguments apply equally to import.meta
.
It makes a huge difference. A module shouldn't behave differently based on who is importing it - metadata should come FROM a module or be provided to it by a host, not be passed to it by a consumer.
You can think it's better all you want, but unrestrained syntax experimentation is a net detriment for the ecosystem.
That is the entire point of making it extensible - so that we don't have to change the syntax in order to experiment or add future standard features. As stated before, the goal of interoperability is not hindered by an extensible syntax. Semantics can be specified based on keys within an object, just as they are for dynamic import already. Why do you think static imports are different?
i think extensibility is good but i am also concerned about attributes which would fork a module into multiple instances. imo those should have to be part of the specifier (or to think about it another way, these module attributes should not be considered when keying a cache). this sort of leaves my opinion in a deadlock... i'll try to think about this some more but just wanted to mention it.
i think extensibility is good but i am also concerned about attributes which would fork a module into multiple instances. imo those should have to be part of the specifier
@devsnek I think you make a good point and perhaps the first compelling point against arbitrary evaluator attributes.
If there was a reserved space for non-semantic evaluator attributes, then I think the spec would have to be clear that these would have absolutely no impact on module identity. It would follow that tooling would therefore have to manipulate the import specifier in generated code if it wanted to allow different attributes to produce different outcomes.
The important part, I think, is that it would give developers a spec-compliant place to signal rich intent to their tooling. The kind of expressiveness that translates very poorly to url query overloads. Tools have been making great use of import.meta
as a namespace for extensibility and I think this would have a comparable impact.
i think extensibility is good but i am also concerned about attributes which would fork a module into multiple instances.
That's a good reason to define the semantics of specific keys, but doesn't explain why it must be done with a keyword. Clearly, these have different semantics:
import('specifier');
import('specifier', {reflect: 'module'});
import('specifier', {reflect: 'asset'});
Therefore, reflect
is part of the cache key. On the other hand, assert
is not. Other keys that are defined besides reflect
and assert
are ignored by the engine. All of this is part of the spec.
Why can't the same be done for import statements?
import foo from 'specifier';
import foo from 'specifier' with {reflect: 'module'};
import foo from 'specifier' with {reflect: 'asset'};
My point is, the syntax can be extensible, while still defining the semantics as you have been. As it is, the syntax for dynamic import already allows this, so I don't see a reason why it cannot also be done for import statements.
If such things are "gleefully ignored", then that will just ensure that future dangerous features, including this one, never advance.
I should have used willful violation. The restriction is unnecessary for bundlers, and forces their users into a considerably worse hack. If we actually cared enough to make it non-dangerous, we would specify an extension space as https://github.com/tc39/proposal-import-reflection/issues/18#issuecomment-1189690226 nicely explains.
It's also disheartening for a TC39 delegate to be publicly advocating willfully violating the spec, and comes across as very bad-faith behavior.
Again, it's a completely unnecessary runtime restriction. Should we specify that you're forbidden from monkey patching the globals prototypes, too?
Things can experiment with syntax all they want - it just makes them noncompliant.
Sure, I never argued that they should be considered valid production code. Shipping an unsupported import assertion to production should be discouraged, but having the bundler transform it beforehand is fine. Else, we should remove every non-stage-4 feature from every codebase, because they're not valid syntax either.
You can think it's better all you want, but unrestrained syntax experimentation is a net detriment for the ecosystem. We all benefit from interoperability, and that means that semantics have to be tightly specified and not wildly host-defined.
We're already hurting from the restriction. We have to invent an entirely new syntax to support import reflection, because we don't want to allow the reflection to be specified in the assertion. Of course having it inside an assert {}
block is weird, but we should have seen this coming and stuck with as {}
.
My point is, the syntax can be extensible, while still defining the semantics as you have been. As it is, the syntax for dynamic import already allows this, so I don't see a reason why it cannot also be done for import statements.
This is my sentiment exactly.
@devsnek Note that modules are already modified based on who imports them. The identity (and caching) of modules already takes into account more than just the import path, and therefore modules can already be forked even if they have the same import path.
@ljharb Sorry, that's already how modules behave. ES6 modules have always been specified to behave differently based on who imports them. That's very unlikely to change.
@Pauan yep i'm aware of module caching semantics. for the purposes of the spec i scope my concerns to imports within the same file. there is some prior art here as well in the assert
proposal.
@devsnek Even within the same module, if you import it differently then it will "fork" it (see @devongovett 's comment). There are several use cases for forking modules. For example, you already know that bundlers want to import Wasm modules, not instances. Putting a hard "no forking" policy will hurt the JS ecosystem and cut off many important use cases.
reflect does not fork it to my knowledge. its returning a different attribute of the same cache entry (the module vs the module's namespace)
@Pauan the cache isn’t part of the spec and isn’t the module; and the concern isn’t duplicating it; it’s altering it. It’s fine to have a million conceptually identical instances from the same specifier; it’s not acceptable to have even two conceptually distinct instances from the same specifier.
@ljharb You're simply incorrect. Go read the thread I linked to.
The cache is a part of the ES6 spec, and it is specified based on an (importingModule, specifier)
tuple. Which means that it's possible for the same specifier to return a completely different module.
Which is why import { currentScript, url } from "js:context";
was discussed as a possible alternative to import.meta.url
, because the "js:context"
module can be different every time it is imported.
And if desired, it would be possible to spec it so that the cache is based on an (importingModule, specifier, metadata)
tuple, so that way modules with different metadata can resolve to different modules.
ES6 modules have never been cached only based on their specifier, it has always been possible to have completely different modules with the same specifier. Your intuition is simply incorrect.
Even in NodeJS the same specifier can result in completely different modules being imported. For example import foo from "foo";
can import different modules (because there can be multiple versions of the package foo
).
That is a fair point. None the less, you can’t import the same specifier twice in the same file and get different modules, except in cases of pathological server behavior with a remote module.
Hi all, I just want to bring up a concern I have with media queries in particular. How do people interpret this code snippet?
import JoyStyle from "./joy.css" as { type: "css", media: "(width > 640px)" }
There’s a number of interpretations I see:
"(width > 640px)"
whenever the viewport changes (say if the user changes the width of their window). Does mean the entire module is evaluated again, because the following logic might have changed?I’m curious what @jonathantneal was intending.
From @justinfagnani example here it seems like A is the interpretation:
import * as desktopStyles from './desktop.css' assert { type: 'css'} with { media: '(width > 640px)' };
import * as mobileStyles from './mobile.css' assert { type: 'css'} with { media: '(width <= 640px)' };
if (desktopStyles !== undefined) {
document.adoptesStyleSheets.push(desktopStyles.default);
}
if (mobileStyles !== undefined) {
document.adoptesStyleSheets.push(mobileStyles.default);
}
The problem is that this is evaluating it only at the initial load time is that the window might be resized (say on a laptop or tablet), and now the other styles should be applied but they since they were initially undefined the page doesn’t work as expected. Which means the user would have to reload the page.
<link>
tags today don’t have this problem, they are each independent and don’t affect each other directly. Which means they can later load if say the viewport changes in size.
You've hit the nail on the head that media queries are closer to conditional loading than a loading attribute.
The recommended pattern today would probably be the following:
await import(matchMedia('(width > 640px)') ? './desktop.css' : './mobile.css', { assert: { 'type': 'css' });
Moving this from a dynamic import into a static import you'd then effectively want the static import to reify the media query into something like a conditional export mechanism.
There are a couple of ways to do conditional exports statically:
Note that a conditional import system that nulls its imports on predicate results basically means a DSL or out-of-band configuration system. Hard-coding media queries is fine, but it's difficult to see how that generalizes easily at all given we don't have many other similar DSLs on the web,
It's probably worth addressing the merits of the first dynamic import example, and (1) above as well before considering such an option.
Yes I agree we should be more eager to use dynamic imports — especially if what we are doing is actually dynamic.
With your example, how would you handle reevaluating the media query when the viewport changed? This is still only “evaluate the media query at initialisation”:
await import(matchMedia('(width > 640px)') ? './desktop.css' : './mobile.css', { assert: { 'type': 'css' } });
The problems with dynamic import are the poor behavior of top-level await wrt to loading, and the likely incompatibility with prescanners.
TLA use will block importing modules from evaluating, which will delay their similar TLA+dynamic-imports, and so-on up to the root of the module graph.
Prescanners are very unlikely to understand the conditional logic and so won't be able to preload any resources. Being a built-in feature would enable preloading.
That is a very good question about what's expected to happen when media queries change. Answering that would be necessary, and some re-evaluation capability would be needed to match HTML.
It seems important to question which part of the code's lifecycle is targeted by the proposed syntax. It does seem like there are some compelling ways the intent could be encoded in today's primitives.
But what I find interesting is that it is very difficult to execute on that intent for the average dev. Doing so involves a depth of understanding that few people in the world hold. However, the proposed syntax and it's intent could be transformed by tooling into the right primitives if such syntax were available.
This is where I believe that the power of an extensible syntax lies, especially when it has no runtime semantics.
We already have infinitely extensible syntax with no runtime semantics - comments. Adding syntax that does nothing at runtime is adding a lot of cost for decidedly non universal value.
Still no one has answered why it's an object literal for dynamic import (in which unknown keys have no runtime semantics) but not import statements. Why can't the syntax be the same? Import assertions already define such a syntax, so clearly it's possible.
@devongovett because it can’t be any other way for dynamic imports (because JavaScript), and there was opposition to having to type the extra boilerplate every time for the syntax case.
I'm concerned that every time there's a new usecase for an import attribute, a new bespoke syntax will need to be invented making the import statement more and more complex, and slowing down the standards process. IMO a syntax more similar to import assertions would keep reflection in import statements both more consistent with dynamic import and with import assertions, and also more extensible for future use cases (which might not make sense for TC39 to standardize, but perhaps another standards body).
Would you be open to using an extensible key/value format, to future-proof reflection in case it's needed for other use cases, even though none are implemented now?