Module namespaces as thenables: Add non-normative spec text

[robpalme]

The conclusion to https://github.com/tc39/proposal-dynamic-import/issues/47 was that it is indeed intentional that the promise instance's final [[PromiseResult]] returned from import():

1. May not always be the module's namespace object
   • ...in the case where it has been "overridden" by an export named "then"
2. May vary on each call to import() for a given specifier
   • ...in the case where the "then"-named export is not idempotent
3. May not be accessible via the existing non-dynamic import syntax

Whilst this is well-specified intentional behavior, reviewers have seemed surprised by this discovery. The capabilities it allows were not mentioned in the original motivations or use-cases. Public educational pieces written on this proposal have not mentioned this capability. The spec wording for Runtime Semantics: HostImportModuleDynamically talks only about consistent results:

if the host environment takes the success path once for a given referencingScriptOrModule, specifier pair, it must always do so, and the same for the failure path"

To increase awareness of this behavior, I'd like to request a spec note be added to emphasize the fact that FinishDynamicImport's step 2.f means the caller of import() does not have the subjectively-intuitive equivalent guarantees of consistency.

As I am new to spec contributions, I will proposal some loosely suggested wording at the end of Runtime Semantics: HostImportModuleDynamically that I hope others may massively refine and reword:

NOTE 1: Even if the success path is consistently taken for a given referencingScriptOrModule, specifier pair, the module can still arbitrarily override the final [[PromiseState]] and [[PromiseResult]] of the promise instance returned by each import() call if the module exports a Promise executor function named "then". This is a natural consequence of FinishDynamicImport calling promiseCapability.[[Resolve]] with the module's [[Namespace]].

Please say if there is some existing policy on the conditions under which we allow/desire non-normative text to be added to the spec.

[domenic]

I don't think we should do this, because we'd end up doing it for every promise-returning API on the platform.

So for example, I'd like to see agreement to add such a note to e.g. IndexedDB before adding it here. (And yes, I know they are from different standards organizations, but I still feel that it's useful to apply criteria across the larger JavaScript ecosystem.)

[Pauan]

@domenic My understanding is that this note would only be needed if the API exports a then variable, so it shouldn't need the note on every Promise-returning API.

[domenic]

No, it'd be needed for any API that returns a promise for an arbitrary developer-controlled value.

[Pauan]

@domenic But the note is specifically about module resolution: it mentions referencingScriptOrModule, specifier pair and import()

Perhaps it would be useful to have a different note for other Promise-returning APIs (like IndexedDB), but this specific note is only for module resolution.

It's not an API specification, it's just an informative note, so consensus across all APIs is not needed.

If other specifications want to, they can add a similar note, but they don't need to. The note is still useful even if it only applies to import()

[domenic]

We're not at the stage of talking about specific note wording yet. We're discussing whether a note should exist at all.

[robpalme]

No, it'd be needed for any API that returns a promise for an arbitrary developer-controlled value.

I disagree. The reason import() is special and worthy of a special note is that users arrive with a strong expectation that importing modules is an idempotent action that provides an idempotent result. That's pretty much the module system's reason-to-exist.

In related material, https://github.com/tc39/ecma262/pull/916 makes it clear that browsers and Node prefer idempotent errors, citing various reasons.

It is crucial for browsers (and, so I've heard, for Node) that errors are cached so that telemetry can be used to identify repeatedly-encountered problems in a page.

So when the spec goes against the generally expected intuition of how a module system should work, I think it appropriate to highlight this in the spec.

[domenic]

Well, I guess we'll just have to agree to disagree then. As I've stated repeatedly, I don't think import() is any different from other promise-returning data-retrieval functions---all of which are idempotent unless faced with unusual thenables.

The related material you cite is fairly orthogonal; it's about the module system itself, and the impact of different states of the graph on future parsing/instantiation/evaluation. It's not about a function like import() which is called by a developer.

If you think import() is truly special in this manner compared to e.g. IndexedDB, then I think we are at an impasse, as I don't think so, and no new arguments have been presented that have worked to change my position. I suggest we close this issue, in that case.

[ljharb]

I'm confused; in IndexedDB, you can extract things that have a "then" function? I thought it was JSON.

[Jamesernator]

IndexedDB is structured clone so it doesn't apply to that, although I understand the general argument.

However this is actually the first (in ECMA262) Promise-returning construct where you can't box the object before it gets resolved into a Promise (programmatically) so it's probably at least worth having a note that typical strategies of "boxing" a Promise will have to be done before importing.

It's pretty likely .thenable modules will be fairly rare so it's almost certainly easy enough to inject an intermediate module that boxes it:

// --- Injected module
import isBoxed from "./isBoxedSymbol.js"
import * as thenableModule from "./thenableModule.js"

export default {
    [isBoxed]: true,
    module: thenableModule
}

// ---- Importer
import isBoxed from "./isBoxedSymbol.js"

// Get a boxed module regardless of whether the module is thenable or not
import('./injectedModule.js').then(obj => {
    if (!obj[isBoxed]) {
        return { [isBoxed]: true, module: obj }
    } else {
        return obj
    }
}

While this solution does mean adding another virtually identical file for every possibly thenable module, under the assumption thenable modules in practice will be quite rare I don't think it's a terrible workaround.

[robpalme]

I'm going to close this on the basis that there are other points in the spec that re-resolve and cause plain objects to be assimilated as thenables. See await and this thread. Likewise there are pre-existing cases of synchronous magic export names properties: toString, valueOf. I regard all these "features" as hazards in the context of module namespace objects, and it would be nice to have a safety-hatch/strict-mode to disable them - but that would be a separate proposal.

So I'm going to interpret dynamic import as being a new point of exposure of what was a pre-existing hazard. @Jamesernator demonstrates a build-time protection against this hazard. For now, the burden will be on JS educators and users to learn about this hazard.

Interestingly it sounds like IndexDB is special, but that's just a technicality.

[Jamesernator]

@robpalme When I said IndexedDB is structured clone so it doesn't apply I don't mean recursive thenable assimilation doesn't apply, it's simply not possible to store a function object using structured clone so I'm not sure what the points about IndexedDB that were mentioned were intending to mean. I'd expect that if IndexedDB could store functions (whatever it would mean to store a function in a database) it would necessarily be subject to recursive thenable assimilation too.

[robpalme]

@Jamesernator Yes, I think you were clear. IndexDB was discussed because @domenic wanted to know why dynamic import should be regarded differently to IndexDB. And it turns out that IndexDB is different; this hazard cannot arise in that context. But, big picture, I'm saying that doesn't matter because there are non-IndexDB cases in the spec. So let's not discuss IndexDB anymore ;-)

[ljharb]

ModuleRecords aren't just plain objects; I think this should be reopened and discussed at the September TC39 meeting.

[robpalme]

@ljharb I sympathize and think that a proposal to prevent module namespace objects being treated as thenables would have merit, but I don't see why dynamic import is the place to do that. The hazard equally applies to calling Promise.resolve() on a namespace.

[ljharb]

It might not be the place; but such a proposal would have to block this one to ensure that it could land without being a breaking change.

ModuleRecords aren't namespaces; they're conceptually different.

[ljharb]

(Assuming such a proposal was the direction the committee decided to go, of course)

[domenic]

A Module record is a spec concept that is never exposed.

All that is ever exposed is the namespace object.

Changing its behavior would already be a breaking change (Promise.resolve(ns)) so indeed is orthogonal to this proposal.

[ljharb]

That's a fair point. However, I think the ergonomics and expectations of import() are such that it's worth discussing as part of this proposal, even if the direction ends up being "not blocking, separate proposal".

[domenic]

I'd rather such a discussion not be related to import (), because it is not really related.