introducing the wrapped module namespace exotic objects

[caridy]

Description

This PR introduces the concept of a wrapped module namespace exotic object.

Goals

1. To provide a more ergonomic way to interact with the ShadowRealm instance.
2. To resolve #364 (missing capability)
3. To resolve #353 (ergonomic of module resolution errors)

Details

A wrapped module namespace exotic object is very similar to a module namespace exotic object, at least in shape and behavior. There are two main differences though:

a) a wrapped module namespace exotic object cannot be part of the module graph, this interface is merely there for programatic access to the exported values of a module, nothing else. b) from a wrapped module namespace exotic object you can only access primitive values, or wrapped values as defined by the semantics of the callable boundary.

The wrapped module namespace exotic object does not hold a direct reference to the corresponding module namespace exotic object, it only wraps the module record and its exported names list.

ShadowRealm.importValue() method becomes obsolete, and instead this PR implements ShadowRealm.import() method that is easier to understand, and similar to the dynamic import() with the main difference that it doesn't provide dynamic scoping, meaning that the result of the invocation is not subject to the context of the caller.

The wrapped module namespace exotic object is always bound to the incubator realm, in fact, it holds a back pointer to the incubator's Realm Record, which is similar to wrapped function exotic objects. This back pointer is used to create any wrapped values.

Open Questions

1. Should wrapped module namespace exotic objects be cached by the host?

As of right now, this PR doesn't cache the result of calling .import() on a shadow realm. Therefore:

const wns1 = await myShadowRealm.import('/something');
const wns2 = await myShadowRealm.import('/something');
wns1 === wns2; // yields false

I don't believe caching is needed from the perspective that wrapped module namespace exotic objects are not participants of the module graph, so checking for its persistent is not going to be common. On the other hand, adding a cache is not super complicated to have another commonality with dynamic import().

2. Should the wrapped module namespace exotic object cache the wrapped value associated to a binding?

As of right now, this PR doesn't cache the result of the [[Get]] operation when the result is a wrapped value. Therefore:

const wns = await myShadowRealm.import('/something');
typeof wns.foo === 'function'; // yields true
wns.foo === wns.foo; // yields false it creates a new wrapped function for `foo` every time

I remember some conversations with @littledan where he was concerned about this particular aspect. I would like to explain this different, with an example of what would happen if you implement your own protocol on top of the callable boundary:

const importer = myShadowRealm.evaluate(`(specifier, name, callback) => {
    const ns = await import(specifier);
    callback(ns[name]);
}`);
importer('/something', 'foo', (value) => {
    value; // will be different every time you call this import with these arguments
});

Basically, these two mechanisms are analogous, and you can reasoning about them from that perspective. An alternative would be to cache the result of the [[Get]] operation at the wrapped module namespace exotic object with a weakmap-like mechanism where the key must be the callable value returned from the [[Module]], and the value must be the wrapped function exotic object.

3. Should Module Namespace Exotic objects be wrapped automatically when crossing the callable boundary?

This PR only creates wrapped module namespace exotic objects when the .import() method of a ShadowRealm is called. There are two main reasons to generalize this for the callable boundary:

a) when a module exports a namespace object, e.g. export * as ns from "/something";, in which case accessing ns of the wrapped module namespace exotic object will throw an error.

b) the ergonomics of passing a module namespace exotic object are great, e.g. :

const importer = myShadowRealm.evaluate(`(specifier, callback) => callback(await import(specifier))`);
importer('/something', (wns) => {
    wns.foo();
});

Note 1: Doing this auto-wrapping is very straight forward in terms of the spec text.

Note 2: A binding name exporting a module namespace exotic object is also going to be wrapped when accessed thru the wrapped module namespace exotic object. Doing so at the call boundary level for arguments and returned values seems like the right thing to do.

[mhofman]

1. Should wrapped module namespace exotic objects be cached by the host?

I'd say no, especially if a module namespace object is made to implicitly cross the callable boundary (which has its own concerns detailed below). Regardless, we had the same argument about callables, and didn't see a valid reason to persist identity.

2. Should the wrapped module namespace exotic object cache the wrapped value associated to a binding?

Definitely not. As you mention, a userland implementation of this would similarly recreate the value for every Get

3. Should Module Namespace Exotic objects be wrapped automatically when crossing the callable boundary?

My main concern on this is that it'd enable a way to brand check module namespaces, which the spec currently does not expose. However I believe the Compartment / Module Loader API will likely expose such a brand check too, and a membrane will likely be the one using the callable boundary anyway, allowing it to intercept module namespace objects and proxy them however needed.

[mhofman]

Do we have spec rendering for PRs on this repo?

[leobalter]

Do we have spec rendering for PRs on this repo?

@mhofman while this repo doesn't have a auto renderer for PRs, you can do this locally by fetching this branch and running the npm install && npm run build on it. The files will be at the dist/ folder.

[caridy]

@littledan convinced me to split this into two separate changes, one is to figure the importing errors, while the other one about the wrapped module namespace exotic objects can be defer to v2.

[caridy]

Closing this to avoid confusion.