tc39 / proposal-defer-import-eval

A proposal for introducing a way to defer evaluate of a module
https://tc39.es/proposal-defer-import-eval
MIT License
208 stars 12 forks source link

Write initial spec text, with eager evaluation of async subgraphs #17

Closed nicolo-ribaudo closed 1 year ago

nicolo-ribaudo commented 1 year ago

Preview: https://nicolo-ribaudo.github.io/proposal-defer-import-eval/ (step-by-step evaluation examples) Debugging tool: https://nicolo-ribaudo.github.io/es-module-evaluation/ (significant example) engine262 implementation: https://github.com/nicolo-ribaudo/engine262/tree/defer-eval (demo with test cases)

The modules evaluation algorithm has been updated to support deferred imports, with eager evaluation of async dependencies. It does so by effectively replacing deferred imports with eager imports to the transitive dependencies using top-level await.

Example:

```js // a import "b"; import defer * as c from "c" import "d" print("a - sync"); ``` ```js // b print("b - sync") ```
```js // c import "e" import "f" print("c - sync"); ``` ```js // e import "g" print("e - sync") ``` ```js // g print("g - start"); await 0; print("g - end") ```
```js // f import "h"; print("f - start"); await 0; print("f - end") ``` ```js // h print("h - start"); await 0; print("h - end") ``` ```js // d print("d - start"); await 0; print("d - end") ```

Would be evaluated as if a was

import "b";
import "g";
import "f";
import "d";
const c = createProxyForEvaluationOf("c");
print("a - sync");

Accessing a property on createProxyForEvaluationOf("c") will trigger evaluation of c and of all its dependencies, which can be done synchronously because all the asynchronous dependencies have been pre-evaluated when evaluating the dependencies of a.

Old description, up to f8f22f5d62dc62b715bde2d5681a33cf4fcd30ee. The algorithm has been re-written. The modules evaluation algorithm has been updated to include three new states: `~async-subgraphs-searching~`, `~async-subgraphs-evaluating-async~` and `~async-subgraphs-evaluated~`. They are ordered after `~link~` and before `~evaluating~`, but they are only used for modules that have been deferred. When evaluating a module _A_ that contains `import defer * as ns from "B"`, where _B_ does not use top-level await, _B_ shouldn't be evaluated but all the modules that it transitively imports that use top-level await must be. - The traversal algorithm sets _B_'s status to `~async-subgraphs-searching~`, and recursively traverses its dependencies setting their status to `~async-subgraphs-searching~` until it finds a module _C_ that uses top-level await. - If there is no such module _C_, all the modules transition to `~async-subgraphs-evaluated~`: all their async dependencies (i.e. none) have been evaluated, and they can be later evaluated synchronously. - If _C_ exists, the traversal algorithm proceeds by normally evaluating _C_ and its dependencies. When _C_'s status transitions to `~evaluating-async~`, all its deferred ancestors transition from `~async-subgraphs-searching~` to `~async-subgraphs-evaluating-async~`. This is similar to how in the case of full imports all the ancestors of the async module transition to `~evaluating-async~`. - If there is any error, all the ancestors transition from `~async-subgraphs-evaluating-async~` to `~evaluated~` since any further import of those modules should just relay the error. - When _C_'s status transitions to `~evaluated~`, its ancestors transition to `~async-subgraphs-evaluated~`. - If at any point any module that is in one of the `~async-subgraphs-...~` states gets eagerly imported by another non-deferred module, its status will transition to `~evaluating~` and it will go through the normal evaluation process, together with its eager dependencies.
Jack-Works commented 1 year ago

When evaluating a module A that contains import defer * as ns from "B", where B does not use top-level await, B shouldn't be evaluated but all the modules that it transitively imports that use top-level await must be.

This algorithm makes it harder to be implemented in bundlers (see https://github.com/webpack/webpack/pull/16567). I suggest making it a link error or totally ignoring the defer.

nicolo-ribaudo commented 1 year ago

@Jack-Works I'd love to learn more about the implementation difficulties. What makes it hard to evaluate asynchronous dependencies with deferred imports, that isn't already hard with normal imports?

Jack-Works commented 1 year ago

@Jack-Works I'd love to learn more about the implementation difficulties. What makes it hard to evaluate asynchronous dependencies with deferred imports, that isn't already hard with normal imports?

it's a bit sloppy for me to say difficulties, I'll try it in the webpack implementation first, then give more feedback.

guybedford commented 1 year ago

@nicolo-ribaudo unfortunately I don't have permissions to merge on this repo either, so we would need someone else to do that.

nicolo-ribaudo commented 1 year ago

I simplified the algorithm and updated the description above. I'm working on updating the step-by-step debugging tool. I already updated the engine262 implementation, and it still passes all the existing modules-related test262 tests.

nicolo-ribaudo commented 1 year ago

Before merging this PR I need to copy the ModuleRequests changes you did in the import source reflection proposal, to align them.