add top level await support for iife

[LongTengDao]

Feature Use Case

Feature Proposal

If the input is:

const { a } = await import('...');
const b = await loadFile('...');
export default a + b;

The output could seems like:

( async function () {
    'use strict';
    const { a } = await import('...');
    const b = await loadFile('...');
    return a + b;
} )();

[joe-barnett]

Would a PR implementing this be accepted? Is there anything that makes this more difficult than it sounds?

[lukastaegert]

There is one problem I see: If the IIFE is creating a global variable (because there are entry exports as in your example), it would create code like this:

var myBundle = (async function () {
    'use strict';
    const { a } = await import('...');
    const b = await loadFile('...');
    return a + b;
})();

That would mean the global variable (here myBundle) would no longer be the default export but a Promise resolving to the default export, which is very likely NOT what users expect. What users might expect is that the variable is not yet assigned until the promise is resolved.

But if the use-case is not related to creating global variables but just for stand-alone modules we could say: This is allowed for IIFE unless there are exports in the entry module, in which case we throw an error.

This might all be handled in iife.ts: You already have the hasExports flag, and you also get usesTopLevelAwait from the FinaliserOptions. You will probably need to adjust some other safe-guards in the code-base, though. What do you think, would that fit your use-case?

[Pauan]

That would mean the global variable (here myBundle) would no longer be the default export but a Promise resolving to the default export, which is very likely NOT what users expect.

I would expect it to return a Promise, that also matches how import() works in script contexts.

(Also I would expect the Promise to return a Module, which contains a default property, just like when using import())

What users might expect is that the variable is not yet assigned until the promise is resolved.

Do you have any evidence for that? Are there other JS loading systems which work that way? And how would the user know when the variable is assigned?

[lukastaegert]

would expect it to return a Promise, that also matches how import() works in script contexts.

The problem is that this has nothing to do with import() but rather with static import. If you do an external import in an IIFE, it assumes you want to get a global variable.

https://rollupjs.org/repl/?version=2.66.1&shareable=JTdCJTIybW9kdWxlcyUyMiUzQSU1QiU3QiUyMm5hbWUlMjIlM0ElMjJtYWluLmpzJTIyJTJDJTIyY29kZSUyMiUzQSUyMmltcG9ydCUyMCU3QmZvbyU3RCUyMGZyb20lMjAnYmFyJyUzQiU1Q25jb25zb2xlLmxvZyhmb28pJTNCJTIyJTJDJTIyaXNFbnRyeSUyMiUzQXRydWUlN0QlNUQlMkMlMjJvcHRpb25zJTIyJTNBJTdCJTIyZm9ybWF0JTIyJTNBJTIyaWlmZSUyMiUyQyUyMm5hbWUlMjIlM0ElMjJteUJ1bmRsZSUyMiUyQyUyMmFtZCUyMiUzQSU3QiUyMmlkJTIyJTNBJTIyJTIyJTdEJTJDJTIyZ2xvYmFscyUyMiUzQSU3QiUyMmZvbyUyMiUzQSUyMmZvbyUyMiUyQyUyMmJhciUyMiUzQSUyMmJhciUyMiU3RCU3RCUyQyUyMmV4YW1wbGUlMjIlM0FudWxsJTdE

So if you do an export, it should create such a global variable. The problem is now that currently, those variables just contain whatever you export. If you add a top-level await and assign them a promise, then you would suddenly need to await also in the importing file. Worse even just assume you have a top-level await not in the main file but in a dependency. Now such a change can even sneak in without you noticing.

I mean we can do whatever we want, but I find it weird to have such subtle changes when using TLA.

[Pauan]

The problem is that this has nothing to do with import() but rather with static import.

IIFE is run in script contexts (ES5), static import is for ES6 modules only, whereas import() works in both ES6 modules and ES5.

The whole point of IIFE is to take the ES6 modules and convert them into an ES5 form, so static import is not relevant for that, but import() is very relevant, since it serves exactly the same purpose as IIFE (importing ES6 modules in ES5 code).

So the closer that IIFE can match the behavior of import(), the better, since developers will already be familiar with using import() in ES5 code.

Promises are the standard way to handle asynchronous code in JS, they are well supported in browsers, have been around for a long time, and are familiar to most JS developers. That is why import() uses Promises.

So if you do an export, it should create such a global variable.

Yes, it should create a global variable which is a Promise that resolves to a Module (which contains the exports of the ES6 module).

Basically it should behave the same as import() (except it assigns the Promise to a global variable). In other words, it should behave like this:

var myBundle = (async function () {
    'use strict';
    const { a } = await import('...');
    const b = await loadFile('...');
    return {
        default: a + b
    };
})();

When using top level await, Webpack also returns a Promise, it behaves exactly like how I described above (a global variable which is a Promise that resolves to a Module).

If you add a top-level await and assign them a promise, then you would suddenly need to await also in the importing file. Worse even just assume you have a top-level await not in the main file but in a dependency. Now such a change can even sneak in without you noticing.

Yes, that is also true for other loaders like Webpack. That is simply how top-level await works, Webpack has exactly the same problems, and Webpack even proposed to add a new import await syntax to help fix that (but that proposal was rejected). This issue has already been decided at the standards level (for better or worse).

The most ideal situation is to make IIFE always use a Promise (regardless of whether top level await is used or not), but that would be a breaking change in Rollup, so it would need a major version bump. So making IIFE use a Promise only when top-level await is used is a good compromise in the meantime (that's also what Webpack does).

[lukastaegert]

IIFE is run in script contexts (ES5), static import is for ES6 modules only, whereas import() works in both ES6 modules and ES5 The whole point of IIFE is to take the ES6 modules and convert them into an ES5 form, so static import is not relevant for that, but import() is very relevant

Thank you for explaining to me what in your opinion IIFE is all about, having maintained Rollup for 5 years I surely have no idea. Just note that "hereas import() works in both ES6 modules and ES5" is just nonsense, please get your terminology right (dynamic import is an ES2020/ES11 feature, "script context" was actually the only correct term).

Apparently you did not understand what I was talking about, or did not take the time to have a look at or understand the implications of the REPL link I provided which should explain to you how Rollup works at the moment.

OF COURSE there is no import in the output! There are imports in the INPUT! And there is a semantic to translate those to global variable accesses in Rollup that needs to be respected in some way when implementing this feature because thousands of libraries are depending on this for their browser builds. (Side note: The Webpack issue is speaking my point, and incidentally I have been part of those discussions from the beginning though I did not participate too actively. The result was that within a module graph, you can import TLA modules without any need for dynamic import(). This will NOT be possible if you put two IIFEs in separate script tags next to each other because you cannot "encode" this information in the static import).

And when the imported library is a providing a Promise as a global variable, that is also fine if you know it, but I am quite sure that many current Rollup users would be very unhappy if we just forced exports to be Promises for all of them. So my question: Just make it a Promise when there is a TLA in the source, which may be even hidden and can lead to unexpected results? Or just postpone the question by forbidding exports in the first iteration (we can always add a solution later without a breaking change, that would be a sensible solution for fast iteration)? Or control it via a config option (I would like that, makes things explicit, no surprises here)? Other ideas?

[LongTengDao]

@lukastaegert

There is one problem I see: If the IIFE is creating a global variable (because there are entry exports as in your example), it would create code like this:

var myBundle = (async function () {
    'use strict';
    const { a } = await import('...');
    const b = await loadFile('...');
    return a + b;
})();

That would mean the global variable (here myBundle) would no longer be the default export but a Promise resolving to the default export, which is very likely NOT what users expect. What users might expect is that the variable is not yet assigned until the promise is resolved.

Oh, good catch... I didn't thought about this.

But if the use-case is not related to creating global variables but just for stand-alone modules we could say: This is allowed for IIFE unless there are exports in the entry module, in which case we throw an error.

This might all be handled in iife.ts: You already have the hasExports flag, and you also get usesTopLevelAwait from the FinaliserOptions. You will probably need to adjust some other safe-guards in the code-base, though. What do you think, would that fit your use-case?

After moments of thinking, I think there could be too many use cases, it seems not good to try to handle them all in rollup engine. And an arbitrary choice is worse. For example, people may want:

var myBundle = ( async function () { return xxx; } )();

var myBundle = ( async function () { return xxx; } )();
myBundle.then(xxx => { myBundle = xxx; });

var myBundle = ( async function () { return xxx; } )();
myBundle.then(xxx => { myBundle = xxx; });
myBundle = undefined;

...

And what's more, no one above can cover the amd/cjs formats.

And when consider your repl (with external import), thing becomes more complex.

If we support this only when the bundling has no external import and no exports, there will have no different with format esm + async function wrapper manually.

So, this feature seems not easy to add. Personally, the issue could be closed at present. (Or of cause it could still keep open to collect other uses' suggestion. I can't give more suggestion now.)

The main reason is, I don't use top level await any more in real project after months of trial, because it makes the order of code running hard to maintain.

[lukastaegert]

Well, there are two IIFE situations

• When there are exports, a global variable is created. This is the problematic one (but could be solved in some way)
• When there are no exports, it is just a function that usually no other script depends upon. If this were an async function, I think this would not cause any problems or defeat expectations.

So we could still add value for users when we implement it for the "no exports" case and for now throw when a variable would be created.

[frank-dspeed]

iife can not realy have top level await top level await is a ESM Feature that is easy to implement in ESM World as every ESM Module is a Promise by Default so in ESM Context your already in a Promise and a iife for ESM is not needed everything is already a iife if it is ESM Code as it is instantiated after import i hope that makes some sense

[frank-dspeed]

i would opt for what @lukastaegert sayed you should wrap your code with async iffe functions so it is clear for you that your getting back promises. you can then produce with that code normal iife builds.

[LongTengDao]

Well, there are two IIFE situations

• When there are exports, a global variable is created. This is the problematic one (but could be solved in some way)
• When there are no exports, it is just a function that usually no other script depends upon. If this were an async function, I think this would not cause any problems or defeat expectations.

So we could still add value for users when we implement it for the "no exports" case and for now throw when a variable would be created.

If the feature not mean lot of work, I think what you said is the best thing that could be done for now.

After all, users can write globalThis.xxx = xxx instead of export default xxx in their entry script to achieve any goal based on this feature, with less manual behaviour.

[Pauan]

@lukastaegert Thank you for explaining to me what in your opinion IIFE is all about, having maintained Rollup for 5 years I surely have no idea.

You are interpreting my comment in a very harsh way, which was not my intention. I was explaining the duality between IIFE and import(), since they serve the same purpose, and thus their behavior should match (which is also how other loaders like Webpack do things). I was justifying the usage of Promise.

Please get your terminology right (dynamic import is an ES2020/ES11 feature, "script context" was actually the only correct term).

Earlier in my post I specifically said "scripting context", I was clearly using ES5 as a short-hand for that, as opposed to ES6 modules. This is technically incorrect but it's commonly used language.

Apparently you did not understand what I was talking about, or did not take the time to have a look at or understand the implications of the REPL link I provided which should explain to you how Rollup works at the moment.

Your REPL shows an external module bar which is outside of the Rollup system (and loaded by the user in some other way).

Supporting asynchronous external imports is a separate issue from top level await, however it is quite easy to support. Rollup would simply have to generate code like this:

(async function (bar) {
    'use strict';

        bar = await bar;

        console.log(bar.foo);
})(bar);

Now it will work regardless of whether bar is a plain value or a Promise, both types of modules will work transparently, without requiring any user intervention or knowledge. This is a backwards-compatible change.

[lukastaegert]

Always injecting an await would be problematic as it would

• generate output that relies on working await, making it incompatible with older browsers without another transform step, even if you do not require this feature
• change execution semantic as it would insert micro task wait inside synchronous code, making it no longer synchronous. If another consumer relies synchronous execution (e.g. a library that does not know about this feature or was not updated to support it), it becomes unusable.

Which gets me back to one of the things I suggested: Make this an opt-in. That means on the other hand that such a flag could control three things:

• support top-level await
• make the exported variable a promise instead of the actual value
• await imports to external variables

E.g. output.asyncIife: true. And throw an error if TLA is used without the flag, pointing users to use it.

[Pauan]

async/await is very old, it works in every browser except for IE. Even Microsoft doesn't support IE anymore and IE only has 0.61% market share, it's time to let IE die.

In the rare situations where a user wants to use top-level await and also needs to support IE, they can use something like regenerator, but that's up to the user. In that case the user will also need to handle polyfills and various other things for IE, that's outside of the scope of Rollup. Existing code doesn't use top-level await, so this won't break anybody's current builds.

How did you plan on supporting top-level await without using async/await? If the user uses top-level await, you have to compile it to something, so you're either compiling it to .then calls, or you're compiling it to await, so you will have to deal with this regardless of external imports. Whatever mechanism you use for compiling top-level await, you simply use that same mechanism for external imports. You just treat the external imports as if they were a top-level await.

E.g. output.asyncIife: true. And throw an error if TLA is used without the flag, pointing users to use it.

I personally don't think that's necessary, since these changes would only happen if top-level await is used, but there isn't anything wrong with doing things like that, it's a safer choice.

However, the end goal should be that top-level await is a first class citizen, there shouldn't be a distinction between top-level await modules and regular modules. So in the long term, it would be good to always generate a Promise regardless of whether top-level await is used or not (this will require a major version bump).

[lukastaegert]

0.61% market share, it's time to let IE die.

There are other older browsers that do not support it, ask caniuse.com, async await is not working on 7% of browsers. That is not negligible for some of our users.

In the rare situations where a user wants to use top-level await

What about the possibly less rare situation that users do NOT want to use TLA but want to consume a library that does? For that, they would either need an opt-in to await all imports or we always await. But always waiting WOULD break a lot of stuff. Basically every existing sync module that depends on a suddenly async when you place them next to each other in script tags.

How did you plan on supporting top-level await without using async/await

I did not plan to at all. Users who use TLA usually also use code-splitting, so they would be using AMD or SystemJS output anyway. But that is the beauty of Open Source, sometimes features are driven by demand and people can just implement them.

I personally don't think that's necessary, since these changes would only happen if top-level await is used

As noted above, such a flag would go two ways.

there shouldn't be a distinction between top-level await modules and regular modules

Maybe, but things tend to die very slowly. What we can hope for is that something that starts as an opt-in would become an opt-out eventually.

[Pauan]

What about the possibly less rare situation that users do NOT want to use TLA but want to consume a library that does?

In that case the user has two options: they can enable top-level await (and deal with the asynchronousness and lack of IE support), or they can choose to not use that library.

There is no way for a user to use a top-level await library in a backwards-compatible way, because top-level await is a fundamentally different (and new) way of writing modules, which will not work with synchronous code.

I did not plan to at all.

That's not what I was asking. You are making an argument that async shouldn't be used because of browser support. But top-level await has to be supported in some way (whether with async or something else), regardless of the external imports issue. So if it's not going to be implemented with async, then what is the alternative?

so they would be using AMD or SystemJS output anyway

If the user writes code which uses top level await, you have to compile it to something. Using AMD or SystemJS doesn't fix that. For example, the SystemJS output uses async + await, so it has exactly the same issue with browser compatibility (and microticks causing asynchronousness). This has nothing to do with IIFE vs SystemJS. If async is good enough for SystemJS, it should be good enough for IIFE too.

[lukastaegert]

In that case the user has two options: they can enable top-level await

Exactly my point that it needs to be an opt-in that you can enable explicitly without the need to artificially add a TLA to the source, but I think we are on the same page here now.

You are making an argument that async shouldn't be used because of browser support.

Actually I only wanted to argue against enabling it "by default" without a way to get the old behaviour. For an opt-in feature, I have no worries about compatibility because then it is a conscious decision.

So in the end I have the impression we mostly agree now.

[Pauan]

So in the end I have the impression we mostly agree now.

In general, yes, though if you're going to have a flag I think it should be a single flag for all the modes (including SystemJS), not specifically for IIFE, since the compatibility and library issues are the same.

[lukastaegert]

since the compatibility and library issues are the same

Maybe, though SystemJS would be the only other format affected at the moment anyway. But the implications are not the same for SystemJS: We do not need to await imports for SystemJS. So consumers do not need to know if a module uses TLA to be able to use it, the SystemJS runtime will take care of it, just like the ES runtime for ESM. On the other hand for IIFE, import awaiting can also break imports if having a Promise as only export is the intended interface for a module.

So I am not sure having the flag for SystemJS as well would provide value for enough people to warrant the additional complexity.

[frank-dspeed]

i have 0 real usecases for tla at all:

in general as every Module is a Promise i can always export Promises i feel like implementing TLA leads in general to confusion as it is only syntax sugar it adds nothing to promise.then(successCallback, throw) i think that would even be the better translation for TLA

There is no real TLA as everything is a Promise and TLA is syntax sugar.

const successCallback = (result) => Promise.resolve(result); // Async Translated! a function that returns a Promise
module.exports = Promise.resolve().then(successCallback, throw)

doing TLA is translated as follows Pseudo Code but it is near my real implementation

const TLAImports = [
 [Promise, importDefinitionVarNameDeconstructorLike]
];
TLAImports.map(([moduleExports, requestedPropertys])=> ({
  //.. object with instantiated moduleExports
})).then(allTlaModules=>{
// here we programatical assign const
const [key, value] = Object.entries(o)[0]; // in real we do something more hacky that iterates but that shows the concept.

  //.. here now all other code gets executed 
  /** 
   It is also Importent to know that TLA Always gets Processed Complet before any 
   other code in the module runs as TLA is Part of the Module instantiation Phase Step even if you use it as a execution step like 
   await Promise.resolve(); it gets Processed before the other code runs.
  TLAImports does not only reference dynamic imports it does in general process every Promise so TLA is like 
  Promise.all().then()
  */
})

well looks a bit like SystemJS already.

[RReverser]

Which gets me back to one of the things I suggested: Make this an opt-in. That means on the other hand that such a flag could control three things:

• support top-level await
• make the exported variable a promise instead of the actual value
• await imports to external variables

E.g. output.asyncIife: true.

This seems like a great plan IMO. I agree awaiting imports like @pauan suggested is the easiest way to go about solving TLA issue for external modules, but I also agree it would be a breaking change for users who don't rely on TLA since suddenly their synchronous IIFEs would become async.

So a separate option to control whether both inputs & outputs should be treated as async seems like the best way to go.

[frank-dspeed]

i am wondering if we can reproduce the desired behavior via references? to the parts that we want to keep async and get the rest as iife? i mean in general we can handl await including top level await as require or static import. as long as the specifier is handle able.

when that is the case we simply always inline it exempt we mark anything matching as external when we mark it as external we simply build it external from the same build cache

i did such stuff in a few projects it worked well it only needed a bit of rethinking how to build in general we end up in slight stacked builds but that is also beneficial for larger projects.

It reduces amount of watched files i guess we should invest into more documentation about staged and stacked builds using the bundle cache to create them.

the documentation about that is simply not written at point of time.

[danrossi]

I have a project using a top level await and wondering if there is a work around to be able to bundle correctly ?

[Pauan]

@danrossi Your best option right now is to use format: "es" and then load your script with type="module". This will work in all modern browsers.

If you really need to support IE (even though Microsoft themself dropped support for IE), then it gets a lot more complicated.

In that case you will have to use something like format: "system" and use SystemJS to load your files.

[danrossi]

Yes I am all for bundling as es6. As the iife is es6. However how to make the exports a global namespace like iife with rollup ? So the end user doesn't need to import anything. Some modules set itself up on load and nothing needs to be imported. Some references the exports in a namespace.

More to that in the modules the exports need to be able to append to a current global namespace if it exists like you can do with iife. Ive got export { ClassObjectName }; It needs to be able to used on inclusion as NAMESPACE.ClassObjectName and not expect end users to have to import all of this inline.

I'm unsure how to make es6 module includes deliverable. One module imports from another. If I include them on the page like this it won't work.

In my project module at the top the imports are externalised. import {ClassObject} from 'lib';

<script type="module" src="../three.js/build/lib.module.js"></script>
<script type="module" src="../build/project.module.js"></script>

The purpose of iife is to not need to expect people to import onto the page. Unless its being bundled into their own app. Which they then need to include their es6 app onto the page like this and the same problem happens.

[Pauan]

However how to make the exports a global namespace like iife with rollup ? So the end user doesn't need to import anything.

You can create a wrapper module to do that:

import { ClassObjectName } from "./path/to/file.js";

globalThis.ClassObjectName = ClassObjectName;

It's not the cleanest solution, but it works. You can also export the entire namespace if you like:

import * as myNamespace from "./path/to/file.js";

globalThis.myNamespace = myNamespace;

I'm unsure how to make es6 module includes deliverable. One module imports from another.

I don't understand what you mean by that.

In my project module at the top the imports are externalised.

Does that work with iife? I thought iife only works with entry points, are you using three.js as an entry point?

[danrossi]

I'll try the wrapper method and see what happens. Can rollup do that automatically ? . Importing an es6 module it has to run just in time functions also. Deliverable as in the end app there is little to no interaction with inline code. Just like with umd/iife. The problem is with three.js but an await call should still be initialized only when the feature is being used. I've moved it from top level and make it resolve promises for feature detection personally.

https://github.com/danrossi/three-webgpu-renderer/blob/master/src/renderers/webgpu/WebGPUBackend.js#L22

here is a couple of three projects I am packaging to iife and using the global namespace inline in tests or importing the module directly which is not the deliverable use case. Packages I deliver personally have to be drop inneable and for end users who may not understand how to import es6.

https://github.com/danrossi/three-webgpu-renderer https://github.com/danrossi/three-troika

[danrossi]

I tested this. the globalThis.namespace = namespace worked.

And then import like

 <script type="module" src="../build/three-webgpu.js"></script>

However if there is an await in the top level. The global namespace isn't available to code on page load until the await is finished.

It blocks the inline code from running when putting it into that block. As far as I can see even a top level async iife, the exports in the namspace may only become available after so will block inline code from seeing the namespace as I just tested.

[Pauan]

However if there is an await in the top level. The global namespace isn't available to code on page load until the await is finished.

Yes, top-level await is inherently async. There is no way to make it synchronous.

So in that case you should make the global namespace into a promise:

globalThis.myNamespace = import("./path/to/file.js");

Now your users can use your library like this:

async function main() {
    let { foo, bar, qux } = await myNamespace;
    // User code goes here...
}

Now it will work correctly even with top-level await.

[danrossi]

My own use case is web player plugins. The player calls the functions registered in the script. No inline code called. Hence why iife is used. Users dont need to understand modules and imports. It runs code just in time on the page. I tried it with the top level async and videojs for one can't see the registered plugin yet on page load. I may have to keep editing a patched file to remove the top level async.

[Pauan]

Hence why iife is used. Users dont need to understand modules and imports.

With my code above, users also don't need to understand modules or imports.

Promises are separate from modules, Promises have been around for an incredibly long time. So your users should be familiar with using Promises for async code. Many web APIs (such as fetch) use Promises, it is a fundamental part of JavaScript.

If your code is async, then your users will need to use Promises, there is no way to avoid that. It is impossible to convert async code into sync code.

iife won't help you, because the problem is not with the modules, the problem is that your code is inherently async.

[lisonge]

https://github.com/lisonge/rollup-plugin-tla

A rollup plugin to add top level await support for iife/umd

[danrossi]

I implemented that plugin. Which wraps correctly however it fails to include implementation usage. Inline code is not the normal usage for either iife and modules apps are in included files. So once included the namespace isn't immediately available.

https://github.com/danrossi/three-webgpu-renderer/blob/master/tests/webgpu_video_iife.html#L30 https://github.com/danrossi/three-webgpu-renderer/blob/master/build/three-webgpu.js#L6

[lisonge]

@danrossi

+ THREE = await THREE
 class CustomVideoTexture extends THREE.Texture {

[danrossi]

after trial and error it seems to be working now thanks. But awaiting a global namespace in a bundle I've yet to test it. As in a converted import global in iife. So "THREE" is not in the iife bundle but global.

https://danrossi.github.io/three-webgpu-renderer/tests/webgpu_video_iife.html https://github.com/danrossi/three-webgpu-renderer/blob/master/tests/webgpu_video_iife.html#L35 https://github.com/danrossi/three-webgpu-renderer/blob/master/rollup.config.js#L182