Built-in Modules

[bterlson]

Built-in modules come up repeatedly around the various proposals. I am making this issue to centralize the discussion such that champions of the eventual proposal have a good central location for information. I will keep this up-to-date if there is further discussion in this issue.

Existing Discussion

• https://github.com/tc39/ecmascript_simd/issues/323

  Syntax Options

  Naming convention inside module specifier

This option entails establishing a naming convention for built-in modules. Has to be compatible with existing ecosystems, eg. we cannot clobber an npm or standard node package name.

Strawman: *-sigil

import "*SIMD";
import {float32x4} from "*SIMD";
import SIMD from "*SIMD";

Strawman: URL scheme

import { float32x4 } SIMD from "std:SIMD";

Distinct syntax for built-in module imports

This option necessitates additional reflective capabilities in the Loader API to request built-in modules by name as opposed to going through the normal module resolution process.

Strawman: IdentifierName instead of StringLiteral

import SIMD;
import {float32x4} from SIMD;
import SIMD from SIMD;

Semantics Requirements

Must defer to loader to resolve built-in modules (important for polyfillability). Loader may see either a special string or possibly an options record indicating that a built-in module is requested.

[tracker1]

I know that it may not work with legacy MacOS, but I'm still in favor of a ':' or '::' prefix (ex: :simd). I do think that submodules for globals should probably just use the forward slash (:foo/bar), so that part can be polyfilled as @ecmascript/foo/bar. Not sure if anyone from npm is in here, but reserving something like @ecmascript as an organization would be a good preemptive step.

[allenwb]

There seems to generally be an assumption in this thread that built-in modules would have to be identified using a string literal module descriptor. But that's not really the case, TC39 gets to define new syntax. To avoid any clashes with host/platform use of string module specifiers, I suggest the following change to the ModuleSpecifier syntax:

ModuleSpecifier :     StringLiteral     BuiltinSpecifier

BuiltinSpecifier :     IdentifierName     BuiltinSpecifier . IdentifierName

With the static semantic restriction that std as the first IdentifierName of a BuiltinSpecifier is reserved for use in naming built-in modules defined by TC39.

So we might have:

import {doc, freeze} from std.decorators;
import streams from web.streams;
import procs from node.processes;
import gcControl from v8.gc;

It isn't clear if there is a requirement to use dynamic import for built-in modules. Because built-ins are built in and presumably already present as part of the implementation it isn't obvious that there would be any advantage to conditionally loading them. But if that functionality is needed, it can be accommodated without any syntactic ambiguity by:

ImportCall :     import ( AssignmentExpression )     import BuiltinSpecifier

[Jessidhia]

Dynamic import definitely would be required if you are writing a module that can gracefully support multiple environments while using their builtins.

For example, a fetch implementation that uses XMLHttpRequest on browsers, and node.net on node.

[ljharb]

Conditional static imports would be necessary for built-in modules, to satisfy polyfilling and multi-env use cases.

[allenwb]

Personally, l would prefer to use an external config map for such situations.

But, as noted in my sketch ImportCall could be made to work with BuiltinSpecifier

[ljharb]

ImportCall is asynchronous; that doesn’t satisfy the use cases i mentioned above. We need dynamic static imports to mak that tenable.

[guybedford]

It feels like many comments in this thread didn't heed the advice from https://github.com/tc39/ecma262/issues/395#issuecomment-184966727.

If we were to have the ability to map bare specifiers to URLs in browsers, then this space might be well suited to builtin modules, and NodeJS could do the same.

For example, import "@simd" (@ as strawman) could be a builtin name. For browsers or NodeJS that doesn't support the builtin, it could be mapped by a browser resolver / manifest and by a custom NodeJS resolver, to wherever its actual shim implementation is - there's no need for the URLs to align at this level.

Strings do seem important though for the shimming process.

[robpalme]

I believe past discussions stated that a fully programmatic resolver hook would be unthinkable to standardize in browsers but do not know the reasons.

Would a more limited static string mapping configuration for specifiers alleviate the concerns blocking the resolver hook?

On Sat, 10 Mar 2018, 11:54 Guy Bedford, notifications@github.com wrote:

It feels like many comments in this thread didn't heed the advice from #395 (comment) https://github.com/tc39/ecma262/issues/395#issuecomment-184966727.

If we were to have the ability to map bare specifiers to URLs in browsers, then this space might be well suited to builtin modules, and NodeJS could do the same.

For example, @simd (@ as strawman) could be a builtin name. For browsers or NodeJS that doesn't support the builtin, it could be mapped by a browser resolver / manifest and by a custom NodeJS resolver, to wherever its actual shim implementation is - there's no need for the URLs to align at this level.

Strings do seem important though for the shimming process.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tc39/ecma262/issues/395#issuecomment-372024186, or mute the thread https://github.com/notifications/unsubscribe-auth/AGni9SWtzJqPS6pRf3RXQvj_33hDcftYks5tc77mgaJpZM4Hbijo .

[allenwb]

@ljharb

We need dynamic static imports to make that tenable.

I'm not sure what you mean by "dynamic static". I can imagine a non-failing static import:

import ?gotFoo foo from std.foo ; 
// If import succeeded gotFoo is true and foo has the default export value
// If import failed gotFoo is false and foo is TDZ uninitialized.

But, why all the barriers to progress. We don't need to boil the ocean. TC39 has one specific need, a way to namespace manage new standard library features using modules. If that is all "built-in modules" supports that's fine. It would be nice if it also can be used for built-in host platform modules. Even if the web platform wouldn't use it, I'm sure node and other environments would.

We have plenty of tools and techniques for bundling or otherwise statically or dynamically configuring JS apps for deployment to various environments. We don't need to invent new configuration management mechanisms before taking the basic step forward that enables modularizing the ES standard library as it grows.

[ljharb]

Having a half-baked mechanism for adding new standard features isn’t sufficient; if it doesn’t have all the polyfillability and secure-ability that globals do, then new features will have to continue to also be added as globals. In other words, TC39’s need can not be met without these features.

[allenwb]

Not half-baked, just limited scope. Many of the problems that are being raised are platform issues and not TC39 issues. As I said on twitter, until TC39 actual defines some standard modules no platform will try to deal with those issues.

[SMotaal]

Just to throw it out there, specifically regarding the "from clause" and what I like to refer to as the standard/platform library equivalent to platform globals or the default namespace — which may be different from various perspectives taken in this thread, so let me refine it a little:

• default namespace is equivalent to platform globals (ie no name collision)
• default namespace is probably actually platform globals in legacy
• default namespace works like platform globals in legacy

If there is only one such namespace (exporting from other namespaces as each platform sees fit) then the "from clause" could simply be dropped in static imports. The questions left unanswered are:

1. how breaking will it be for tooling to adapt the new syntax?

   This point was very important when considering the "from clause", because strings and identifiers are miles apart in the amount of work a tool will need to do to analyze modules (especially critical for loaders).

2. what could this look like?

   import { Object, fs, process };
   !fs // true
   !process // false
   if (process) process.exit(); // bye

3. what scenarios would justify dynamic imports?

   no idea, but possible

4. what would a dynamic import look like?

   import global;
   global.dynamicThing // => export default { async get dynamicThing() { … } }

Edit: A key goal here is to make it possible to transition polyfilling into a module space if they choose and not affect existing behaviour all while allowing new code to be structured away from globals. I don't make any claim regarding how to address the mutation of import <global namespace as IdentifierName> which I gather will be currently synonymous to the current global scope (then maybe evolve).

Update: Should the syntax need a little more verbosity it is possible to consider things like import default global or import default {…} but this is secondary (the idea of skipping resolution and not forcing platforms to diverge throughout is really the main point).

[tomzhang]

There seems to generally be an assumption in this thread that built-in modules would have to be identified using a string literal module descriptor. But that's not really the case, TC39 gets to define new syntax. To avoid any clashes with host/platform use of string module specifiers, I suggest the following change to the ModuleSpecifier syntax:

ModuleSpecifier :     StringLiteral     BuiltinSpecifier

BuiltinSpecifier :     IdentifierName     BuiltinSpecifier . IdentifierName

With the static semantic restriction that std as the first IdentifierName of a BuiltinSpecifier is reserved for use in naming built-in modules defined by TC39.

So we might have:

import {doc, freeze} from std.decorators;
import streams from web.streams;
import procs from node.processes;
import gcControl from v8.gc;

It isn't clear if there is a requirement to use dynamic import for built-in modules. Because built-ins are built in and presumably already present as part of the implementation it isn't obvious that there would be any advantage to conditionally loading them. But if that functionality is needed, it can be accommodated without any syntactic ambiguity by:

ImportCall :     import ( AssignmentExpression )     import BuiltinSpecifier

I do think this is the best one for naming URL which clear enough for human understanding

[Muradalsaaideh]

There seems to generally be an assumption in this thread that built-in modules would have to be identified using a string literal module descriptor. But that's not really the case, TC39 gets to define new syntax. To avoid any clashes with host/platform use of string module specifiers, I suggest the following change to the ModuleSpecifier syntax:

ModuleSpecifier :     StringLiteral     BuiltinSpecifier

BuiltinSpecifier :     IdentifierName     BuiltinSpecifier . IdentifierName

With the static semantic restriction that std as the first IdentifierName of a BuiltinSpecifier is reserved for use in naming built-in modules defined by TC39.

So we might have:

import {doc, freeze} from std.decorators;
import streams from web.streams;
import procs from node.processes;
import gcControl from v8.gc;

It isn't clear if there is a requirement to use dynamic import for built-in modules. Because built-ins are built in and presumably already present as part of the implementation it isn't obvious that there would be any advantage to conditionally loading them. But if that functionality is needed, it can be accommodated without any syntactic ambiguity by:

ImportCall :     import ( AssignmentExpression )     import BuiltinSpecifier