add `+module` to mime type for `script type=module`

[bmeck]

Please add +module to the mime type since this is a different parsing goal and fails current parsing for application/javascript.

[bmeck]

relevant to: https://github.com/whatwg/html/pull/443 https://github.com/whatwg/loader/issues/121

[domenic]

Why? What use case does this solve?

[domenic]

Please see https://wiki.whatwg.org/wiki/FAQ#Is_there_a_process_for_adding_new_features_to_a_specification.3F instead of jumping to a proposed solution.

[bmeck]

@domenic I consider this a bug because a mime type is being overloaded. Is that considered a feature?

[domenic]

MIME types are used to distinguish executable code from images mainly. If there's another use case in mind, let us know, but there's no need to make mime types up for every different possible variation. Adding a new mime type is a big burden on server deployments and needs to be counterbalanced by user-facing gains.

[bmeck]

Unclear on relation to images. I can't send down the parser goal from the server right now and the accept does not present the parser goal to the server.

[domenic]

Can you describe your use case in more detail? If browsers added this mime type to the list of acceptable mime types for script type module, what scenarios would be enabled that were previously impossible?

[domenic]

You mentioned the Accept header. HTML does not (and will not) use that for reasons discussed in https://wiki.whatwg.org/wiki/Why_not_conneg. The only place the script mime types matter is that responses with non-JS mime types are interpreted as error responses (similar to 404).

[bmeck]

If responses are the only thing that matters I guess I can close this; the module goal is a strict super set of the script goal (albeit with newer/nicer things). I have slight misgivings about the overloading since it changes where var is bound, but /shrug.

[annevk]

We might use Accept for wasm I think. But I agree that we don't need it for this and requiring a new MIME type would make adoption of modules way harder than necessary.

[bmeck]

@annevk it does change semantics though with making var statements suddenly not globally available.

[annevk]

Yeah, I agree that theoretically a new MIME type can be justified, but practically I don't really see a good reason to do it.

[littledan]

I think this is a reasonable idea, though I don't have a great understanding of what deployment difficulties it might create. Modules parse with different syntax than scripts; it's not just strict mode (which you could anyway get by eval'ing an XMLHttpRequest result, for example). For a script embedded in a page, this is indicated in the same document by type="module", but for a standalone js file, the module-ness is not attached to it anymore, and just indicated by the one who's including it.

[domenic]

Still haven't found an answer for

If browsers added this mime type to the list of acceptable mime types for script type module, what scenarios would be enabled that were previously impossible?

[annevk]

It would have a "benefit" if it was the only type we accepted for modules. That way you can prevent classic from executing as module and vice versa. It also allows the browser to syntax-highlight module scripts correctly, when viewed as standalone file.

[zcorpan]

Not vice versa since classic scripts ignore MIME type... although I suppose it could be changed to "ignore for everything except the module MIME type", if we really wanted. But we should keep in mind that requiring MIME types has historically not been without pain in the ass for Web developers.

[annevk]

@zcorpan given https://www.w3.org/Bugs/Public/show_bug.cgi?id=27852 we can probably do vice versa if we wanted to.

@domenic it's been a little hard to get it out of @bmeck, but I think the justification is that they have a platform where the host language's implementation uses the MIME type to figure out classic or module script. They would like their platform's solution to not break when the same scripts get delivered to browsers.

It would be relatively trivial for us to support this case by just adding text/javascript-module as a MIME type and either always recognizing it or only recognizing it when type=module. I do think though that if we do this we also want to accept the classic script MIME types for module scripts because otherwise adoption would be too hard. But doing that does not necessarily interfere with @bmeck's solution to the host language integration problem as I understand it.

[bmeck]

@annevk I got things sorted out, it was confusion on the right body to go to. We are ok now by not going through WHATWG. I mistook this as the place to register a mime, when it is really IANA I should talk to directly.

[annevk]

@bmeck note that even if you register a MIME type, you'd still need to change the HTML Standard if you want it to be recognized by browsers.

[Jxck]

if script/modules fetches from non browser user agent, how I can find whether response is JS or ESM ?

for example, I wanna fetch /script and then save with .js if response is JS, and .mjs if ESM. (url doesn't always have extension .js or .mjs like /index is sometimes html and that information is included in content-type: text/html)

fetch('/script').then(res => {
  const type = res.headers.get('content-type')
  if (type === 'application/javascript') {
    return saveBodyInFile("script.js", res.text());
  } else if (type === 'application/javascript+modules') {
    return saveBodyInFile("script.mjs", res.text());
  }
})

this seems help env like node.js who need to know whether response from http is JS or EJS but not know metadata in HTML (means <input type=module>) ?

[domenic]

You can't find out without knowing the context. It's easy to produce files that are both valid classic and module scripts. Without a script element or import statement to judge which the author intended, either or both could be correct.

This is somewhat similar to how you can't tell the difference between LICENSE.txt and README.txt via mime types; both are text/plain. (Not sure this is the best analogy, but it's what I can come up with at the moment...)

[Jxck]

It's easy to produce files that are both valid classic and module scripts.

thats seems not the point of this. if that’s solves, what wa the discussion around . mjs around node.js. and in html we don’t need type=module too.

I think that the point is “execution context need to know it’s script or esm before exec/parsing it, so put them in METADATA” in browser, that is type=module in node, that is .mjs in protocol, mime-type

why only in mime-type, its ok to leave both are same ?

[domenic]

Protocols are not useful on their own. They are useful in the context of consumers. Browser consumers are able to differentiate one way. Node is able to differentiate another. I'd suggest any other consumers figure out their own counterparts for differentiating. E.g. perhaps your script could take a command-line flag.

[annevk]

FWIW, I disagree with that and I wish I had tried somewhat harder for a new MIME type, but the ship has sailed at this point. Perhaps at some point we can have some kind of text/javascript;is=module annotation, but since it's not required you can't ever really rely on it.

[bmeck]

@annevk https://tools.ietf.org/html/draft-ietf-dispatch-javascript-mjs-01#section-4.6 has the goal= parameter.

[littledan]

@bmeck Wait, what does this mean?

Restrictions on usage: The file extension .mjs must be parsed using the Module grammar of [ECMA-262]

Presumably, web browsers don't implement this. Should the "must" somehow only apply to Node.js?

[bmeck]

@littledan correct, there is not a "file extension" at all in web browsers. The URLs web browsers use are not representative of file paths. This does not apply to web browsers; this is like any other file extension mapping done by the server (not by the URL) like .php => text/html or .js => text/javascript.

The "must" applies to anything with knowledge of file extensions (which excludes browsers); that restriction is not limited to Node.js.

[annevk]

Browsers have knowledge about file extensions actually for the purposes of <input type=file>. If we started to recognize .mjs there we'd map it to text/javascript and if that ends up in a <script> it would just be parsed like normal JavaScript.

[annevk]

See https://github.com/w3c/FileAPI/issues/51 about standardizing that.

[bmeck]

@annevk wouldn't that invalidate the note in the MIME sniffing standard? If so, wouldn't .mjs map to text/javascript; goal=module?

[annevk]

It wouldn't, because the resource is not retrieved over HTTP. And even if it would map to that, it would still be parsed as ordinary JavaScript when fetched via <script>.

[bmeck]

@annevk <script> doesn't even respect MIME so isn't that expected? https://codepen.io/bradleymeck/pen/qVONmN?editors=0011

[annevk]

I'm just illustrating how the must requirement you cited above ends up being false in major implementations.

[bmeck]

@annevk we can amend the draft then. Would changing it to have an additional usage restriction that goal=module is required for .mjs be enough?

[annevk]

Unless you actually change the HTML Standard that <script> ends up rejecting goal=module (it's a case-insensitive match?) I don't think it'll work.

[bmeck]

@annevk <script> doesn't respect MIMEs, I think that is out of scope.

[annevk]

How can it be out-of-scope for such a broad requirement?

[bmeck]

@annevk the same way that I can serve text/html and <script> doesn't treat the content as HTML

[bmeck]

We can't change <script> to suddenly have any changes based upon MIME since it doesn't use MIME currently is my point.

[annevk]

It actually ends up rejecting a number of MIME types these days: https://fetch.spec.whatwg.org/#should-response-to-request-be-blocked-due-to-mime-type?

[bmeck]

@annevk I'm open to suggestions, but still think it doesn't respect MIMEs since it lets you use text/html as text/javascript; goal=script

[annevk]

That's true, but we also don't have a statement somewhere that says that resources labeled with text/html must always be parsed as HTML I think. Any such statement would be false and not implementable by most clients.

[bmeck]

@annevk That seems a different argument about if MIME is reliable in all situations.

[annevk]

It's https://github.com/whatwg/html/issues/558#issuecomment-341145487 rephrased.

[bmeck]

@annevk open to suggestions but don't agree about the usage restriction being false?

[annevk]

It's weird actually. Why is a usage restriction phrased as a requirement on clients? If it was something like ".mjs must only be used for resources intended to be parsed as module scripts" that'd be more acceptable.

[bmeck]

@annevk wouldn't that phrasing effectively remove the restriction entirely? It is about intent rather than any sort of concrete property of a resource.

[annevk]

@bmeck what is the restriction you're trying to enforce? Currently what you have reads as a restriction on a consumer of the resource, which seems wrong for a usage restriction (those should apply to the producer only).

[bmeck]

@annevk the ability to author to the Module goal of ECMAScript and not have it ambiguous with Script. This has different limitations on environments that are server/client (Server/Browser) or just environment (Node).

Currently what you have reads as a restriction on a consumer of the resource, which seems wrong for a usage restriction (those should apply to the producer only).

I don't see how they only apply to producers.

[jakubrpawlowski]

Already having issues with it in Chrome 62: Failed to load module script: The server responded with a non-JavaScript MIME type of "application/octet-stream". Strict MIME type checking is enforced for module scripts per HTML spec.

I'm trying to use type="module" with "index.mjs" <script type="module" src="index.mjs"></script> When changing file extension .mjs to .js it works. When changing from "module" to "text/javascript" it also works. They just don't play well together.