Added Mozilla-compliant AST specification

[RReverser]

Generic ECMAScript+JSX parsers and some other tools use extended Mozilla AST format as intermediate representation.

This PR describes those extensions for new spec consumers that want to work with AST.

[sebmarkbage]

We should probably change these to use the JSX prefix instead of the XJS prefix. The XJS prefix was really just a legacy artifact.

[sebmarkbage]

Do you think that we should make this a stand-alone document that we refer to as normative? It's technically not necessary to comply with this AST to comply with the spec. It is a normative recommendation though.

[benjamn]

I'd like to keep this in sync with https://github.com/benjamn/ast-types/blob/master/def/fb-harmony.js, if possible. Even better, we might be able to generate the documentation automatically (see https://github.com/benjamn/ast-types/issues/37).

[RReverser]

@sebmarkbage: Do you think that we should make this a stand-alone document that we refer to as normative? It's technically not necessary to comply with this AST to comply with the spec. It is a normative recommendation though.

It can be published as separate document, yes, but I disagree that it's not necessary to treat is as spec.

If we don't, we can get same mess than was in JS world before Mozilla AST became de-facto standard for AST format and a lot of tools started building around it.

Of course, there might be some transpilers or other tools that might not need to use it, or there might be parsers for other languages that have different AST format. However, for ECMAScript-based parsers and tools this should be strict so any adapted or newly created tool (incl. parsers) could refer to same AST specification and their outputs/inputs would be interoperable.

[RReverser]

@sebmarkbage: We should probably change these to use the JSX prefix instead of the XJS prefix. The XJS prefix was really just a legacy artifact.

I don't like that prefix as well, but changing it requires a lot of changes in ast-types, react, esprima-fb, acorn-jsx, estraverse-fb and maybe other existing tools. Are you sure it's a right time for that?

[RReverser]

@benjamn I'd like to keep this in sync with https://github.com/benjamn/ast-types/blob/master/def/fb-harmony.js, if possible. Even better, we might be able to generate the documentation automatically (see benjamn/ast-types#37).

I like the idea if that's possible. But I don't get XJSText vs Literal battle - why did you write TODO Esprima should return XJSText instead.? Wouldn't it be better to reuse as much types from standard ECMAScript as possible? Or is it just for isString check?

[RReverser]

@benjamn How about adding it as hand-written for now and later just replacing section with auto-generated content from your scripts? (tools section will remain handwritten anyway)

[sebmarkbage]

@RReverser everyone responsible to update those tools are already on this thread. We can just decide to do it right now. A quick search and replace. :)

That way we don't confuse the many other contributors that may come into this later as this grows.

I think we should keep it handwritten so that contributors don't have to get ramped up on another tool chain to contribute to the spec. Generally tooling is downstream from the spec. One less thing to bikeshed.

If the Mozilla AST isn't codegen:ed neither, it seems like it might be useful to write a tool that converts either spec to JS calls.

[RReverser]

@sebmarkbage It's not complete list of tools that depend on existing format. Just a bit more examples:

• https://github.com/Raynos/mercury-jsx
• https://github.com/insin/msx
• https://github.com/lalitkapoor/js-sql
• others

And it's without projects that contain copies of esprima / JSXTransformer inside.

If we do this change, let's at least submit it as major update for our packages so dependencies won't suddenly break.