jchitel commented 6 years ago

Import/Export Additions

Description:

Exports are weird right now because they are sort of tacked on to declarations. They have their own scope for names, which sort of makes sense, but it can be done more intuitively. Also, as of now, exports are the only way to declare constants. It turned out that way because we wanted to be able to export expressions but had no way to declare non-exported expressions. We have an entire framework to support them, so we may as well just do it.

So these are the import forms we need to support:

import from "module": ImportName: import default
import from "module": { {name}, {name} as {alias}, ... }: import named
import from "module": ImportName, { {name}, {name} as {alias}, ... }: import named and default
import from "module": * as {namespace}: wildcard import
import from "module": { {name}, {name} as {alias}, * as {namespace} }: import named and wildcard
import from "module": ImportName, * as {namespace}: default and wildcard import
import from "module": ImportName, { {name}, {name} as {alias}, * as {namespace} }: default, named and wildcard import

Currently 1 and 2 are already supported, and 3-7 are not yet supported.

And these are the export forms we need to support:

export default {named declaration}: export default in place declaration
export default {anon declaration}: export default anonymous declaration
export default {name}: export default already declared declaration
export {named declaration}: export declaration with declaration name
export {name} = {named declaration}: export declaration with another name
export {name} = {expression}: export constant under name
export { {name}, {name} as {alias}, ... }: export already declared declaration(s)

Currently 1 and 3 are already supported, and 2 and 4 are not yet supported. 5 and 6 are currently supported and will be removed. 7 currently exists in a different form for single exports, it will be redone to support multiple exports.

And these are fusion import/export declarations (export forwarding):

export from "module": ExportName: Default from another -> Named from yours
export default from "module": Default from another -> Default from yours
export default from "module": { {name} }: Named from another -> Default from yours
export from "module:" { {name}, {name} as {alias}, ... }: Named from another -> Named from yours
export default from "module": *: All from another -> Default from yours (grouped under a namespace object)
export from "module": * as {name}: All from another -> Named from yours (grouped under a namepsace object)
export from "module": *: All from another -> All from yours (ungrouped, just a 1:1 forward)

Currently none of these are supported.

With the above, all possible kinds of imports and exports should be supported.

New syntactic features:

New Tokens

CONST ::= "const"

New NonTerminals:

ConstantDeclaration ::= CONST IDENT EQUALS Expression
AnonDeclaration ::= AnonTypeDeclaration | AnonFunctionDeclaration | Expression
AnonTypeDeclaration ::= TYPE EQUALS Type
AnonFunctionDeclaration ::= FUNC Type TypeParamList? ParameterList FAT_ARROW (Expression | Block)
NamedExports ::= LBRACE (IDENT | IDENT AS IDENT) (+ sep COMMA) RBRACE
ExportForwards ::= ImportList
                 | MULTIPLY
DefaultExportForwards ::= LBRACE IDENT RBRACE
                        | MULTIPLY

Extended NonTerminals:

Program ::= ImportDeclaration* (Declaration | ExportDeclaration)*
Declaration ::= TypeDeclaration | FunctionDeclaration | ConstantDeclaration
ImportDeclaration ::= IMPORT FROM STRING_LITERAL COLON ImportList
ImportList ::= IDENT
              | NamedImports
              | IDENT COMMA NamedImports
              | MULTIPLY AS IDENT
              | IDENT COMMA MULTIPLY AS IDENT
NamedImports ::= LBRACE (IDENT | IDENT AS IDENT | MULTIPLY AS IDENT) (+ sep COMMA) RBRACE
ExportDeclaration ::= EXPORT DEFAULT (Declaration | AnonDeclaration | IDENT)
                    | EXPORT (Declaration | NamedExports)
                    | EXPORT FROM STRING_LITERAL COLON ExportForwards
                    | EXPORT DEFAULT FROM STRING_LITERAL (COLON DefaultExportForwards)?
Type ::= ... | Type DOT IDENT

New semantic features:

All this basically boils down to the existing framework we have, except wildcards and rest imports, which I will address at the end
The new import types just add imports the same way they did before, the existing framework already supports it
Exports must always have a name. If there is a default export with a named declaration, the declaration is processed normally and the default export is set to that name.
Anonymous default exports are only exposed at typecheck time, not to the developer, under a special identifier for type checking.
Named exports with a separate name are no longer possible unless it is an already declared value, in which case there is an alias that points to the module scoped value.
All named exports are pointers to some other declaration. Export names still have their own scope.
Export forwarding is just sugar for an import combined with an export. The export names are just linked directly to the imported declarations.
Now for wildcards and rest imports.
Wildcard imports will take all exports of a module and group them on a namespace, which is a special sort of readonly struct.
The type of namespaces is not accessible from code, but it does have a type (this may change in the future).
Rest imports work the same way with namespaces, but they have to be more complex because they need to determine which exports haven't been explicitly imported.
If the default export is not explicitly imported, the wildcard will include the default export as a field.

Transformation:

Type checking resolves all exports to a common framework, so transformation will likely need no changes.
Namespace imports may require changes.

jchitel commented 6 years ago

Ok, the syntax (all parser-impl, CST, and AST logic) is done.

The remaining work:

[x] ~~Refactor the TypeChecker to use the new import/export formats~~
[x] ~~Add type checking logic for namespaces~~
[x] Rework translator logic to handle new constant declarations and namespace accesses
[x] Fix/add tests for the new syntax

Things to address:

[x] ~~Because of how translation works, it may be necessary to replace Expression in exports with an actual AnonConstantDeclaration.~~ Yes, this made things way simpler.

jchitel commented 6 years ago

Additional problem:

[x] The results of namespace accesses can have type parameters, so it's no longer just identifier types that can have type arguments. This means we need to rework specific types to be a left-recursive suffix instead of a simple kind of type.

This will actually be nice for several reasons:

The logic for resolving the types of identifiers doesn't need to be duplicated between identifiers and specific types.
It opens the door to be able to easily make other types generic as well (if that makes sense).

jchitel commented 6 years ago

Ok, tests are done and all passing. We could probably use more coverage, but we're fine for the time being. Closing my first feature!

jchitel / renlang