Feature/ast parser packaging

[andrewemeryanz]

Modifies the package dependency order so that ast no longer depends on parser. This is achieved by:

1. Creating a parse package that houses Scanner and TreeElement.
2. Making the parser package now dependent on ast.

The following public components are affected by a package change:

• parser.Scanner -> parse.Scanner
• parser.NewScanner -> parse.NewScanner
• parser.NewScannerAt -> parse.NewScannerAt
• parser.NewBareScanner -> parse.NewBareScanner
• parser.TreeElement -> parse.TreeElement
• ast.NewExtRefTreeElement -> parser.NewExtRefTreeElement
• ast.FromParserNode -> parser.FromParserNode
• ast.ToParserNode -> parser.ToParserNode

[ghost]

Couple of things:

1. Having on a single letter difference between two exported packages is slightly unsafe (parse and parser), parse should be renamed to reflect that it is just an interface package or something
2. I'm curious as to the reason behind this refactoring? What is the end-goal here?

I was working on a branch on my fork with the intention of having the parser generate the AST directly (removing the TreeElement type), hard to see if that is what you're aiming for or some other end point?

My opinion is that TreeElement should not be an exported type, the AST node types and the Scanner interface should be the exported API for the parser, this means merging the parser and ast packages (which involves fiddly refactoring which is easy to cause conflicts, so coordination).

Probably worth questioning if the Unparse functionality is actually required, I think if that were removed moving the parsers to return the AST nodes directly could be acheiavble.

[andrewemeryanz]

1. Having on a single letter difference between two exported packages is slightly unsafe (parse and parser), parse should be renamed to reflect that it is just an interface package or something

Do you have a suggestion?

2. I'm curious as to the reason behind this refactoring? What is the end-goal here?

The eventual goal is to have a Term reference the ast.Node from which it was parsed. Currently, the ast package depends on the parser package. These changes invert this relationship with minimal disruption.

I was working on a branch on my fork with the intention of having the parser generate the AST directly (removing the TreeElement type), hard to see if that is what you're aiming for or some other end point? My opinion is that TreeElement should not be an exported type, the AST node types and the Scanner interface should be the exported API for the parser, this means merging the parser and ast packages (which involves fiddly refactoring which is easy to cause conflicts, so coordination).

No comment on TreeElement other that it doesn't appear to display clear functionality at present. It was only touched because Scanner is a TreeElement.

[ghost]

The eventual goal is to have a Term reference the ast.Node from which it was parsed.

why? (maybe we should discuss this in slack)

[andrewemeryanz]

The purpose is to aid debugging. There’s currently no way to trace a term back to the source if there’s an error in the grammar. The ast.Node gives us access to the Scanner file position.

[andrewemeryanz]

Closing as unnecessary. Prefer a larger refactor of ast and parser packages.