Refactor AST

[mgree]

Pash currently uses ASTs in three forms:

• semi-structured Python lists of the form ["Command", [n.ncmd.linno, to_assigns (n.ncmd.assign), to_args (n.ncmd.args), redirs (n.ncmd.redirect)]]
• Single-level AstNode objects with constructor enum field and variously named arguments
• Hybrids of the two

We should have just one, well structured AST, from libdash on down.

class Command(ABC):
  ...

class SimpleCommand(Command):
  linno: int
  assigns: ...
  args: ...
  redirs: ...

class If(Command):
  cond: Command
  then_b: Command
  else_b: Command

...

class ArgChar(ABC):
  ...

Word = [ArgChar]

class C(ArgChar):
  c: str

class E(ArgChar):
  c: str

class V(ArgChar):
  fmt: ParamterFormat
  null: bool
  var: str
  arg: Optional[Word]

We should use this representation uniformly. We propose the following restructuring:

• [x] create a shasta library that defines Command, ArgChar, with pretty() (for pretty printing as shell) and json() (for rendering as JSON)
• [ ] have libdash depend on shasta and produce appropriate ASTs
• [x] have pash use these ASTs throughout

[angelhof]

PR #673 completes a part of the above requirements, defining classes for all command_nodes (but not for arguments, redirections, etc). It also cleans up many places where there was mixed use of typed and untyped AST objects, only using the typed ones now.

It should be straightforward to extend this to its own separate shasta library, while also defining the rest of the classes now that the initial structure is there. The two modules that will need straightforward modification are:

1. untyped_to_ast.py
2. ast_node.py

Then we would also need to make a few straightforward modifications to ast_to_ast.py and ast_to_ir.py so that they access fields and not indices of the unstructured objects.

[angelhof]

PR #675 extends PaSh to use the new ASTs throughout.

[angelhof]

@mgree I have two questions:

1. When you say a pretty() function, do you mean that this (https://github.com/mgree/libdash/blob/master/libdash/printer.py) should be moved out of libdash and into shasta?
2. Should the module that turns a json ast to an AstNode object (https://github.com/binpash/pash/blob/rest-ast-nodes/compiler/shell_ast/untyped_to_ast.py) be in shasta or somewhere else? It feels to me that it should be in shasta, but I am not sure. Or are you envisioning a shasta that only contains the ast definition?

[mgree]

I'd say in shasta, on both counts!

[angelhof]

Closing this! Will make an issue in libdash to be updated to import shasta, return proper ASTs, and remove the printer.