Question about syntax tree

[m-kru]

@kraigher I was loosely thinking about adding support for sources formatting. Right now it does not look like an easy task, because the syntax tree looks more like an Abstract Syntax Tree, not Concrete Syntax Tree and there is no easy access to separators (spaces, tabs, newlines). Such information of course could be retrieved based on the positions, however it wouldn't look like a clean solution (at least for me). I am not a compilers expert, but do you think this is pure AST? It holds information about positions, and AST usually does not contain such information.

Have you ever been thinking about reformatting functionality? Have you had any idea how you would like to achieve this functionality?

[kraigher]

The difficult part is keeping the comments when formatting. For example you could have a comment between 'if' and 'then' so there are a lot of cases to consider. Maybe the solution is to keep the 'if' and 'then' tokens in the AST and associate leading and trailing comments to them. There was support added to associate leading and trailing comments with tokens.

-- leading
if -- trailing

Regarding spacing I should be normalized when formatting the AST anyway so the original spacing is irrelevant. If the original spacing is of interest it can be taken from the positional information if you do not care about tab vs spaces.

[m-kru]

Actually, AST with comments is not an AST anymore, if we want to be accurate. In such case another scenario is to create CST to be used for formatting and language server, and create AST from CST, if someone ever needs pure AST.

[kraigher]

Sure I might have misused the term AST. I have not attached any meaning to the name other than being the data structure for the syntax tree. CST might be a more accurate terminology.

[eine]

Ref #20 #22

[m-kru]

@kraigher one more remark about indentation when generating VHDL from syntax tree. It looks like right now it is also not easy, as nodes for different blocks (for example for While loop) does not hold information about the SrcPos of whole block. I can see 2 possible solutions. Add SrcPos for such blocks or add elsif|else|end|begin tokens to the syntax tree. What do you think, maybe you have already had better idea?

[kraigher]

Yes those tokens need to be added to the CST to know the position. But as I said in the previous comment they need to added also since they may have leading and trailing comments.