search

xsawyerx / guacamole

Guacamole is a parser toolkit for Standard Perl. It provides fully static BNF-based parsing capability to a reasonable subset of Perl.
https://metacpan.org/pod/Guacamole
20 stars 8 forks source link

WIP: Include the location (position, line, column) of each element #75

Closed xsawyerx closed 4 years ago

xsawyerx commented 4 years ago

Marpa gives the position of each element. I then use Marpa to add the line number and column number of the element.

This is still not working because the cleanup phase doesn't deal with it well. I don't understand the cleanup phase. :/

@vickentiy, I might need your help here.

xsawyerx commented 4 years ago

We still need to adjust AST.pm. I'll take a stab at it, but I can't promise I'll succeed. :)

vickenty commented 4 years ago

I think you can just chuck all cleanup code in AST.pm. Most of the code there doesn't work any more, since we renamed a lot of stuff, some is just wrong (fold_ident). The only thing that still works is the one that removes parens and braces from nodes to make dumps look nicer, but this isn't strictly necessary.

vickenty commented 4 years ago

It might be also nice to keep track end positions in addition to start positions. You could draw squiggly lines under the errors instead just pointing at the beginning. Also when serializing, it allows to preserve whitespace and comments between tokens without adding them to the parse like PPI.

xsawyerx commented 4 years ago

There isn't an "end", but there's "start" and "length" which would achieve the "end." I'll add it.

I'll try to remove the AST.pm file and see what happens.

xsawyerx commented 4 years ago

Removed AST.pm entirely. I also completely restructured the data we get back. Instead of an array to array to array, it's now a structure of a hash with type, name, locations, and children. (Perhaps we should move this to a class.)

In any case, Dumper.pm now breaks and I don't understand it well enough to fix.

xsawyerx commented 4 years ago

I fixed the Dumper.