kaby76 / Trash

Toolkit for grammars
MIT License
76 stars 5 forks source link

bug: unexplained error when transforming large grammar #30

Open SKalt opened 3 years ago

SKalt commented 3 years ago

I pointed the sharp end of trgroup v0.11.2 at the trconverted postgres grammar, and I got

System.Exception: Exception of type 'System.Exception' was thrown.
   at NWayDiff.Classical`1.classical_lcs(List`1 a, List`1 b, Int32 i, Int32 j, Dictionary`2 memo)
   at NWayDiff.Difdef_impl`1.add_vec_to_diff_classical(Diff`1& a, Int32 fieldid, List`1 b)
   at NWayDiff.Difdef_impl`1.add_vec_to_diff(Diff`1& a, Int32 fileid, List`1 b)
   at NWayDiff.Difdef_impl`1.add_vec_to_diff(Diff`1& a, Int32 fileid, List`1 b)
   at NWayDiff.Difdef_impl`1.merge(Int32 fmask)
   at NWayDiff.Difdef`1.merge()
   at LanguageServer.Transform.Group(List`1 nodes, Document document)
   at Trash.CGroup.Execute(Config config) in C:\Users\kenne\Documents\GitHub\Domemtech.Trash\trgroup\CGroup.cs:line 84
   at Trash.Program.MainInternal(String[] args) in C:\Users\kenne\Documents\GitHub\Domemtech.Trash\trgroup\Program.cs:line 68
   at Trash.Program.Main(String[] args) in C:\Users\kenne\Documents\GitHub\Domemtech.Trash\trgroup\Program.cs:line 14

I'm unable to parse what's going wrong. Also, it's worth noting the System.Exception did not halt the program in a way that exited my grammar-trasnform pipeline despite having set -eo pipefail in my shell. trgroup ran for 10+ minutes with low/no CPU consumption after the error occurred.

Do you know what's happening here?

kaby76 commented 3 years ago

Where is the pre-converted grammar?

But, it doesn't work for either grammar at grammars-v4/sql/postgresql. For the parser grammar, it crashes here, which is definitely wrong code. I need to refamiliarize myself with this unification code.

SKalt commented 3 years ago

You can create the grammar I'm converting by

curl -L https://raw.githubusercontent.com/postgres/postgres/master/src/backend/parser/gram.y > /tmp/gram.y
trparse /tmp/gram.y | trconvert | trprint > /tmp/gram.g4
trparse /tmp/gram.g4 | trgroup | trprint > /tmp/done.g4
# trprint is required since piping from trparse $x.y | trconvert | other_trash_command
# seems to be missing nodes, according to error messages that I forgot how to reproduce
# maybe piping to trrename or trkleene would do it? 
kaby76 commented 3 years ago

Ok, I will try that after I fix the LCS diffing algorithm. It's broken in one place as I noted above.