search

GumTreeDiff / gumtree

An awesome code differencing tool
https://github.com/GumTreeDiff/gumtree/wiki
GNU Lesser General Public License v3.0
933 stars 174 forks source link

Including PLSQL #359

Open MariusKochFHDW opened 5 months ago

MariusKochFHDW commented 5 months ago

Hey there,

i was just set on a new Project to find a difftool for PLSQL and i recently found gumtree. I already have the antlr4 grammar and parser for PLSQL and have a python script to generate an AST. I already seen the tree-sitter but can't really figure out how to do it.

Thank you in advance :)

jrfaller commented 5 months ago

Hi! Can you find the parser you want here : https://github.com/bonede/tree-sitter-ng ? Cheers.

MariusKochFHDW commented 5 months ago

No, unfortunatly there isn't any supported parser for this programming language.

jrfaller commented 5 months ago

Is there any tree-sitter parser you know that might do the trick ?

MariusKochFHDW commented 5 months ago

I dont see any. Is there another trick i can use to integrate it?

jrfaller commented 5 months ago

so maybe you try the antlr-4 integration but I have never really used it seriously

MariusKochFHDW commented 5 months ago

Where can i find it?

jrfaller commented 4 months ago

We have a helper here -> https://github.com/GumTreeDiff/gumtree/blob/main/gen.antlr4/src/main/java/com/github/gumtreediff/gen/antlr4/AbstractAntlr4TreeGenerator.java But I have no sample project to showcase how to use it 😢

MariusKochFHDW commented 4 months ago

Thank you for the response :) I will have a look at it and tell you if it works.

Hoping for the best 😄

jrfaller commented 4 months ago

Maybe look at how we do for antlr 3 projects like this -> https://github.com/GumTreeDiff/gumtree/tree/main/gen.antlr3-r

MariusKochFHDW commented 4 months ago

Hey there again!

I just tried a bunch of stuff and came to the conclusion that the best/easiest way for me to implement PLSQL into Gumtree is the following:

  1. I'm using the antlr4 Lexer and Parser to generate a AST in the form of the antlr4
  2. From there i would love to just send it to either the API or a Method that can handle the antlr4 AST and format it into the XML format gumtree can handle.

Is there some kind of Method or API call that does the trick? Im pretty new to coding so i hope i didnt overlooked anything importent. Thankss :)

jrfaller commented 4 months ago

Hi! Normally if you have a antlr4 grammar you should be able to skip the XML part and use directly the antlr generated objects to convert them to a GumTree AST it would be much more efficient?

MariusKochFHDW commented 4 months ago

Hello again, i just cant find any way to get it running. FYI: I'm producing a AST in python for PLSQL with a grammar i had to optimize a little bit. I tried to run the python script within the gumtree webdiff -x build_tree.py build_tree2.py. I gave it another shot with trying to use the -... g antlr-antlr example.pkh. It starts the server but there isn't any diff that happend.

Can you give me a detailed explaination how to use the convert the generated objects to GumTree ASTs and then have a detailed Diff for them? Thank you in advance :))

jrfaller commented 4 months ago

Hi! Once you have converted your parser specific format to GumTree's agnostic format (a Tree and a TreeContext) every other things come for free (such as the diff :D). For antlr3 grammars it is done via subclassing our helper class like here : https://github.com/GumTreeDiff/gumtree/blob/main/gen.antlr3-r/src/main/java/com/github/gumtreediff/gen/antlr3/r/RTreeGenerator.java . I have no such example for antlr4 grammars but as far as I can see you have 3 methods to override here https://github.com/GumTreeDiff/gumtree/blob/main/gen.antlr4/src/main/java/com/github/gumtreediff/gen/antlr4/AbstractAntlr4TreeGenerator.java (and also don't forget to add the annotation to register the parser, as done in the antlr3 example). For the webdiff command it's trickier because you have to include the antlr4 projects in the distribution here : https://github.com/GumTreeDiff/gumtree/blob/main/dist/build.gradle . Sorry this a lot of stuff to do 😢

MariusKochFHDW commented 4 months ago

Hey man, almost everything done! I created a antlr3 folder and put an old grammar in there and registered everything. It works just like i hoped. The only issue is, that for bigger files (200KB+) the hilighting is getting offset to the text it's marking :( Is there any way to fix it? Otherwise it's something we can live with :)

Thank you a lot for your help Cheers!

jrfaller commented 4 months ago

Hi! Very good news! It's strange that you have offsetting problems I think it might come from the parser or the helper. It requires some debugging to understand why the position is wrong (fairly easy to do with a breakpoint if you know the grammar well). Cheers!