search

GumTreeDiff / gumtree

An awesome code differencing tool
https://github.com/GumTreeDiff/gumtree/wiki
GNU Lesser General Public License v3.0
932 stars 173 forks source link

Use gumtree API in IntelliJ on Windows #275

Open ztwater opened 2 years ago

ztwater commented 2 years ago

I have added tree-sitter-parser.py to my system path and it could run from cmd. But when I called the API (e.g. JavaTreeSitterTreeGenerator().generateFrom().file(file); ) from IDE or from commands like "gumtree..." I got this error:

java.io.IOException: Cannot run program "tree-sitter-parser.py" (in directory "C:\Users\24426\AppData\Local\Temp"): CreateProcess error=193, %1 is not a valid Win32 application
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
at com.github.gumtreediff.gen.ExternalProcessTreeGenerator.readStandardOutput(ExternalProcessTreeGenerator.java:35)
at com.github.gumtreediff.gen.treesitter.AbstractTreeSitterGenerator.generate(AbstractTreeSitterGenerator.java:37)
at com.github.gumtreediff.gen.TreeGenerator.generateTree(TreeGenerator.java:41)
at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.reader(TreeGenerator.java:119)
at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.file(TreeGenerator.java:90)
at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.file(TreeGenerator.java:100)

Could you please tell me how to solve this error? I guess processbuilder has something to do with the OS...

jrfaller commented 2 years ago

Hi!

I guess the shebang #!/usr/bin/env python does not work on Windows 😄

However I don't know how to get a Python file running, but if you manage to make a bat file don't hesitate to make a PR!

Cheers.

ztwater commented 2 years ago

Thanks a lot! I found that ProcessBuilder can not direct run python scripts, so I changed the arguments of getCommandLine() in AbstractTreeSitterGenerator.java (see here). It is running now but a parse error occurs in TreeIoUtils.java.

while (r.hasNext()) {
    XMLEvent e = r.nextEvent();
    if (e instanceof StartElement) {
javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,1]
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:652)
    at java.xml/com.sun.xml.internal.stream.XMLEventReaderImpl.nextEvent(XMLEventReaderImpl.java:83)
    at com.github.gumtreediff.io.TreeIoUtils$XmlInternalGenerator.generate(TreeIoUtils.java:784)
    at com.github.gumtreediff.gen.TreeGenerator.generateTree(TreeGenerator.java:41)
        ...
jrfaller commented 2 years ago

Can you print the XML being parser to see what the problem is?

ztwater commented 2 years ago

This is the XML tree (from a java code snippet on GitHub) that generated by TreeGenerator:

<?xml version="1.0" ?>
<tree type="program" pos="0" length="598">
    <tree type="type_declaration" pos="0" length="598">
        <tree type="modifiers" pos="0" length="6">
            <tree type="visibility" pos="0" length="6" label="public"/>
        </tree>
        <tree type="type_keyword" pos="7" length="5" label="class"/>
        <tree type="identifier" pos="13" length="3" label="foo"/>
        <tree type="type_body" pos="16" length="582">
            <tree type="method_declaration" pos="18" length="578">
                <tree type="modifiers" pos="18" length="19">
                    <tree type="visibility" pos="18" length="6" label="public"/>
                    <tree type="final" pos="25" length="5" label="final"/>
                    <tree type="static" pos="31" length="6" label="static"/>
                </tree>
                <tree type="type" pos="38" length="6" label="String"/>
                <tree type="identifier" pos="45" length="7" label="uniplus"/>
                <tree type="formal_parameters" pos="52" length="10">
                    <tree type="formal_parameter" pos="53" length="8">
                        <tree type="type" pos="53" length="6" label="String"/>
                        <tree type="identifier" pos="60" length="1" label="s"/>
                    </tree>
                </tree>
                <tree type="block" pos="63" length="533">
                    <tree type="if_statement" pos="69" length="47">
                        <tree type="if" pos="69" length="2" label="if"/>
                        <tree type="parenthesized_expression" pos="72" length="17">
                            <tree type="binary_expression" pos="73" length="15">
                                <tree type="method_invocation" pos="73" length="10">
                                    <tree type="identifier" pos="73" length="1" label="s"/>
                                    <tree type="identifier" pos="75" length="6" label="length"/>
                                    <tree type="argument_list" pos="81" length="2"/>
                                </tree>
                                <tree type="comparison_operator" pos="84" length="2" label="=="/>
                                <tree type="decimal_integer_literal" pos="87" length="1" label="0"/>
                            </tree>
                        </tree>
                        <tree type="block" pos="90" length="26">
                            <tree type="return_statement" pos="100" length="10">
                                <tree type="string_literal" pos="107" length="2" label="&quot;&quot;"/>
                            </tree>
                        </tree>
                    </tree>
                    <tree type="block_comment" pos="121" length="55" label="/* This is just the minimum; sb will grow as needed. */"/>
                    <tree type="local_variable_declaration" pos="181" length="55">
                        <tree type="type" pos="181" length="12" label="StringBuffer"/>
                        <tree type="variable_declarator" pos="194" length="41">
                            <tree type="identifier" pos="194" length="2" label="sb"/>
                            <tree type="affectation_operator" pos="197" length="1" label="="/>
                            <tree type="object_creation_expression" pos="199" length="36">
                                <tree type="new" pos="199" length="3" label="new"/>
                                <tree type="type" pos="203" length="12" label="StringBuffer"/>
                                <tree type="argument_list" pos="215" length="20">
                                    <tree type="binary_expression" pos="216" length="18">
                                        <tree type="decimal_integer_literal" pos="216" length="1" label="2"/>
                                        <tree type="arithmetic_operator" pos="218" length="1" label="+"/>
                                        <tree type="binary_expression" pos="220" length="14">
                                            <tree type="decimal_integer_literal" pos="220" length="1" label="3"/>
                                            <tree type="arithmetic_operator" pos="222" length="1" label="*"/>
                                            <tree type="method_invocation" pos="224" length="10">
                                                <tree type="identifier" pos="224" length="1" label="s"/>
                                                <tree type="identifier" pos="226" length="6" label="length"/>
                                                <tree type="argument_list" pos="232" length="2"/>
                                            </tree>
                                        </tree>
                                    </tree>
                                </tree>
                            </tree>
                        </tree>
                    </tree>
                    <tree type="expression_statement" pos="241" length="16">
                        <tree type="method_invocation" pos="241" length="15">
                            <tree type="identifier" pos="241" length="2" label="sb"/>
                            <tree type="identifier" pos="244" length="6" label="append"/>
                            <tree type="argument_list" pos="250" length="6">
                                <tree type="string_literal" pos="251" length="4" label="&quot;U+&quot;"/>
                            </tree>
                        </tree>
                    </tree>
                    <tree type="for_statement" pos="262" length="306">
                        <tree type="local_variable_declaration" pos="267" length="10">
                            <tree type="type" pos="267" length="3" label="int"/>
                            <tree type="variable_declarator" pos="271" length="5">
                                <tree type="identifier" pos="271" length="1" label="i"/>
                                <tree type="affectation_operator" pos="273" length="1" label="="/>
                                <tree type="decimal_integer_literal" pos="275" length="1" label="0"/>
                            </tree>
                        </tree>
                        <tree type="binary_expression" pos="278" length="14">
                            <tree type="identifier" pos="278" length="1" label="i"/>
                            <tree type="comparison_operator" pos="280" length="1" label="&lt;"/>
                            <tree type="method_invocation" pos="282" length="10">
                                <tree type="identifier" pos="282" length="1" label="s"/>
                                <tree type="identifier" pos="284" length="6" label="length"/>
                                <tree type="argument_list" pos="290" length="2"/>
                            </tree>
                        </tree>
                        <tree type="update_expression" pos="294" length="3">
                            <tree type="identifier" pos="294" length="1" label="i"/>
                            <tree type="increment_operator" pos="295" length="2" label="++"/>
                        </tree>
                        <tree type="block" pos="299" length="269">
                            <tree type="expression_statement" pos="309" length="49">
                                <tree type="method_invocation" pos="309" length="48">
                                    <tree type="identifier" pos="309" length="2" label="sb"/>
                                    <tree type="identifier" pos="312" length="6" label="append"/>
                                    <tree type="argument_list" pos="318" length="39">
                                        <tree type="method_invocation" pos="319" length="37">
                                            <tree type="identifier" pos="319" length="6" label="String"/>
                                            <tree type="identifier" pos="326" length="6" label="format"/>
                                            <tree type="argument_list" pos="332" length="24">
                                                <tree type="string_literal" pos="333" length="4" label="&quot;%X&quot;"/>
                                                <tree type="," pos="337" length="1" label=","/>
                                                <tree type="method_invocation" pos="339" length="16">
                                                    <tree type="identifier" pos="339" length="1" label="s"/>
                                                    <tree type="identifier" pos="341" length="11" label="codePointAt"/>
                                                    <tree type="argument_list" pos="352" length="3">
                                                        <tree type="identifier" pos="353" length="1" label="i"/>
                                                    </tree>
                                                </tree>
                                            </tree>
                                        </tree>
                                    </tree>
                                </tree>
                            </tree>
                            <tree type="if_statement" pos="367" length="125">
                                <tree type="if" pos="367" length="2" label="if"/>
                                <tree type="parenthesized_expression" pos="370" length="40">
                                    <tree type="binary_expression" pos="371" length="38">
                                        <tree type="method_invocation" pos="371" length="16">
                                            <tree type="identifier" pos="371" length="1" label="s"/>
                                            <tree type="identifier" pos="373" length="11" label="codePointAt"/>
                                            <tree type="argument_list" pos="384" length="3">
                                                <tree type="identifier" pos="385" length="1" label="i"/>
                                            </tree>
                                        </tree>
                                        <tree type="comparison_operator" pos="388" length="1" label="&gt;"/>
                                        <tree type="field_access" pos="390" length="19">
                                            <tree type="identifier" pos="390" length="9" label="Character"/>
                                            <tree type="identifier" pos="400" length="9" label="MAX_VALUE"/>
                                        </tree>
                                    </tree>
                                </tree>
                                <tree type="block" pos="411" length="81">
                                    <tree type="expression_statement" pos="425" length="4">
                                        <tree type="update_expression" pos="425" length="3">
                                            <tree type="identifier" pos="425" length="1" label="i"/>
                                            <tree type="increment_operator" pos="426" length="2" label="++"/>
                                        </tree>
                                    </tree>
                                    <tree type="block_comment" pos="430" length="52" label="/****WE HATES UTF-16! WE HATES IT FOREVERSES!!!****/"/>
                                </tree>
                            </tree>
                            <tree type="if_statement" pos="501" length="61">
                                <tree type="if" pos="501" length="2" label="if"/>
                                <tree type="parenthesized_expression" pos="504" length="18">
                                    <tree type="binary_expression" pos="505" length="16">
                                        <tree type="binary_expression" pos="505" length="3">
                                            <tree type="identifier" pos="505" length="1" label="i"/>
                                            <tree type="arithmetic_operator" pos="506" length="1" label="+"/>
                                            <tree type="decimal_integer_literal" pos="507" length="1" label="1"/>
                                        </tree>
                                        <tree type="comparison_operator" pos="509" length="1" label="&lt;"/>
                                        <tree type="method_invocation" pos="511" length="10">
                                            <tree type="identifier" pos="511" length="1" label="s"/>
                                            <tree type="identifier" pos="513" length="6" label="length"/>
                                            <tree type="argument_list" pos="519" length="2"/>
                                        </tree>
                                    </tree>
                                </tree>
                                <tree type="block" pos="523" length="39">
                                    <tree type="expression_statement" pos="537" length="15">
                                        <tree type="method_invocation" pos="537" length="14">
                                            <tree type="identifier" pos="537" length="2" label="sb"/>
                                            <tree type="identifier" pos="540" length="6" label="append"/>
                                            <tree type="argument_list" pos="546" length="5">
                                                <tree type="string_literal" pos="547" length="3" label="&quot;.&quot;"/>
                                            </tree>
                                        </tree>
                                    </tree>
                                </tree>
                            </tree>
                        </tree>
                    </tree>
                    <tree type="return_statement" pos="573" length="21">
                        <tree type="method_invocation" pos="580" length="13">
                            <tree type="identifier" pos="580" length="2" label="sb"/>
                            <tree type="identifier" pos="583" length="8" label="toString"/>
                            <tree type="argument_list" pos="591" length="2"/>
                        </tree>
                    </tree>
                </tree>
            </tree>
        </tree>
    </tree>
</tree>

When I looked into the XMLEventReader in TreeIoUtils.java, I got <?xml version="null" encoding='null' standalone='no'?> as the first element. Strangely, r.hasNext() returns true while I got parse error when I run nextEvent().

jrfaller commented 2 years ago

OK strange, the XML your parsed seems correct