Unhandeld Javaparser Exceptions

[jan-gerling]

In rare cases, the call to SourceCodeUtils.removeComments can throw a ParseException:

How to deal with these classes?

All found examples (31):

• com.github.javaparser.ParseProblemException: (line 93,col 20) Parse error. Found ".", expected "("
• com.github.javaparser.ParseProblemException: Parse error. Found "package", expected one of ";" "@" "\u001a" "abstract" "class" "default" "enum" "final" "import" "interface" "module" "native" "open" "private" "protected" "public" "static" "strictfp" "synchronized" "transient" "transitive" "volatile"
• com.github.javaparser.ParseProblemException: (line 127,col 115) Parse error. Found ";", expected one of "!=" "%" "%=" "&" "&&" "&=" ")" "" "=" "+" "+=" "-" "-=" "->" "/" "/=" "::" "<" "«=" "<=" "=" "==" ">" ">=" "»=" "»>=" "?" "^" "^=" "instanceof" "|" "|=" "||"
• com.github.javaparser.ParseProblemException: (line 88,col 10) Parse error. Found "/", expected one of "enum" "exports" "module" "open" "opens" "provides" "requires" "strictfp" "to" "transitive" "uses" "with"
• com.github.javaparser.ParseProblemException: (line 2253,col 12) Parse error. Found "(", expected one of "enum" "exports" "module" "open" "opens" "provides" "requires" "strictfp" "to" "transitive" "uses" "with"
• com.github.javaparser.ParseProblemException: Parse error. Found "package", expected one of ";" "@" "\u001a" "abstract" "class" "default" "enum" "final" "import" "interface" "module" "native" "open" "private" "protected" "public" "static" "strictfp" "synchronized" "transient" "transitive" "volatile"
• com.github.javaparser.ParseProblemException: Lexical error at line 39, column 68. Encountered: "\n" (10), after : "\"After appending all elements of Vector,"
• com.github.javaparser.ParseProblemException: (line 101,col 9) Parse error. Found "@", expected one of "(" "enum" "exports" "false" "module" "new" "null" "open" "opens" "provides" "requires" "strictfp" "super" "this" "to" "transitive" "true" "uses" "with"
• com.github.javaparser.ParseProblemException: Lexical error at line 186, column 2. Encountered: "#" (35), after : ""
• com.github.javaparser.ParseProblemException: Lexical error at line 186, column 2. Encountered: "#" (35), after : ""
• com.github.javaparser.ParseProblemException: Lexical error at line 29, column 2. Encountered: "#" (35), after : ""
• com.github.javaparser.ParseProblemException: Lexical error at line 203, column 2. Encountered: "#" (35), after : ""
• com.github.javaparser.ParseProblemException: Lexical error at line 213, column 2. Encountered: "#" (35), after : ""
• com.github.javaparser.ParseProblemException: Lexical error at line 65, column 2. Encountered: "#" (35), after : ""
• com.github.javaparser.ParseProblemException: (line 17,col 6) Parse error. Found ".", expected one of "%=" "&=" "(" "*=" "++" "+=" "--" "-=" "/=" ";" "«=" "=" "»=" "»>=" "^=" "|="
• com.github.javaparser.ParseProblemException: (line 34,col 35) Parse error. Found ")", expected one of "%=" "&=" "*=" "++" "+=" "--" "-=" "/=" ";" "«=" "=" "»=" "»>=" "^=" "|="
• com.github.javaparser.ParseProblemException: (line 60,col 32) Parse error. Found ".", expected one of "!=" "%=" "&" "&&" "&=" ")" "*=" "+=" "-=" "->" "/=" "::" "«=" "=" "==" "»=" "»>=" "?" "^" "^=" "|" "|=" "||"
• com.github.javaparser.ParseProblemException: (line 82,col 67) Parse error. Found "nooch" , expected "}"
• com.github.javaparser.ParseProblemException: (line 81,col 67) Parse error. Found "nooch" , expected "}"
• com.github.javaparser.ParseProblemException: (line 46,col 4) Parse error. Found "nooch" , expected "}"
• com.github.javaparser.ParseProblemException: (line 17,col 113) Parse error. Found ",", expected "}"
• com.github.javaparser.ParseProblemException: (line 123,col 10) Parse error. Found "extends", expected one of "enum" "exports" "module" "open" "opens" "provides" "requires" "strictfp" "to" "transitive" "uses" "with"
• com.github.javaparser.ParseProblemException: (line 55,col 5) Parse error. Found "«", expected one of ";" "<" "@" "abstract" "boolean" "byte" "char" "class" "default" "double" "enum" "exports" "final" "float" "int" "interface" "long" "module" "native" "open" "opens" "private" "protected" "provides" "public" "requires" "short" "static" "strictfp" "synchronized" "to" "transient" "transitive" "uses" "void" "volatile" "with" "{" "}"
• com.github.javaparser.ParseProblemException: (line 23,col 33) Parse error. Found ",", expected one of "%=" "&=" "*=" "++" "+=" "--" "-=" "/=" ";" "«=" "=" "»=" "»>=" "^=" "|="
• com.github.javaparser.ParseProblemException: (line 11,col 30) Parse error. Found "«", expected one of ";" "<" "@" "abstract" "boolean" "byte" "char" "class" "default" "double" "enum" "exports" "final" "float" "int" "interface" "long" "module" "native" "open" "opens" "private" "protected" "provides" "public" "requires" "short" "static" "strictfp" "synchronized" "to" "transient" "transitive" "uses" "void" "volatile" "with" "{" "}"
• com.github.javaparser.ParseProblemException: Lexical error at line 18, column 30. Encountered: "\ufffd" (65533), after : ""
• com.github.javaparser.ParseProblemException: Lexical error at line 18, column 30. Encountered: "\ufffd" (65533), after : ""
• com.github.javaparser.ParseProblemException: (line 18,col 24) Parse error. Found "public", expected "("
• com.github.javaparser.ParseProblemException: (line 76,col 60) Parse error. Found "}", expected one of "!=" "%" "%=" "&" "&&" "&=" "" "=" "+" "+=" "," "-" "-=" "->" "/" "/=" "::" ";" "<" "«=" "<=" "=" "==" ">" ">=" "»=" "»>=" "?" "^" "^=" "instanceof" "|" "|=" "||"
• com.github.javaparser.ParseProblemException: (line 102,col 5) Parse error. Found "}", expected one of ";" "@" "\u001a" "abstract" "class" "default" "enum" "final" "import" "interface" "module" "native" "open" "private" "protected" "public" "static" "strictfp" "synchronized" "transient" "transitive" "volatile"
• com.github.javaparser.ParseProblemException: (line 19,col 55) Parse error. Found "«", expected one of ";" "@" "\u001a" "abstract" "class" "default" "enum" "final" "import" "interface" "module" "native" "open" "private" "protected" "public" "static" "strictfp" "synchronized" "transient" "transitive" "volatile"

[mauricioaniche]

Maybe we can fall back to a String manipulation method if JavaParser fails? CK has an utility method that counts lines looking at the stringified code only.

On Fri, 6 Mar 2020 at 19:44, Jan Gerling notifications@github.com wrote:

In rare cases, the call to SourceCodeUtils.removeComments https://github.com/refactoring-ai/predicting-refactoring-ml/blob/9210fa3e1b52fa156e7d80730c65619f8292abfe/data-collection/src/main/java/refactoringml/util/SourceCodeUtils.java#L8 can throw a ParseException: [image: image] https://user-images.githubusercontent.com/29139613/76112522-ab55dc80-5fe2-11ea-9de1-f22d23aff930.png

How to deal with these classes?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/refactoring-ai/predicting-refactoring-ml/issues/94?email_source=notifications&email_token=AAAYTTD7VEVYFUYDYQCFAQLRGE77JA5CNFSM4LDFYCH2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4ITF7KEA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAYTTCG5WUFIRNAFOJ25DDRGE77JANCNFSM4LDFYCHQ .

--

Maurício Aniche Delft University of Technology http://www.mauricioaniche.com

[mauricioaniche]

As discussed:

• Completely remove SourceCodeUtils.removeComments from our source code
• CK will implement SLOC as metric, and we can use it. (@mauricioaniche)