java.lang.ArrayIndexOutOfBoundsException during parsing

[VitaSokolova]

Hi! I tried to parse a very simple file: class Sample (private val propertyA: String)

using the code from example

fun main() {
    val source = AstSource.File(
        "src/jvmMain/kotlin/sample/Sample.kt"
    )
    val kotlinFile = KotlinGrammarAntlrKotlinParser.parseKotlinFile(source)
    kotlinFile.summary()
        .onSuccess { astList ->
            astList.forEach(Ast::print)
        }.onFailure { errors ->
            errors.forEach(::println)
        }
}

And got such error

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 170
    at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinAstParserImpl.toAstTerminal(AntlrKotlinAstParserImpl.kt:27)
    at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinAstParserImpl.parse(AntlrKotlinAstParserImpl.kt:82)
    at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinAstParserImpl.parse(AntlrKotlinAstParserImpl.kt:75)
    at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinAstParserImplKt.antlrKotlinParser(AntlrKotlinAstParserImpl.kt:138)
    at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinAstParserImplKt.antlrKotlinParser(AntlrKotlinAstParserImpl.kt:98)
    at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinParser.parse(AntlrKotlinParser.kt:20)
    at kotlinx.ast.grammar.kotlin.common.KotlinGrammarParser$DefaultImpls.parseKotlinFile(KotlinGrammarParser.kt:9)
    at kotlinx.ast.grammar.kotlin.target.antlr.kotlin.KotlinGrammarAntlrKotlinParser.parseKotlinFile(KotlinGrammarAntlrKotlinParser.kt:10)
    at sample.MainKt.main(Main.kt:14)
    at sample.MainKt.main(Main.kt)

[drieks]

Hi @VitaSokolova,

thank you for reporting this!

I can confirm this Bug when using AstSource.File:

AstSource.File("src/jvmMain/kotlin/sample/Sample.kt")

but using AstSource.String seems to work:

AstSource.String("class Sample (private val propertyA: String)")

It is caused by this bug: https://github.com/Strumenta/antlr-kotlin/issues/38 I added a Workaround to kotlinx.ast, please try the current master version, it should work now as expected.

[VitaSokolova]

I'm not so good at working with jitpack, does master branch deploy look like this: api 'com.github.kotlinx.ast:grammar-kotlin-parser-antlr-kotlin:master' ?

[VitaSokolova]

According to jitpack the last build (39d99c7b4b) had failed (

[drieks]

Hi,

you can use the same way you used to run the example. You can ignore the checks linked here https://github.com/kotlinx/ast/commit/39d99c7b4bd90cdcebe190fadb3e03f915b2f815, these checks are not yet correctly configured.

You can find the jitpacks builds here: https://jitpack.io/#drieks/antlr-kotlin

syntax for build.gradle:

allprojects {
        repositories {
            ...
            maven { url 'https://jitpack.io' }
        }
    }
dependencies {
            implementation 'com.github.drieks:grammar-kotlin-parser-antlr-kotlin:9fde270e3a'
    }

Feel free to paste your build.gradle and settings.gradle files here if you need more help! There are some different ways to setup jitpack.

You can't use a string like "master", because in gradle and maven, you always have to supply a "stable" version string, and "master" can change every time. You can see the lastest version string in jitpack, currently "9fde270e3a".

"grammer-kotlin" means "this dependency will parse kotlin code", "parser-antlr-kotlin" means "antlr-kotlin is used for this". If you see an exception, you can also try "grammar-kotlin-parser-antlr-java", this will also parse kotlin code, but using antlr-java for this. The API is the same, only the import statement is "kotlinx.ast.grammar.kotlin.target.antlr.java.KotlinGrammarAntlrJavaParser", not "import kotlinx.ast.grammar.kotlin.target.antlr.kotlin.KotlinGrammarAntlrKotlinParser".

[drieks]

Sorry, you can use "master-SNAPSHOT" as the version, "SNAPSHOT" will tell gradle that the version can change.

You are right, version 39d99c7b4b is broken, sorry. (I looked at the antlr-kotlin repository..) I will try to fix the build this evening. How do you run the example? Did you create a small project and added kotlinx.ast?

[VitaSokolova]

I pushed my small project, you can find it there https://github.com/VitaSokolova/ImpactAnalysisProject

[drieks]

Hi @VitaSokolova, sorry for the late answer. The build of antlr-kotlin was broken, the version was missing so that the default version was added to gradle metadata. That caused the build of kotlinx.ast to fail. I fixed the kotlinx.ast build here: https://github.com/drieks/antlr-kotlin/commit/b09d76328f7d8c38417174d5621bbdfb52b6405e and the build of kotlinx.ast here: https://github.com/kotlinx/ast/commit/5db757217c18432c65533b943ef36aba650639cc

When you replace this line:

api 'com.github.kotlinx.ast:grammar-kotlin-parser-antlr-kotlin:master'

with:

api 'com.github.kotlinx.ast:grammar-kotlin-parser-antlr-kotlin:5db757217c'

your project should work now.

[VitaSokolova]

Hi, I will upset you again( but I faced the same issue on com.github.kotlinx.ast:grammar-kotlin-parser-antlr-kotlin:5db757217c version. You can find an example to run and proof it in my repository

[drieks]

Hi @VitaSokolova,

your project works fine for me when running in Intellij Idea.

I added a "shadow jar" task to gradle: https://github.com/VitaSokolova/ImpactAnalysisProject/pull/1 (A shadow jar contains all dependencies required to run program)

I also added this to gradle (I don't know why it is required, it is releated to gradle metadata and should normally be automatically included):

jvmMain {
            dependencies {
                api 'com.github.kotlinx.ast:grammar-kotlin-parser-antlr-kotlin-jvm:5db757217c'
            }
        }

You can now run "gradle clean buildJar" from the command line, or by using the "gradle" tab in Idea, you can click on "ImpactAnalysisProject / Tasks / other / buildJar". The jar will be created as build/libs/ImpactAnalysisProject.jar.

Please try to run this jar:

java -jar build/libs/ImpactAnalysisProject.jar

This will create this output for me:

java -jar build/libs/ImpactAnalysisProject.jar
PackageHeader(sample)
importList
KlassDeclaration(class Sample)
  KlassDeclaration(constructor)
    KlassDeclaration(val propertyA String)
      KlassModifier(private, visibilityModifier)

If this does not work, please copy the jar into the root folder of the repository and just commit it, I will look into the jar to find the problem.