OkBuck throws exception with Kotlin 1.3.20

[ZacSweers]

Caused by: java.lang.IllegalStateException: No valid configuration found for 'kapt' in project 'project ':tooling:intellij:intellij-patcher''
        at com.uber.okbuck.core.annotation.AnnotationProcessorCache.getConfiguration(AnnotationProcessorCache.java:183)
        at com.uber.okbuck.core.annotation.AnnotationProcessorCache.hasEmptyAnnotationProcessors(AnnotationProcessorCache.java:117)
        at com.uber.okbuck.core.model.jvm.JvmTarget.getAptScopeForConfiguration(JvmTarget.java:129)
        at com.uber.okbuck.core.model.jvm.JvmTarget.getApt(JvmTarget.java:117)
        at com.uber.okbuck.composer.jvm.JvmLibraryRuleComposer.compose(JvmLibraryRuleComposer.java:29)
        at com.uber.okbuck.generator.BuckFileGenerator.createRules(BuckFileGenerator.java:122)
        at com.uber.okbuck.generator.BuckFileGenerator.lambda$createRules$1(BuckFileGenerator.java:81)
        at com.google.common.collect.SingletonImmutableBiMap.forEach(SingletonImmutableBiMap.java:65)
        at com.uber.okbuck.generator.BuckFileGenerator.createRules(BuckFileGenerator.java:73)
        at com.uber.okbuck.generator.BuckFileGenerator.generate(BuckFileGenerator.java:50)
        at com.uber.okbuck.OkBuckGradlePlugin.lambda$apply$7(OkBuckGradlePlugin.java:310)

The said project doesn't use kapt or annotationProcessor

apply plugin: "java-library"
apply plugin: "org.jetbrains.kotlin.jvm"
apply plugin: "application"

dependencies {
    implementation deps.build.commonsCodec
    implementation deps.build.commonsIo
    implementation deps.build.javassist
    implementation deps.build.ztZip
    implementation deps.kotlin.stdLib
    implementation deps.kotlin.stdLibJdk8

    testImplementation project(":libraries:foundation:test:test-core")
}

mainClassName = "com.ubercab.intellijpatcher.IntellijPatcher"

run {
    args System.getProperty("exec.args", "").split(",")
}

[artem-zinnatullin]

mainClassName = "com.ubercab.intellijpatcher.IntellijPatcher"

lmao

we have this

mainClassName = 'com.lyft.buck.imlfixer.MainKt'

[kageiit]

Im trying to switch us over to download kotlin compiler from github and hit an interesting issue - https://youtrack.jetbrains.com/issue/KT-29837

Suggestions welcome at this point, but im not willing to accept setting up a separate artifact mirror for the compiler zips ourselves

[ZacSweers]

What does the gradle plugin do?

[ZacSweers]

Specifically - I'd be surprised if consuming from github is the only way to make this work.

Context from our internal ticket details:

---

Kotlin Compiler release zip contains all the artifacts that is needed by the compiler classpath. (https://github.com/JetBrains/kotlin/releases/download/v1.3.20/kotlin-compiler-1.3.20.zip)

This zip compiler setup is used by buck & a bunch of other companies. We need to make okbuck use this zip as kotlin_home instead of using a custom generation by using specific compiler jar and its dependencies.

Work that needs to be done to achieve this:

• Make okbuck use kotlin compiler zip (https://github.com/JetBrains/kotlin/releases/download/v1.3.20/kotlin-compiler-1.3.20.zip). Okbuck creates the http_archive rule which will be used to download the zip based on the kotlin compiler version. This will also decouple kotlin compiler and kotlin code version and make them independent of each other.
• Use the jar from the kotlin-compiler output, like kotlin-allopen etc. (note kotlin-allopen was renamed to allopen-compiler-plugin in 1.3.20)
• Re-revert this commit (https://github.com/facebook/buck/commit/01ae6e7c43d35a55118622d7ab91e087008e852f) in buck and make any required changes to how the Kotlinc object is setup with the kotlin compiler zip. Also, add all the jars in the <zip>/lib folder (or only the required ones) to the classpath of the compiler. Kotlin 1.3.20 adds a new jar trove4j.jar which requires a buck upgrade. Adding all jar's in the zip will ensure that newer version of kotlin won't fail on old version of buck.

this will unblock: gradle upgrade to 5.1: AGP upgrade to 3.3.0: https://github.com/uber/okbuck/pull/796 Kotlin Upgrade to 1.3.20: https://github.com/uber/okbuck/pull/802

[ZacSweers]

It seems to me like the real issue is more that buck is using this zip, when it maybe shouldn't?

[kageiit]

Using the zip is the right solution because as we have seen with this issue

• Newer kotlin versions require new jars to be present in kotlin_home/lib when running kotlinc due to refactors/new dependencies in upstream kotlin code
• Using the zip would ensure all the necessary deps for every kotlin update are in sync and we dont have to guess which deps to supply and materialize on the filesystem
• There are changes to be made on buck also, but for inorder to buck to download the kotlin compiler zip via a http_rule, okbuck needs to put in the sha256sum of the zip artifact in the generated BUCK file. If okbuck cannot download this file via the regular dependency mechanisms, we cannot calculate this checksum. If we download it through a simple download task, it gets cleared whenever gradle clean is run and would need to be redownloaded again which makes little sense as everything else is cached in GRADLE_USER_HOME/caches

[ZacSweers]

The kotlin gradle plugin must be able to deal with this in a way that plays nice with the gradle/maven ecosystem right?

[kageiit]

Im not sure about how the gradle plugin does things. We are only interested in the cli compiler. The gradle plugin probably has additional plumbing. The cli has to rely on file paths etc. to find its deps. We want to follow their recommendation to use the zip from https://kotlinlang.org/docs/tutorials/command-line.html

[kageiit]

I think I found a creative solution to the problem without needing to modify buck to switch it to the compiler zip artifact and to support future kotlin versions by keeping changes in okbuck only. I'll upload the pr today

I'll make a separate change to buck as a follow-up since it's better to have it for folks that don't use okbuck