Support Kotlin multiplatform builds

[bubenheimer]

I'd like to combine separate Gradle projects for generation of Android client protobuf Java (lite) sources and server protobuf Java sources into a single project via a Kotlin multiplatform Gradle project. Clearly the intention of the protobuf Gradle plugin is to generate sources for different languages & flavors within a single project; it should support this use case. Conceptually it seems the most sensible way to set up Gradle-based protobuf code generation when Android is involved.

Some problems I encountered:

• Single Gradle protobuf block not allowing me to specify different codegen options for builtins & plugins for the various multiplatform targets, in particular option("lite").
• Plugin generates code only for Android build configurations "debug" and "release". No code is generated for the "jvm" multiplatform target (the server protobuf sources).
• Plugin by default compiles protobuf source from "src/main/proto" rather than "src/commonMain/proto" or any of the other actual multiplatform sourcesets.

[vitorhugods]

If someone is looking for a hack around it, here's what I've done in Kalium's protobuf-codegen and protobuf modules:

Protobuf-Codegen

Is a Kotlin/JVM project that contains the .proto files and uses PBandK + this Gradle plugin to generate the Protobuf files.

Protobuf

Is a Kotlin Multiplatform (iOS, JS, JVM and Android), that:

• Will mark :protobuf-codegen:generateProto as a task required in order to compile
• Will get the generated Kotlin/JVM from protobuf-codegen module and move them inside its own sourceset.
• Contains PBandK runtime so it can encode/decode on all platforms.

It's a weird hack, but it works pretty well!

In the end, shared code can just add dependency to project(":protobuf") and use Protobuf (de)serialisation normally.

[Maragues]

@vitorhugods Thanks for that example! It worked for me.

A modification is needed if your KMP project targets Android. Instead of declaring the dependency using a forEach

compileTasks.forEach {

You must use whenTaskAdded

val generateProtoTask = this as GenerateProtoTask

compileTasks.whenTaskAdded {
  dependsOn(generateProtoTask)
}

Otherwise, Android tasks aren't present when GenerateProtoTask is configured and building Android doesn't generate protobuf code.

[Maragues]

This version fixes configuration cache

val copyTask = tasks.register<Copy>("CopyGeneratedProtobuf") {
    val generateProtoTasks = codegenProject.tasks
        .withType(GenerateProtoTask::class.java)
        .matching { !it.isTest }

    dependsOn(generateProtoTasks)

    from(generateProtoTasks)

    val outDirs = generateProtoTasks.flatMap { it.outputSourceDirectorySet.srcDirs }

    outDirs.forEach { generatedDirectory ->
        val targetDirectory = File(generatedFilesBaseDir.get().asFile, generatedDirectory.name)
        into(targetDirectory)
    }

    doLast {
        outDirs.forEach { generatedDirectory ->
            require(generatedDirectory.deleteRecursively()) {
                "Failed to remove contents of ${generatedDirectory.absolutePath}"
            }

        }
    }
}

tasks
    .matching { it is KotlinCompile || it is KotlinNativeCompile }
    .whenTaskAdded {
        dependsOn(copyTask)
    }

I'm sure it can be improved, but it's good enough for me.

[garyp]

Hi, I'm the pbandk author 👋🏼 I'm glad the example code from pbandk was helpful for you @vitorhugods and @Maragues. I just figured out a more elegant way of supporting Kotlin Multiplatform with the protobuf gradle plugin. I have updated the pbandk example code in this commit (it's currently part of a PR that hasn't been merged yet) if you want to see the required changes.

This approach should be applicable for other protobuf plugins as well, not only pbandk. Create a separate gradle sub-project to hold the .proto files and run the protobuf compiler, let's call it :protobuf-codegen. Place the .proto files in the protobuf-codegen/src/main/proto directory and create a protobuf-codegen/build.gradle.kts file with:

import com.google.protobuf.gradle.*

plugins {
    // The protobuf gradle plugin will fail if the project doesn't include the `java-library`, `java`, or one of the
    // Android plugins.
    `java-library`
    // Use the Kotlin Multiplatform plugin to publish the generated Kotlin code as a Multiplatform library rather
    // rather than a Java library.
    kotlin("multiplatform")
    id("com.google.protobuf")
}

val pbandkVersion = "0.14.3"
val protobufVersion = "4.26.1"

kotlin {
    // This project needs to declare the same target platforms that will be used by the consuming projects
    js {
        browser()
    }
    // add other target platforms as needed

    sourceSets {
        commonMain {
            dependencies {
                // Add dependencies required by the generated Kotlin code
                api("pro.streem.pbandk:pbandk-runtime:$pbandkVersion")
            }
        }
    }
}

protobuf {
    protoc {
        artifact = "com.google.protobuf:protoc:$protobufVersion"
    }
    plugins {
        id("pbandk") {
            artifact = "pro.streem.pbandk:protoc-gen-pbandk-jvm:$pbandkVersion:jvm8@jar"
        }
    }
    generateProtoTasks {
        ofSourceSet("main").forEach { task ->
            task.builtins {
                remove("java")
            }
            task.plugins {
                id("pbandk") {
                    // Publish the code generated by pbandk as part of the `:protobuf-codegen` project's
                    // `commonMain` source set. This allows other Kotlin Multiplatform sub-projects to consume the
                    // pbandk-generated Kotlin code using a regular gradle project dependency.
                    val outputDir = task.getOutputDir(this)
                    project.kotlin.sourceSets.commonMain.configure {
                        // `builtBy` ensures that gradle will automatically run the `generateProto` task before trying
                        // to compile the generated Kotlin code
                        kotlin.srcDir(project.files(outputDir).builtBy(task))
                    }
                }
            }
        }
    }
}

tasks {
    compileJava {
        // The protobuf gradle plugin requires this project to apply the `java-library` plugin. But since we're only
        // generating Kotlin code, we need to disable the `compileJava` task. Otherwise gradle will complain that there
        // is no Java code available to compile.
        enabled = false
    }
}

Then the other Kotlin Multiplatform sub-projects that need to consume the generated Kotlin code can do so via a regular project dependency with no additional configuration required in the consuming project. For example, if you have a Kotlin Multiplatform sub-project named :app, then app/build.gradle.kts can declare an implementation(project(":protobuf-codegen")) dependency in its commonMain source set in order to make use of the generated protobuf code.

[AkramBensalem]

When I add android library plugin, It doesn't generate the code