:run and :runShadow show different behavior

[curtisullerich]

Please check the User Guide before submitting "how do I do 'x'?" questions!

Shadow Version

7.1.2

Gradle Version

7.4.2

Expected Behavior

A simple gRPC client is able to issue an RPC when run with both ./gradlew run and ./gradlew runShadow

Actual Behavior

with runShadow, the RPC attempt fails with this exception:

Exception in thread "main" io.grpc.StatusException: UNKNOWN
        at io.grpc.Status.asException(Status.java:550)
        at io.grpc.kotlin.ClientCalls$rpcImpl$1$1$1.onClose(ClientCalls.kt:296)
        at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:562)
        at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:743)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:722)
        at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
        at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.nio.channels.UnsupportedAddressTypeException
        at java.base/sun.nio.ch.Net.checkAddress(Net.java:146)
        at java.base/sun.nio.ch.Net.checkAddress(Net.java:157)
        at java.base/sun.nio.ch.SocketChannelImpl.checkRemote(SocketChannelImpl.java:816)
        at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:839)
        at io.netty.util.internal.SocketUtils$3.run(SocketUtils.java:91)
        at io.netty.util.internal.SocketUtils$3.run(SocketUtils.java:88)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)
        at io.netty.util.internal.SocketUtils.connect(SocketUtils.java:88)
        at io.netty.channel.socket.nio.NioSocketChannel.doConnect(NioSocketChannel.java:322)
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:248)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1342)
        at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:548)
        at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:533)
        at io.netty.channel.ChannelDuplexHandler.connect(ChannelDuplexHandler.java:54)
        at io.grpc.netty.WriteBufferingAndExceptionHandler.connect(WriteBufferingAndExceptionHandler.java:157)
        at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:548)
        at io.netty.channel.AbstractChannelHandlerContext.access$1000(AbstractChannelHandlerContext.java:61)
        at io.netty.channel.AbstractChannelHandlerContext$9.run(AbstractChannelHandlerContext.java:538)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

Gradle Build Script(s)

import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
import com.google.protobuf.gradle.*

plugins {
    application
    kotlin("jvm") version "1.7.10"
    id("com.github.johnrengelman.shadow") version "7.1.2"
    id("com.google.protobuf") version "0.8.19"
}

// These address this error:
// 'compileJava' task (current target is 17) and 'compileKotlin' task (current target is 1.8) jvm target compatibility
// should be set to the same Java version.
val javaVersion = JavaVersion.VERSION_17
java { sourceCompatibility = javaVersion; targetCompatibility = javaVersion }
tasks.withType<KotlinCompile> { kotlinOptions { jvmTarget = javaVersion.toString() } }

group = "com.test"
version = "0.1"

repositories { mavenCentral() }

dependencies {
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.3")
    implementation("io.grpc:grpc-protobuf:1.48.1")
    implementation("io.grpc:grpc-kotlin-stub:1.3.0")

    // The RPC fails only when I do :runShadow. :run works. The stacktrace is all in netty, but when I compile without
    // netty there is no netty in the jar, making me think there is no transitive dependency on netty. It fails with
    // both netty and netty-shaded, but with different stack traces.
    // With netty-shaded, it says:
    // Caused by: io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AnnotatedConnectException: connect(..) failed: Address family not supported by protocol: /localhost:8980
    // With netty, it just says:
    // Caused by: java.nio.channels.UnsupportedAddressTypeException
    implementation("io.grpc:grpc-netty:1.48.1")
    implementation("com.google.protobuf:protobuf-kotlin:3.21.4")
}

// for gRPC
protobuf {
    protoc { artifact = "com.google.protobuf:protoc:3.21.4" }
    plugins {
        // We only need this for Kotlin, but it doesn't fully link without the Java sources.
        id("grpc") { artifact = "io.grpc:protoc-gen-grpc-java:1.48.1" }
        id("grpckt") { artifact = "io.grpc:protoc-gen-grpc-kotlin:1.3.0:jdk8@jar" }
    }
    generateProtoTasks {
        all().forEach {
            it.plugins { id("grpc"); id("grpckt") }
            // The plugins above (I think) generate the Java proto code, but not the extra Kotlin stuff like DSLs.
            // Including this adds a /kotlin folder to the generated code which contains all the extra stuff.
            it.builtins { id("kotlin") }
        }
    }
}

sourceSets {
    main {
        java {
            // These are the default generated source dirs, but they have to be added manually currently:
            // https://github.com/google/protobuf-gradle-plugin/issues/109
            // The plugin generates both a grpc dir with Java files, and grpckt with Kotlin files. I believe the grpckt
            // contents includes all the functionality of the grpc (java) contents, but I'm not positive. The kotlin
            // files might just be stubs for the java version? so maybe this compiles but doesn't run?
            srcDirs("build/generated/source/proto/main/grpc")
            srcDirs("build/generated/source/proto/main/grpckt")
            // This contains the Java classes like the request and response messages.
            srcDirs("build/generated/source/proto/main/java")
            // This contains the Kotlin extensions to those java classes, like DSLs.
            srcDirs("build/generated/source/proto/main/kotlin")
        }
    }
}

tasks.jar { manifest.attributes["Main-Class"] = "com.test.MainKt" }
project.setProperty("mainClassName", "com.test.MainKt") // for shadowJar

Content of Shadow JAR (jar tf <jar file> - post link to GIST if too long)

https://gist.github.com/curtisullerich/ddc07cf6869135c50b5fc20ef1f272d4

I uploaded a sample project here that exhibits the behavior: https://github.com/curtisullerich/shadow-grpc

[curtisullerich]

A friend found the answer. This ended up being all it took to make it work:

tasks.named("shadowJar", ShadowJar::class) {
    mergeServiceFiles()
}

I wasn't familiar with this aspect of jar files before. I wonder if shadowJar could show a warning when service files are getting omitted/clobbered during the jar building process?