search

grpc / grpc-java

The Java gRPC implementation. HTTP/2 based RPC
https://grpc.io/docs/languages/java/
Apache License 2.0
11.42k stars 3.84k forks source link

Address types of NameResolver 'unix' for '...' not supported by transport exception in Fat Jar #10853

Closed ModAlpha closed 7 months ago

ModAlpha commented 9 months ago

I have ktor server app and I'm using gradle as build tool and using this dependency for gRPC implementation("io.grpc:grpc-netty-shaded:1.61.0") on my mac M1 and jdk 17 and gradle 8.5.0 and I'm connecting to another service via gRPC

 private fun connect() {
        val host = config.property("mainService.host").getString()
        val port = config.property("mainService.port").getString()

        val channel = ManagedChannelBuilder.forAddress(host, port.toInt()).usePlaintext().build()
        stub = InvoiceInternalServiceGrpcKt.InvoiceInternalServiceCoroutineStub(channel)
    }

when running the app via IntelliJ IDEA all fine but when running the app via fat jar this exception occur

java.lang.IllegalArgumentException: Address types of NameResolver 'unix' for '...' not supported by transport
        at io.grpc.internal.ManagedChannelImpl.getNameResolver(ManagedChannelImpl.java:750)
        at io.grpc.internal.ManagedChannelImpl.getNameResolver(ManagedChannelImpl.java:771)
        at io.grpc.internal.ManagedChannelImpl.<init>(ManagedChannelImpl.java:635)
        at io.grpc.internal.ManagedChannelImplBuilder.build(ManagedChannelImplBuilder.java:662)
        at io.grpc.ForwardingChannelBuilder2.build(ForwardingChannelBuilder2.java:254)
        at ai.pedant.infrastructure.services.DocumentService.connect(DocumentService.kt:83)
        at ai.pedant.infrastructure.services.DocumentService.<init>(DocumentService.kt:24)
        at ai.pedant.di.ServiceModuleKt$serviceModule$1$3.invoke(ServiceModule.kt:14)
        at ai.pedant.di.ServiceModuleKt$serviceModule$1$3.invoke(ServiceModule.kt:14)
        at org.koin.core.instance.InstanceFactory.create(InstanceFactory.kt:51)
        at org.koin.core.instance.SingleInstanceFactory.create(SingleInstanceFactory.kt:46)
        at org.koin.core.instance.SingleInstanceFactory$get$1.invoke(SingleInstanceFactory.kt:53)
        at org.koin.core.instance.SingleInstanceFactory$get$1.invoke(SingleInstanceFactory.kt:51)
        at org.koin.mp.KoinPlatformTools.synchronized(KoinPlatformTools.kt:20)
        at org.koin.core.instance.SingleInstanceFactory.get(SingleInstanceFactory.kt:51)
        at org.koin.core.registry.InstanceRegistry.resolveInstance$koin_core(InstanceRegistry.kt:116)
        at org.koin.core.scope.Scope.resolveValue(Scope.kt:246)
        at org.koin.core.scope.Scope.resolveInstance(Scope.kt:231)
        at org.koin.core.scope.Scope.get(Scope.kt:210)
        at ai.pedant.di.ServiceModuleKt$serviceModule$1$1.invoke(ServiceModule.kt:21)
        at ai.pedant.di.ServiceModuleKt$serviceModule$1$1.invoke(ServiceModule.kt:12)
        at org.koin.core.instance.InstanceFactory.create(InstanceFactory.kt:51)
        at org.koin.core.instance.SingleInstanceFactory.create(SingleInstanceFactory.kt:46)
        at org.koin.core.instance.SingleInstanceFactory$get$1.invoke(SingleInstanceFactory.kt:53)
        at org.koin.core.instance.SingleInstanceFactory$get$1.invoke(SingleInstanceFactory.kt:51)
        at org.koin.mp.KoinPlatformTools.synchronized(KoinPlatformTools.kt:20)
        at org.koin.core.instance.SingleInstanceFactory.get(SingleInstanceFactory.kt:51)
        at org.koin.core.registry.InstanceRegistry.resolveInstance$koin_core(InstanceRegistry.kt:116)
        at org.koin.core.scope.Scope.resolveValue(Scope.kt:246)
        at org.koin.core.scope.Scope.resolveInstance(Scope.kt:231)
        at org.koin.core.scope.Scope.get(Scope.kt:210)
        at ai.pedant.di.GrpcContollerModuleKt$grpcControllerModule$1$1.invoke(GrpcContollerModule.kt:19)
        at ai.pedant.di.GrpcContollerModuleKt$grpcControllerModule$1$1.invoke(GrpcContollerModule.kt:10)
        at org.koin.core.instance.InstanceFactory.create(InstanceFactory.kt:51)
        at org.koin.core.instance.SingleInstanceFactory.create(SingleInstanceFactory.kt:46)
        at org.koin.core.instance.SingleInstanceFactory$get$1.invoke(SingleInstanceFactory.kt:53)
        at org.koin.core.instance.SingleInstanceFactory$get$1.invoke(SingleInstanceFactory.kt:51)
        at org.koin.mp.KoinPlatformTools.synchronized(KoinPlatformTools.kt:20)
        at org.koin.core.instance.SingleInstanceFactory.get(SingleInstanceFactory.kt:51)
        at org.koin.core.registry.InstanceRegistry.resolveInstance$koin_core(InstanceRegistry.kt:116)
        at org.koin.core.scope.Scope.resolveValue(Scope.kt:246)
        at org.koin.core.scope.Scope.resolveInstance(Scope.kt:231)
        at org.koin.core.scope.Scope.get(Scope.kt:210)
        at ai.pedant.GrpcServer$special$$inlined$inject$default$2.invoke(KoinComponent.kt:74)
        at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
        at ai.pedant.GrpcServer.get_vendorWebController(GrpcServer.kt:19)
        at ai.pedant.GrpcServer.<init>(GrpcServer.kt:31)
        at ai.pedant.ApplicationKt.startGrpcServer(Application.kt:48)
        at ai.pedant.ApplicationKt.access$startGrpcServer(Application.kt:1)
        at ai.pedant.ApplicationKt$module$2.invokeSuspend(Application.kt:42)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
lc525 commented 8 months ago

I'm seeing this on 1.61.0 as well (also on grpc-netty-shaded and jvm 17), looks like a regression due to changes in .getNameResolver. This worked fine in previous versions (I'm upgrading from 1.57.2).

Briefly looking at the code, .forAddress(..) ends up building a target for the channel in GrpcUtil.class#L542 using a null URI scheme.

Then, the string result is parsed again as a URI and the default scheme of the NameResolverRegistry is used: ManagedChannelImpl.getNameResolver#L732

For both of us, that default scheme ends up being unix (i.e. unix domain sockets), which is not was asked for when initially doing .forAddress(...).

lc525 commented 8 months ago

Some progress on this & solution: the NameResolverProvider being used is picked by the JDK from META-INF/services/io.grpc.NameResolverProvider.

The default in that file for the grpc-core.jar is io.grpc.internal.DnsNameResolverProvider.

However, the grpc-netty-shaded.jar also has the same file in META-INF, but this time containing io.grpc.internal.UdsNameResolverProvider (the Unix Domain Socket resolver provider).

When building the fat jar, this latter file is what ends up for me in META-INF, leading to unix being the default scheme being considered, and the dns scheme not being present. We need the file from grpc-core in the fat jar.

The solution would be to use the shadow gradle plugin and have something like

shadowJar {
    mergeServiceFiles() 
}

in your build file in order to merge the settings across dependencies.

In other words, I believe the original report is not a bug but intended behaviour.

D12EA177E12 commented 8 months ago

Is there a (clean) solution for folks using Ant? (Yes, I am *required to use Ant for my project)

Currently, I need to manually concatenate (i.e. merge) the contents of all instances of the NameResolverProvider service descriptor file into a single file, delete all instances from the jar, and then finally copy the merged service descriptor back into the jar.

ejona86 commented 8 months ago

@ModAlpha, you are very likely making a "fat" jar without configuring the combining tool to merge service files. The fix depends on what tool you are using to make the fat jar. Looks like I've provided a breadcrumb at https://github.com/grpc/grpc-java/issues/10826#issuecomment-1896254509 before.

sergiitk commented 7 months ago

Seems like this is resolved. If not, comment, and it can be reopened.

jaredrummler commented 5 months ago

I encountered this today with a KTor project deployed to App Engine. I use shadowJar to generate my fatJar and the fix was to add mergeServiceFiles() to the shadowJar task in Gradle. Here's the relevant part of my build.gradle.kts:

tasks.register<ShadowJar>("shadowJar") {
    mergeServiceFiles()
    // other configurations
}

I am using the latest version of grpc.