search

grpc-ecosystem / grpc-spring

Spring Boot starter module for gRPC framework.
https://grpc-ecosystem.github.io/grpc-spring/
Apache License 2.0
3.43k stars 809 forks source link

GRPC client fails to start in Spring Cloud environment #1025

Closed yuluo-yx closed 6 months ago

yuluo-yx commented 6 months ago

The context

The bug

Caused by: java.lang.IllegalStateException: Failed to start the grpc server
    at net.devh.boot.grpc.server.serverfactory.GrpcServerLifecycle.start(GrpcServerLifecycle.java:74) ~[grpc-server-spring-boot-autoconfigure-2.15.0.RELEASE.jar:2.15.0.RELEASE]
    at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:179) ~[spring-context-6.0.11.jar:6.0.11]
    ... 13 common frames omitted
Caused by: java.io.IOException: Failed to bind to address 0.0.0.0/0.0.0.0:9090
    at io.grpc.netty.shaded.io.grpc.netty.NettyServer.start(NettyServer.java:326) ~[grpc-netty-shaded-1.58.0.jar:1.58.0]
    at io.grpc.internal.ServerImpl.start(ServerImpl.java:185) ~[grpc-core-1.58.0.jar:1.58.0]
    at io.grpc.internal.ServerImpl.start(ServerImpl.java:94) ~[grpc-core-1.58.0.jar:1.58.0]
    at net.devh.boot.grpc.server.serverfactory.GrpcServerLifecycle.createAndStartGrpcServer(GrpcServerLifecycle.java:113) ~[grpc-server-spring-boot-autoconfigure-2.15.0.RELEASE.jar:2.15.0.RELEASE]
    at net.devh.boot.grpc.server.serverfactory.GrpcServerLifecycle.start(GrpcServerLifecycle.java:72) ~[grpc-server-spring-boot-autoconfigure-2.15.0.RELEASE.jar:2.15.0.RELEASE]
    ... 14 common frames omitted
Caused by: java.net.BindException: Address already in use: bind
    at java.base/sun.nio.ch.Net.bind0(Native Method) ~[na:na]
    at java.base/sun.nio.ch.Net.bind(Net.java:555) ~[na:na]

Steps to Reproduce

Steps to reproduce the behavior:

  1. git clone https://github.com/yuluo-yx/sca-k8s-demo.git
  2. start nacos server 2.3.0
  3. run provider and client
yuluo-yx commented 6 months ago

Any idea? 🥲🥲🥲

ST-DDT commented 6 months ago

According to thw config the server should start on port 9000 but in the error message it uses 9090.

Have you checked locally whether the issue can be reproduced there and whether it is caused by the server alone or only in combination with the client? Also a common cause for these kind of issues are kubernetes autogenerated env variables where a service called grpc-server automatically creates an env property grpc_server_port which is then used by spring to set the property accordingly.

yuluo-yx commented 6 months ago

According to thw config the server should start on port 9000 but in the error message it uses 9090.

Have you checked locally whether the issue can be reproduced there and whether it is caused by the server alone or only in combination with the client? Also a common cause for these kind of issues are kubernetes autogenerated env variables where a service called grpc-server automatically creates an env property grpc_server_port which is then used by spring to set the property accordingly.

I am sure this problem exists locally. This problem only exists on the client side and the server side is normal.

And, I did not start it in the k8s environment, k8s is just the name of my project, please exclude it. You can run the project demo I provided and comment out the code related to nacos, and you can still reproduce this problem.

ST-DDT commented 6 months ago

The problem is that common contains both the server and the client, thus when you start provider and consumer on the same machine, they both try to be server on port 9090 AFACIT.

yuluo-yx commented 6 months ago

The problem is that common contains both the server and the client, thus when you start provider and consumer on the same machine, they both try to be server on port 9090 AFACIT.

Tks, when I removed the server dependency introduced in common from the client, it started successfully.

    <dependencies>
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-k8s-common</artifactId>
            <version>2024.01.08</version>
            <exclusions>
                <exclusion>
                    <groupId>net.devh</groupId>
                    <artifactId>grpc-server-spring-boot-starter</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>
yuluo-yx commented 6 months ago

I have a question, how should I handle the following exception when calling the service? @ST-DDT 

java.lang.IllegalStateException: call already closed
    at com.google.common.base.Preconditions.checkState(Preconditions.java:512) ~[guava-32.0.1-android.jar:na]
    at io.grpc.internal.ServerCallImpl.closeInternal(ServerCallImpl.java:213) ~[grpc-core-1.58.0.jar:1.58.0]
    at io.grpc.internal.ServerCallImpl.close(ServerCallImpl.java:208) ~[grpc-core-1.58.0.jar:1.58.0]
    at io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onError(ServerCalls.java:389) ~[grpc-stub-1.58.0.jar:1.58.0]
    at io.grpc.stub.ServerCalls.asyncUnimplementedUnaryCall(ServerCalls.java:460) ~[grpc-stub-1.58.0.jar:1.58.0]
    at com.alibaba.cloud.grpc.lib.SCAk8sProviderGrpc$SCAk8sProviderImplBase.providerA(SCAk8sProviderGrpc.java:144) ~[classes/:na]
    at com.alibaba.cloud.k8s.service.ProviderServiceImpl.providerA(ProviderServiceImpl.java:32) ~[classes/:na]
    at com.alibaba.cloud.grpc.lib.SCAk8sProviderGrpc$MethodHandlers.invoke(SCAk8sProviderGrpc.java:286) ~[classes/:na]
ST-DDT commented 6 months ago

Dont call super.providerA(request, responseObserver);

yuluo-yx commented 6 months ago

Dont call super.providerA(request, responseObserver);

Tks. I solved it. 🫡

yuluo-yx commented 6 months ago

When I request the client interface, the following exception occurs. Do you have any solution?

java.lang.illegalstateexception: call already closed

ST-DDT commented 6 months ago

When I request the client interface, the following exception occurs. Do you have any solution?

java.lang.illegalstateexception: call already closed

Can you please give me some more code or a reference that I can work with? The error is likely caused by some code that either calls onCompleted() or onError() before calling onNext. Please also include the full stacktrace.

yuluo-yx commented 6 months ago

When I request the client interface, the following exception occurs. Do you have any solution? java.lang.illegalstateexception: call already closed

Can you please give me some more code or a reference that I can work with? The error is likely caused by some code that either calls onCompleted() or onError() before calling onNext. Please also include the full stacktrace.

java.lang.IllegalStateException: call already closed
    at com.google.common.base.Preconditions.checkState(Preconditions.java:512) ~[guava-32.0.1-android.jar:na]
    at io.grpc.internal.ServerCallImpl.closeInternal(ServerCallImpl.java:213) ~[grpc-core-1.58.0.jar:1.58.0]
    at io.grpc.internal.ServerCallImpl.close(ServerCallImpl.java:208) ~[grpc-core-1.58.0.jar:1.58.0]
    at io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onError(ServerCalls.java:389) ~[grpc-stub-1.58.0.jar:1.58.0]
    at io.grpc.stub.ServerCalls.asyncUnimplementedUnaryCall(ServerCalls.java:460) ~[grpc-stub-1.58.0.jar:1.58.0]
    at com.alibaba.cloud.grpc.lib.SCAk8sProviderGrpc$SCAk8sProviderImplBase.providerA(SCAk8sProviderGrpc.java:144) ~[classes/:na]
    at com.alibaba.cloud.k8s.service.ProviderServiceImpl.providerA(ProviderServiceImpl.java:28) ~[classes/:na]
    at com.alibaba.cloud.grpc.lib.SCAk8sProviderGrpc$MethodHandlers.invoke(SCAk8sProviderGrpc.java:286) ~[classes/:na]
    at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182) ~[grpc-stub-1.58.0.jar:1.58.0]
    at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35) ~[grpc-api-1.58.0.jar:1.58.0]
    at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23) ~[grpc-api-1.58.0.jar:1.58.0]
    at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40) ~[grpc-api-1.58.0.jar:1.58.0]
    at io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86) ~[grpc-api-1.58.0.jar:1.58.0]
    at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:351) ~[grpc-core-1.58.0.jar:1.58.0]
    at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:860) ~[grpc-core-1.58.0.jar:1.58.0]
    at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[grpc-core-1.58.0.jar:1.58.0]
    at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) ~[grpc-core-1.58.0.jar:1.58.0]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]
    at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na]

Consumer Method A

@Override
public String consumerB() {

System.out.println(blockingStub);

try {
    ResponseBody responseBody = blockingStub.providerB(
            RequestBody
                    .newBuilder()
                    .setConsumer(CONSUMER_NAME + "B")
                    .build());
    return responseBody.getMessage();
}
catch (Exception e) {
    throw new RuntimeException(e);
}
}

Provider Method A

@Override
public void providerA(RequestBody request, StreamObserver<ResponseBody> responseObserver) {

responseObserver.onNext(
        ResponseBody
                .newBuilder()
                .setMessage("This response from provider A!")
                .build()
);

responseObserver.onCompleted();
super.providerA(request, responseObserver);
}

Tks! 🫡

https://github.com/yuluo-yx/sca-k8s-demo.git My latest code is automatically sent to the github repository in a timely manner

ST-DDT commented 6 months ago

super.providerA(request, responseObserver);

Do not call super.providerA as mentioned before.

at io.grpc.stub.ServerCalls.asyncUnimplementedUnaryCall(ServerCalls.java:460) ~[grpc-stub-1.58.0.jar:1.58.0] at com.alibaba.cloud.grpc.lib.SCAk8sProviderGrpc$SCAk8sProviderImplBase.providerA(SCAk8sProviderGrpc.java:144) ~[classes/:na]

yuluo-yx commented 6 months ago

super.providerA(request, responseObserver);

Do not call super.providerA as mentioned before.

at io.grpc.stub.ServerCalls.asyncUnimplementedUnaryCall(ServerCalls.java:460) ~[grpc-stub-1.58.0.jar:1.58.0] at com.alibaba.cloud.grpc.lib.SCAk8sProviderGrpc$SCAk8sProviderImplBase.providerA(SCAk8sProviderGrpc.java:144) ~[classes/:na]

Thank you so much! 🫡🫡