search

grpc-ecosystem / grpc-spring

Spring Boot starter module for gRPC framework.
https://grpc-ecosystem.github.io/grpc-spring/
Apache License 2.0
3.48k stars 812 forks source link

如何解决服务发现时grpc服务端端口不一致的关联 #939

Open Layfolk-zcy opened 1 year ago

Layfolk-zcy commented 1 year ago

1、服务发现时通过DiscoveryClientNameResolver获取服务列表 2、服务端服务启动grpc服务端但是ServerBuilder指定端口时不能和springBoot服务端口相同 所以想知道如何进行关联的,目前进行代码阅读后尝试自己建立通信启动时报

java.io.IOException: Failed to bind
    at io.grpc.netty.shaded.io.grpc.netty.NettyServer.start(NettyServer.java:264) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.internal.ServerImpl.start(ServerImpl.java:183) ~[grpc-core-1.33.1.jar:1.33.1]
    at io.grpc.internal.ServerImpl.start(ServerImpl.java:90) ~[grpc-core-1.33.1.jar:1.33.1]
    at com.sunyard.server.GrpcServerRunner.startGrpcServer(GrpcServerRunner.java:64) [classes/:na]
    at com.sunyard.server.GrpcServerRunner.onApplicationEvent(GrpcServerRunner.java:50) [classes/:na]
    at com.sunyard.server.GrpcServerRunner.onApplicationEvent(GrpcServerRunner.java:34) [classes/:na]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176) [spring-context-5.3.27.jar:5.3.27]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169) [spring-context-5.3.27.jar:5.3.27]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143) [spring-context-5.3.27.jar:5.3.27]
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:421) [spring-context-5.3.27.jar:5.3.27]
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:378) [spring-context-5.3.27.jar:5.3.27]
    at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:940) [spring-context-5.3.27.jar:5.3.27]
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:586) [spring-context-5.3.27.jar:5.3.27]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:147) [spring-boot-2.7.11.jar:2.7.11]
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:731) [spring-boot-2.7.11.jar:2.7.11]
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408) [spring-boot-2.7.11.jar:2.7.11]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:307) [spring-boot-2.7.11.jar:2.7.11]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) [spring-boot-2.7.11.jar:2.7.11]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) [spring-boot-2.7.11.jar:2.7.11]
    at com.sunyard.GrpcServerDemoTwo.main(GrpcServerDemoTwo.java:37) [classes/:na]
Caused by: java.net.BindException: Address already in use: bind
    at sun.nio.ch.Net.bind0(Native Method) ~[na:1.8.0_201]
    at sun.nio.ch.Net.bind(Net.java:433) ~[na:1.8.0_201]
    at sun.nio.ch.Net.bind(Net.java:425) ~[na:1.8.0_201]
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) ~[na:1.8.0_201]
    at io.grpc.netty.shaded.io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:134) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:550) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1334) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:973) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannel.bind(AbstractChannel.java:248) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:356) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[grpc-netty-shaded-1.33.1.jar:1.33.1]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_201]
ST-DDT commented 1 year ago

English

Sorry, I don't understand your request. Also I don't see any stacktrace lines associated to this library that I would expect.

Layfolk-zcy commented 1 year ago

English

Sorry, I don't understand your request. Also I don't see any stacktrace lines associated to this library that I would expect.

The DiscoveryClientNameResolver class resolves the Ip and port in the service list of the registry to establish a connection to the grpc server. However, my server's ImplBase implementation is registered through the ServerBuilder method and sets the grpc server listening port. The listening port of this ServerBuilder is different from the port resolved by DiscoveryClientNameResolver. Therefore, a connection cannot be established between the client and the server, So I will use the same port as the SpringBoot service port that ServerBulider listens to. Therefore, when the SpringBoot project on the grpc server starts, the above exception message will be generated.

I would like to know how the spring boot grpc project handles this situation

ST-DDT commented 1 year ago

I have separate ports for rest and grpc.

The discovery name resolver cheats a bit by checking the metadata for a port overwrite.

https://github.com/yidongnan/grpc-spring-boot-starter/blob/9495d5e1f825ad384e96917bdb3a30b09373bd57/grpc-client-spring-boot-autoconfigure/src/main/java/net/devh/boot/grpc/client/nameresolver/DiscoveryClientNameResolver.java#L173

Alternatively you can register the service twice. Once for rest, once for grpc.

Layfolk-zcy commented 1 year ago

I have separate ports for rest and grpc.

The discovery name resolver cheats a bit by checking the metadata for a port overwrite.

https://github.com/yidongnan/grpc-spring-boot-starter/blob/9495d5e1f825ad384e96917bdb3a30b09373bd57/grpc-client-spring-boot-autoconfigure/src/main/java/net/devh/boot/grpc/client/nameresolver/DiscoveryClientNameResolver.java#L173

或者，您可以注册该服务两次。一次休息，一次用于grpc。

Now it is necessary to distinguish the GRPC service port launched by ServerBuilder from the spring boot service port by obtaining it from the Matedata in the registry

Layfolk-zcy commented 1 year ago

I would like to inquire about how to implement load policy changes on the client side through ManagedChannel. During the demo test, I closed ManagedChannel through the interface and recreated a new ManagedChannel, modifying the policy value of the defaultLoadBalancingPolicy() method, and encountered an exception when recreating it

2023-07-23 22:08:02.397  WARN 6452 --- [ault-executor-1] io.grpc.internal.ManagedChannelImpl      : Subchannel.requestConnection() should be called from SynchronizationContext. This warning will become an exception in a future release. See https://github.com/grpc/grpc-java/issues/5015 for more details

java.lang.IllegalStateException: Not called from the SynchronizationContext
    at com.google.common.base.Preconditions.checkState(Preconditions.java:511) ~[guava-29.0-android.jar:na]
    at io.grpc.SynchronizationContext.throwIfNotInThisSynchronizationContext(SynchronizationContext.java:135) [grpc-api-1.33.1.jar:1.33.1]
    at io.grpc.internal.ManagedChannelImpl.logWarningIfNotInSyncContext(ManagedChannelImpl.java:2190) [grpc-core-1.33.1.jar:1.33.1]
    at io.grpc.internal.ManagedChannelImpl.access$4900(ManagedChannelImpl.java:111) [grpc-core-1.33.1.jar:1.33.1]
    at io.grpc.internal.ManagedChannelImpl$SubchannelImpl.requestConnection(ManagedChannelImpl.java:1907) [grpc-core-1.33.1.jar:1.33.1]
    at com.sunyard.loadbalance.AbstractLoadBalancer.handleResolvedAddresses(AbstractLoadBalancer.java:105) [classes/:na]
    at io.grpc.internal.AutoConfiguredLoadBalancerFactory$AutoConfiguredLoadBalancer.tryHandleResolvedAddresses(AutoConfiguredLoadBalancerFactory.java:154) [grpc-core-1.33.1.jar:1.33.1]
    at io.grpc.internal.ManagedChannelImpl$NameResolverListener$1NamesResolved.run(ManagedChannelImpl.java:1663) [grpc-core-1.33.1.jar:1.33.1]
    at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:95) [grpc-api-1.33.1.jar:1.33.1]
    at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:127) [grpc-api-1.33.1.jar:1.33.1]
    at io.grpc.internal.ManagedChannelImpl$NameResolverListener.onResult(ManagedChannelImpl.java:1677) [grpc-core-1.33.1.jar:1.33.1]
    at com.sunyard.loadbalance.resolver.DiscoveryClientNameResolver$Resolve.resolveInternal(DiscoveryClientNameResolver.java:330) [classes/:na]
    at com.sunyard.loadbalance.resolver.DiscoveryClientNameResolver$Resolve.run(DiscoveryClientNameResolver.java:287) [classes/:na]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_201]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_201]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_201]

Thank you very much for your advice

ST-DDT commented 1 year ago

Please include the relevant source code. Currently all I can tell is that you have to run the resolution/final setting of the addresses in the synchronisation context. Our implementation should already do that.

https://github.com/yidongnan/grpc-spring-boot-starter/blob/9495d5e1f825ad384e96917bdb3a30b09373bd57/grpc-client-spring-boot-autoconfigure/src/main/java/net/devh/boot/grpc/client/nameresolver/DiscoveryClientNameResolver.java#L238

Layfolk-zcy commented 1 year ago

Please include the relevant source code. Currently all I can tell is that you have to run the resolution/final setting of the addresses in the synchronisation context. Our implementation should already do that.

https://github.com/yidongnan/grpc-spring-boot-starter/blob/9495d5e1f825ad384e96917bdb3a30b09373bd57/grpc-client-spring-boot-autoconfigure/src/main/java/net/devh/boot/grpc/client/nameresolver/DiscoveryClientNameResolver.java#L238

In my test demo project, the implementation of DiscoveryClientNameResolver.java is directly implemented using the code in the current project(grpc-spring-boot-starter) without any modifications. In my demo, I implemented Picker,LoadBalancer and ServerBuilder.

https://github.com/Layfolk-zcy/grpc-parent/blob/master/grpc-base-demo/src/main/java/com/sunyard/loadbalance/resolver/DiscoveryClientNameResolver.java

This interface is exposed for modification of ManagedChannel https://github.com/Layfolk-zcy/grpc-parent/blob/master/grpc-base-demo/src/main/java/com/sunyard/controller/GrpcClientLoadBalanceChange.java#L41

this is my rep https://github.com/Layfolk-zcy/grpc-parent/

ST-DDT commented 1 year ago

Thanks for the information. Looks like I have to investigate this in more detail. My time is currently very limited so this might take a while.

Layfolk-zcy commented 1 year ago

Thanks for the information. Looks like I have to investigate this in more detail. My time is currently very limited so this might take a while. Thank you for your help. I can wait to hear from you