search

spring-cloud / spring-cloud-gateway

An API Gateway built on Spring Framework and Spring Boot providing routing and more.
http://cloud.spring.io
Apache License 2.0
4.5k stars 3.3k forks source link

Add support for multiple proto files in one descriptor for JsonToGrpcGatewayFilterFactory #3341

Open nihadguluzade opened 5 months ago

nihadguluzade commented 5 months ago

Boot version: 3.2.3 Cloud version: 2023.0.0 Gateway version: 4.1.0

I have 2 proto files. I generate the proto descriptor file using the protobuf-maven-plugin. Then, I set the name of the descriptor file in the application properties as described in documentation. When I send the JSON request to the gateway it throws the java.util.NoSuchElementException: No Service found exception. However, if I generate the descriptor file that describes only 1 proto file, no exception occurs.

A workaround is to generate separate descriptor files for each proto file. Is there any way to make it work for a descriptor file that descripes multiple proto files?

Stack trace:

java.util.NoSuchElementException: No Service found
    at org.springframework.cloud.gateway.filter.factory.JsonToGrpcGatewayFilterFactory$GRPCResponseDecorator.getMethodDescriptor(JsonToGrpcGatewayFilterFactory.java:248)
    Suppressed: The stacktrace has been enhanced by Reactor, refer to additional information below: 
Error has been observed at the following site(s):
    *__checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ HTTP POST "/grpc" [ExceptionHandlingWebHandler]
Original Stack Trace:
        at org.springframework.cloud.gateway.filter.factory.JsonToGrpcGatewayFilterFactory$GRPCResponseDecorator.getMethodDescriptor(JsonToGrpcGatewayFilterFactory.java:248)
        at org.springframework.cloud.gateway.filter.factory.JsonToGrpcGatewayFilterFactory$GRPCResponseDecorator.<init>(JsonToGrpcGatewayFilterFactory.java:195)
        at org.springframework.cloud.gateway.filter.factory.JsonToGrpcGatewayFilterFactory$1.filter(JsonToGrpcGatewayFilterFactory.java:106)
        at org.springframework.cloud.gateway.filter.OrderedGatewayFilter.filter(OrderedGatewayFilter.java:44)
        at org.springframework.cloud.gateway.handler.FilteringWebHandler$DefaultGatewayFilterChain.lambda$filter$0(FilteringWebHandler.java:117)
        at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:45)
        at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
        at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
        at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
        at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
        at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
        at reactor.core.publisher.Mono.subscribe(Mono.java:4563)
        at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:265)
        at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
        at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
        at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:165)
        at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)
        at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)
        at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:82)
        at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.innerNext(FluxConcatMapNoPrefetch.java:259)
        at reactor.core.publisher.FluxConcatMap$ConcatMapInner.onNext(FluxConcatMap.java:865)
        at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
        at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)
        at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:158)
        at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
        at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:82)
        at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.innerNext(FluxConcatMapNoPrefetch.java:259)
        at reactor.core.publisher.FluxConcatMap$ConcatMapInner.onNext(FluxConcatMap.java:865)
        at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)
        at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onNext(MonoPeekTerminal.java:180)
        at reactor.core.publisher.MonoFilterWhen$MonoFilterWhenMain.onNext(MonoFilterWhen.java:136)
        at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2571)
        at reactor.core.publisher.MonoFilterWhen$MonoFilterWhenMain.request(MonoFilterWhen.java:183)
        at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.request(MonoPeekTerminal.java:139)
        at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2367)
        at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onSubscribe(FluxOnErrorResume.java:74)
        at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onSubscribe(MonoPeekTerminal.java:152)
        at reactor.core.publisher.MonoFilterWhen$MonoFilterWhenMain.onSubscribe(MonoFilterWhen.java:100)
        at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)
        at reactor.core.publisher.Mono.subscribe(Mono.java:4563)
        at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.onNext(FluxConcatMapNoPrefetch.java:207)
        at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.onNext(FluxDematerialize.java:98)
        at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.onNext(FluxDematerialize.java:44)
        at reactor.core.publisher.FluxIterable$IterableSubscription.slowPath(FluxIterable.java:335)
        at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:294)
        at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.request(FluxDematerialize.java:127)
        at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.innerComplete(FluxConcatMapNoPrefetch.java:275)
        at reactor.core.publisher.FluxConcatMap$ConcatMapInner.onComplete(FluxConcatMap.java:889)
        at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onComplete(Operators.java:2231)
        at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onComplete(MonoPeekTerminal.java:299)
        at reactor.core.publisher.MonoFilterWhen$MonoFilterWhenMain.onNext(MonoFilterWhen.java:141)
        at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2571)
        at reactor.core.publisher.MonoFilterWhen$MonoFilterWhenMain.request(MonoFilterWhen.java:183)
        at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.request(MonoPeekTerminal.java:139)
        at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.request(Operators.java:2331)
        at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.request(Operators.java:2331)
        at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.request(FluxConcatMapNoPrefetch.java:339)
        at reactor.core.publisher.MonoNext$NextSubscriber.request(MonoNext.java:108)
        at reactor.core.publisher.FluxMap$MapSubscriber.request(FluxMap.java:164)
        at reactor.core.publisher.MonoFlatMap$FlatMapMain.request(MonoFlatMap.java:194)
        at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.request(Operators.java:2331)
        at reactor.core.publisher.FluxMap$MapSubscriber.request(FluxMap.java:164)
        at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.request(Operators.java:2331)
        at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.request(FluxConcatMapNoPrefetch.java:339)
        at reactor.core.publisher.MonoNext$NextSubscriber.request(MonoNext.java:108)
        at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2367)
        at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2241)
        at reactor.core.publisher.MonoNext$NextSubscriber.onSubscribe(MonoNext.java:70)
        at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.onSubscribe(FluxConcatMapNoPrefetch.java:164)
        at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:201)
        at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:83)
        at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
        at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
        at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
        at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
        at reactor.core.publisher.Mono.subscribe(Mono.java:4563)
        at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:265)
        at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
        at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)
        at reactor.core.publisher.MonoDeferContextual.subscribe(MonoDeferContextual.java:55)
        at reactor.netty.http.server.HttpServer$HttpServerHandle.onStateChange(HttpServer.java:1169)
        at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:710)
        at reactor.netty.transport.ServerTransport$ChildObserver.onStateChange(ServerTransport.java:481)
        at reactor.netty.http.server.HttpServerOperations.onInboundNext(HttpServerOperations.java:652)
        at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:114)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at reactor.netty.http.server.HttpTrafficHandler.channelRead(HttpTrafficHandler.java:238)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:333)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:454)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
        at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:833)

Digging into the source code of JsonToGrpcGatewayFilterFactory class I see that it takes the first descriptor from the file:

DescriptorProtos.FileDescriptorSet fileDescriptorSet = DescriptorProtos.FileDescriptorSet.parseFrom(descriptorFile);
DescriptorProtos.FileDescriptorProto fileProto = fileDescriptorSet.getFile(0);
+ Descriptors.FileDescriptor fileDescriptor = Descriptors.FileDescriptor.buildFrom(fileProto, new Descriptors.FileDescriptor[0]);
Descriptors.ServiceDescriptor serviceDescriptor = fileDescriptor.findServiceByName(config.getService());
if (serviceDescriptor == null) {
    throw new NoSuchElementException("No Service found");
}

This is the plugin I use to create the descriptor file:

            <plugin>
                <groupId>org.xolstice.maven.plugins</groupId>
                <artifactId>protobuf-maven-plugin</artifactId>
                <version>${protobuf-plugin.version}</version>
                <configuration>
                    <protocArtifact>com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}</protocArtifact>
                    <pluginId>grpc-java</pluginId>
                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}</pluginArtifact>
                    <writeDescriptorSet>true</writeDescriptorSet>
                    <descriptorSetOutputDirectory>${project.build.directory}/classes/</descriptorSetOutputDirectory>
                    <descriptorSetFileName>descriptor.pb</descriptorSetFileName>
                    <includeDependenciesInDescriptorSet>true</includeDependenciesInDescriptorSet>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>compile-custom</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

My proto files if it matters:

syntax = "proto2";

package com.test.io.greet.dto;

option java_multiple_files = true;

message HiRequest {
  required string name = 1;
  optional string surname = 2;
}

message HiReply {
  optional string msg = 1;
}

service GreetService {
  rpc SayHi(HiRequest) returns (HiReply);
}
syntax = "proto2";

package com.test.io.admin.dto;

option java_multiple_files = true;

message HelloRequest {
  required string name = 1;
  required string surname = 2;
}

message HelloReply {
  optional string msg = 1;
}

service AdminService {
  rpc SayHello(HelloRequest) returns (HelloReply);
}

My application properties:

spring:
  cloud:
    gateway:
      routes:
        - id: json-grpc
          uri: http://localhost:9090
          predicates:
            - Path=/grpc/**
          filters:
            - name: JsonToGrpc
              args:
                protoDescriptor: classpath:descriptor.pb
                protoFile: classpath:admin.proto
                service: AdminService
                method: SayHello

Any help is appreciated.

spencergibb commented 5 months ago

Calling @abelsromero or @Albertoimpl

spring-cloud-issues commented 5 months ago

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Albertoimpl commented 4 months ago

Thanks a lot for reporting @nihadguluzade, you are right, we just take the one descriptor. We should add support for more than one. I can't think of a workaround with the current implementation aside from having one descriptor file.