Yelp / nrtsearch

A high performance gRPC server on top of Apache Lucene
Apache License 2.0
265 stars 40 forks source link

Integrating nrtSearch gRPC server with .Net7 gRPC client with JsonTranscoding #552

Open Jeevananthan-23 opened 1 year ago

Jeevananthan-23 commented 1 year ago

I could like to understand how should communicate with nrtSearch (Java) gRPC server with .Net7 gRPC client found the connectivity error.

To have knowledge of what gRPC Json Transcoding here is similar to grpc-gateway but it deserializes JSON into Protobuf messages, then invokes the gRPC service directly without creating a reverse-proxy server.

io.grpc.netty.shaded.io.grpc.netty.NettyServerTransport notifyTerminated
INFO: Transport failed
io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception: HTTP/2 client preface string missing or corrupt. Hex dump for received bytes: 16030100ff010000fb0303cd22c1d466ccc4e30f1d5b0306
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception.connectionError(Http2Exception.java:108)
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.readClientPrefaceString(Http2ConnectionHandler.java:306)
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.decode(Http2ConnectionHandler.java:239)
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:438)
    at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507)
    at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446)
    at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
    at io.grpc.netty.shaded.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
    at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
    at io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:833)

error while connecting java grpc server to .net client

sarthakn7 commented 1 year ago

From your link:

Register transcoding in server startup code by adding AddJsonTranscoding. For example, services.AddGrpc().AddJsonTranscoding().

It appears that this transcoding needs to be supported in the server, which we don't do in nrtsearch. Also I think that transcoding package is native to .NET so it can't be supported in nrtsearch. You either need to use the provided gateway OR use transcoding in a proxy like envoy OR use grpc directly.

Jeevananthan-23 commented 1 year ago

If I choose to run grpc directly with .net which port address should I have to talk to primary port https\\:localhost:6000. This port is now configured in yaml. The below error is happening whenever I try to connect with nrtsearch grpc server.

io.grpc.netty.shaded.io.grpc.netty.NettyServerTransport notifyTerminated
INFO: Transport failed
io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception: HTTP/2 client preface string missing or corrupt. Hex dump for received bytes: 16030100ff010000fb0303cd22c1d466ccc4e30f1d5b0306
sarthakn7 commented 1 year ago

I am not familiar with .net but it appears that you are sending an http request instead of grpc. You can try using the lucene-client which sends grpc requests (examples in README).

Jeevananthan-23 commented 1 year ago

If I choose to run grpc directly with .net which port address should I have to talk to primary port https\\:localhost:6000. This port is now configured in yaml. The below error is happening whenever I try to connect with nrtsearch grpc server.

io.grpc.netty.shaded.io.grpc.netty.NettyServerTransport notifyTerminated
INFO: Transport failed
io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception: HTTP/2 client preface string missing or corrupt. Hex dump for received bytes: 16030100ff010000fb0303cd22c1d466ccc4e30f1d5b0306

The above error is the case due to TLS not being enabled Link: https://stackoverflow.com/a/46123281