gRPC-web to gRPC translation

stas-neverov commented 1 year ago

What should we add or change to make your life better?

Support gRPC-web to gRPC translation.

Why is this important to you?

Many language platforms lack gRPC-web support - C++, Python, Node.js. The only reverse proxy officially supporting gRPC-web to gRPC translation is Envoy. But if .NET YARP is already employed in some product and it supports gRPC-web to gRPC translation then it would be a more lightweight solution to integrate services written in other languages and expose their API to HTML frontend.

Tratcher commented 1 year ago

That should work with YARP if you follow the instructions here: https://learn.microsoft.com/en-us/aspnet/core/grpc/grpcweb?view=aspnetcore-7.0#configure-grpc-web-in-aspnet-core

Be sure to include the DefaultEnabled = true setting.

cc: @JamesNK

If this does work then we should add it to the docs: https://microsoft.github.io/reverse-proxy/articles/grpc.html#grpc-web

stas-neverov commented 1 year ago

@Tratcher, thanks for your response.

Do you mean that Grpc.AspNetCore.Web could perform the gRPC-web-to-gRPC translation for a gRPC service that runs in a different process and then YARP could redirect to that gRPC service running in a separate process?

The setup I'm looking for is the following:

A few C++ and Python gRPC services are running as independent processes.
They register themselves in some service registry.
There is a ASP.NET Core reverse proxy that gets gRPC-web requests from web UI, translates them to gRPC, then maps the request to one of gRPC services based on metadata from the service registry, and proxies the gRPC requests to those gRPC services.

Tratcher commented 1 year ago

Yes, Grpc.AspNetCore.Web could perform the gRPC-web-to-gRPC translation in the YARP process, and then YARP could proxy the request as gRPC to the destination.

For the service registry, see the different ways of configuring YARP: https://microsoft.github.io/reverse-proxy/articles/config-files.html https://microsoft.github.io/reverse-proxy/articles/config-providers.html https://microsoft.github.io/reverse-proxy/articles/direct-forwarding.html

stas-neverov commented 1 year ago

It almost worked for me:

fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
      An unhandled exception has occurred while executing the request.
      System.NotSupportedException: Writing to the response stream during a gRPC call is not supported.
         at Grpc.AspNetCore.Web.Internal.GrpcWebFeature.get_Stream()
         at Microsoft.AspNetCore.Http.DefaultHttpResponse.get_Body()
         at Yarp.ReverseProxy.Forwarder.HttpForwarder.SendAsync(HttpContext context, String destinationPrefix, HttpMessageInvoker httpClient, ForwarderRequestConfig requestConfig, HttpTransformer transformer)
         at Yarp.ReverseProxy.Forwarder.ForwarderMiddleware.Invoke(HttpContext context)
         at Yarp.ReverseProxy.Health.PassiveHealthCheckMiddleware.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
         at Grpc.AspNetCore.Web.Internal.GrpcWebMiddleware.HandleGrpcWebRequest(HttpContext httpContext, ServerGrpcWebContext grcpWebContext)
         at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

gRPC-web requests are translated to gRPC, YARP calls downstream gRPC service and gets HTTP 200 response, but then the above exception occures. Please advise how to troubleshoot this. Thank you!

JamesNK commented 1 year ago

This is a limitation of the middleware. PR to fix - https://github.com/grpc/grpc-dotnet/pull/1993

gRPC has a custom NuGet feed, so you should be able to get the updated NuGet package from their a day after the PR is merged. Alternatively you could do something like wrap IHttpResponseBodyFeature feature, and create a stream from the writer like the PR change is doing.

I recommend waiting a couple of days.

stas-neverov commented 1 year ago

I've tested the fix with the Grpc.AspNetCore.Web nightly build. Everything works fine.

Thank you, @JamesNK, @Tratcher!

karelz commented 1 year ago

We should document it in YARP, how to do it.

MikeThomas-Dev commented 6 days ago

I would like to pick up on this issue as I am currently struggling to setup the system described here to be supported by yarp.

The setup

grpc C++ server, listening on Unix Domain Socket (UDS)
ASP.NET Core service with yarp, proxying incoming requests to UDS as described here, general grpc-web configuration as described here
Angular SPA hosted by ASP.NET Core service using grpc-web

All libraries and frameworks are used in the current master/latest version. grpc-web protobuf/generated code uses wireformat grpcwebtext. (because server streaming is required) Same behavior is reproducible when using wireformat grpcweb.

Current issue

grpc-web unary call from Angular SPA returns with response:

 "message": "Missing :te header",

Breakpoint in grpc C++ server method implementation is not hit.

grpc C++ server configuration

    ApiService service;
    grpc::EnableDefaultHealthCheckService(true);
    grpc::reflection::InitProtoReflectionServerBuilderPlugin();
    grpc::ServerBuilder builder;
    std::string serverAddress = "unix:E:/source/grpc_prototype/api.sock";

    builder.AddListeningPort(serverAddress, grpc::InsecureServerCredentials());
    builder.RegisterService(&service);
    std::unique_ptr<grpc::Server> server(builder.BuildAndStart());
    std::cout << "Server listening on " << serverAddress << std::endl;
    server->Wait();

yarp configuration

Appsettings.json:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning",
      "Yarp": "Debug"
    }
  },
  "AllowedHosts": "*",
  "Kestrel": {
    "Endpoints": {
      "http": {
        "Url": "https://localhost:5000"
      }
    }
  },
  "ReverseProxy": {
    "Routes": {
      "route1": {
        "ClusterId": "cluster1",
        "Match": {
          "Path": "{**catch-all}"
        }
      }
    },
    "Clusters": {
      "cluster1": {
        "HttpRequest": {
          "Version": "2",
          "VersionPolicy": "RequestVersionExact"
        },
        "Destinations": {
          "destination1": {
            "Address": "http://localhost"
          }
        }
      }
    }
  }
}

Program.cs

using System.Net.Sockets;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddCors(options =>
{
  options.AddPolicy("customPolicy", builder =>
  {
    builder.AllowAnyOrigin()
    .AllowAnyHeader()
    .AllowAnyMethod()
    .WithExposedHeaders("Grpc-Status", "Grpc-Message", "Grpc-Encoding", "Grpc-Accept-Encoding");
  });
});

builder.Services.AddReverseProxy()
  .LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"))
 .ConfigureHttpClient((context, handler) =>
 {
   handler.ConnectCallback = async (context, cancellation) =>
   {
     var udsEndPoint = new UnixDomainSocketEndPoint("E:/source/grpc_prototype/pi.sock");
     var socket = new Socket(AddressFamily.Unix, SocketType.Stream, ProtocolType.Unspecified);

     try
     {
       await socket.ConnectAsync(udsEndPoint, cancellation).ConfigureAwait(false);
       return new NetworkStream(socket, true);
     }
     catch
     {
       socket.Dispose();
       throw;
     }
   };
 });

builder.Services.AddGrpc();

var app = builder.Build();

app.UseCors("customPolicy");

app.MapReverseProxy();

app.UseGrpcWeb(new GrpcWebOptions { DefaultEnabled = true });

app.Run();

grpc-web client configuration

    const grpcClient = new ContextClient("https://localhost:5000");
    const contextRequest = new ContextRequest();
    contextRequest.setSamplerequestcontent("This is the sample request content text");
    grpcClient.getContext(contextRequest, null, (err: grpcWeb.RpcError, response: api_pb.ContextReply) => {
      console.log(err);
      console.log(response);
    });

@Tratcher Help would be highly appreciated, or should I open a new issue and link this one there?

PS: While waiting for a reply I tried to create the setup described here using a C# server without UDS. Using wire format grpc-web for the grpc-web client code the calls were successfully forwarded. When switching to wire format grpc-web-text the grpc server reported an Invalid.DataException: "Unexpected compressed flag value in message header". For this I could share a minimal reproduction sln. Please advice if the yarp repository is the correct place for a new issue regarding this.

microsoft / reverse-proxy