Support for content-type: application/grpc-web-text

[mier85]

So I was trying to use the in-process proxy to wrap my grpc-server to receive grpc-web requests. I formerly used envoy and it worked with the same client javascript code, however I am running into an issue with the grpc-web-text content-type. Is that content-type actually supported? I saw that there is some string replaced inside the wrapper :

https://github.com/improbable-eng/grpc-web/blob/b89ec6300fb9ce3f604ff02c22e1c106fe29a3b9/go/grpcweb/wrapper.go#L198

Effectively this converts the context header from application/grpc-web-text toapplication/grpc-text, which GRPC does not like ( it returns invalid gRPC request content-type ). After replacing that with application/grpc+json, via a wrapper, the grpc handler is successfully continuing, just to later fail with the message: "grpc: received message larger than max (1094795585 vs. 4194304)"

I suspect that grpc-web-text is just not supported at this point and so the necessary steps to transform it aren't happening thus?

[johanbrandhorst]

You've hit the nail on the head, support has not yet been implemented. Indeed, I think you might be the first to try the improbable js client with the Envoy proxy implementation at all. Good to know that was working at least (in your use cases).

This being an open source project, would you be interested in contributing a fix for this bug? I haven't looked at the spec recently, but I imagine this will need a bit of extra logic in the proxy itself.

[mier85]

Hi, thanks for the answer! Sure, I'd be glad to contribute. I'll check the specs and come up with a PR

[JBetser]

Hi mier, Thanks for your post. I have hit exactly the same issue as you: var client = new pckg.client("http://hostname:8001"); var metadata = { 'content-type': 'application/grpc-web-text' }; var req = new pckg.Void(); let tickers = client.getExchangeTickers(req, metadata, function (err, response) { ... returns a 500 internal server error with code 14 (unavailable) when I use var metadata = { 'content-type': 'application/grpc-web+proto' }; or var metadata = { 'content-type': 'application/grpc-web+json' }; I get exactly the same error message as you: grpc: received message larger than max (1094795585 vs. 4194304) I am really looking forward to this fix! :) the server I use is on IIS, and it looks like envoy is proper pain to make envoy work on windows. If I can be of any help please let me know. I'm not a golang programmer but I guess I can do testing ;)

[johanbrandhorst]

@JBetser, forgive me, but have you tried simply using the binary encoding option? Or using the client in this repo? Both should work with the proxy.

[JBetser]

I need a javascript client working on a web page (this is for a html widget) so I am using grpc/grpc-web javascript client. your grpc-web-proxy is very useful for me to be able to route grpc web request to my grpc server (and it seems to handle CORS nicely), but I cannot see how I could use the golang client Thanks

[johanbrandhorst]

This repo has an alternative implementation of a JavaScript client which works out of the box with the proxy, but you can also use the grpc/grpc-web client in binary mode. Have you tried either of those options? You need not wait for this bug to be fixed.

[JBetser]

@johanbrandhorst you are right! I just realized the binary mode has to be specified during the generation of schema client js files with protoc. I wrongly assumed that by changing the metadata the client would switch to binary dynamically. I am going to test that right now. And if it does not work I will definitely try with your javascript client. Thanks very much for your help

[johanbrandhorst]

@mier85 This might help: https://github.com/envoyproxy/envoy/blob/10dcbb13d04768cf3e9bbca55ccefc2be0e704ef/source/extensions/filters/http/grpc_web/grpc_web_filter.cc#L78. It's the source for the C++ proxy that grpc/grpc-web uses, which supports the base64 text content type.

[johanbrandhorst]

Hi @mier85, are you still interested in implementing this?

[AnaitBI]

what is the status of this issue? I also need this.

Can't find any way to make it work.

my grpc client is react js, server grpc in csharp Docker on windows 10 with envoy proxy running.

I run this command and generated my js files:

sudo protoc -I=. proto.proto --js_out=import_style=commonjs:generated --plugin=protoc-gen-grpc-web=/home/grpc-web-master/javascript/net/grpc/web/protoc-gen-grpc-web --grpc-web_out=import_style=commonjs,mode=grpcwebtext:generated

Now in the client side:

import { ProtocolServiceClient } from '../output/proto_grpc_web_pb'
import { UserInfo } from "../output/proto_pb";

const grpcHost = "http://127.0.0.1:8085";
const client = new ServiceClient(grpcHost, {}, {});

const request = new UserInfo();
request.setName('a');
const metadata = {};
client.connect(request, metadata ,function(err, response) {
  if (err) {
    console.log(err.code);
    console.log(err.message);
  } else {
    console.log(response);
  }
});

Tried to add metaadata to the request with so many types of metadata, nothing worked

metadata = {'content-type': 'application/grpc-web-text'};
metadata = {'content-type': 'application/grpc'};
metadata = {'content-type': 'application/grpc-web'};
metadata = {'content-type': 'application/grpc-web+proto'};
metadata = {'content-type': 'application/grpc-web+json'};

And the result is: POST http://127.0.0.1:8085/ProtocolProtos.ProtocolService/Connect 503 (Service Unavailable) And receive back a gRPC error code 14 => upstream connect error or disconnect/reset before headers

What am I missing? @johanbrandhorst @JBetser @mier85 Please help me, I'm a struggling with this for a very long time.

Here is my envoy.yaml file:

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address:
      protocol: TCP
      address: 0.0.0.0
      port_value: 9903
static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 8085
    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        config:
          codec_type: auto
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match: { prefix: "/" }
                route:
                  cluster: protocol_service
                  max_grpc_timeout: 10s
              cors:
                allow_origin:
                - "*"
                allow_methods: GET, PUT, DELETE, POST, OPTIONS
                allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
                max_age: "1728000"
                expose_headers: custom-header-1,grpc-status,grpc-message
                enabled: true
          http_filters:
          - name: envoy.grpc_web
          - name: envoy.cors
          - name: envoy.router
  clusters:
  - name: protocol_service
    connect_timeout: 0.25s
    type: LOGICAL_DNS
    # Comment out the following line to test on v6 networks
    # dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    http2_protocol_options: {}
    hosts:
      - socket_address:
          address: 0.0.0.0
          port_value: 9090

And here is my Dockerfile:

FROM envoyproxy/envoy:latest
COPY envoy.yaml /etc/envoy/envoy.yaml
CMD /usr/local/bin/envoy -c /etc/envoy/envoy.yaml

To run the image I did:

docker build -f envoyDockerfile -t envoy:v14 . docker run -d --name envoyv148085 -p 9903:9903 -p 8085:8085 envoy:v14

[johanbrandhorst]

@AnaitBI I'm afraid you've confused this repo with github.com/grpc/grpc-web. You're using their proxy and generator, so this issue is not affecting you. You probably want to raise an issue against that repo instead.

[AnaitBI]

@johanbrandhorst Ok, This seemed that we are facing the same issues

[evanj]

I've made an initial attempt to support this: #292 . It makes grpcwebproxy work at least for successful unary requests.

There are capitalization differences with the grpc-status headers that make it not work for failed responses, which will need another fix (or possibly a specification/code update: HTTP headers are specified as being case insensitive, so it is a bit surprising that the grpc-web client does not consider them to be).

[johanbrandhorst]

The capitalisation only concerns in body trailers keys AFAIK.

[evanj]

Oops yes, I had an unrelated bug in my client that was causing me to be confused. This change appears to "work", at least for unary requests/responses! The grpc-web client itself doesn't handle some edge/error conditions well, but otherwise my PR "works" for my use case.

[johanbrandhorst]

Proxy support for this was added in #292 \o/. Client support and tests will be added later.