akka-grpc gives incomplete response when using grpc-web

[UnsolvedCypher]

Versions used

Akka version: akka-grpc version 2.1.0

Expected Behavior

Akka gRPC should provide valid responses when a gRPC-web endpoint is hit.

Actual Behavior

Akka gRPC provides a very truncated response when using Akka's built-in gRPC web or using a proxy like Envoy.

Reproducible Test Case

Here is a repository demonstrating the issue: https://github.com/UnsolvedCypher/akka-grpc-bug

Steps to reproduce

1. Build the project with sbt and run the main method in Main.scala
2. This will start up a gRPC server on :8080 and a gRPC-web server on :8081
3. You can hit both :8080 and :8081 with a gRPC request, and note that the response is returned correctly. The protobuf is located in main/protobuf/hello.proto
4. You can hit :8081 with a gRPC-web request, and note that an invalid (and very short) response is returned.
5. Optionally, you can install Envoy Proxy, and run envoy -c envoy.yaml in the repository's root directory. This will start a proxy on :9000 which will route gRPC-web requests to :8080 where Akka is listening
6. You can hit :9000 with a gRPC-web request, and like with Akka's built-in gRPC-web, you will get a very truncated response

Note that this issue affects only akka-grpc 2.1.0, and downgrading to 2.0.0 fixes all of these issues.

Please let me know if there is further information I can provide!

[raboof]

Thanks for the detailed report, that is very valuable! We did fix some HTTP/2 things that could possibly be related in Akka HTTP 10.2.7 (e.g. https://github.com/akka/akka-http/pull/3913), so my first step would be to try if the problem can still be reproduced after updating to Akka HTTP 10.2.7. Akka HTTP releases are binary compatible so it should be a drop-in replacement, or you could use a snapshot version of Akka gRPC.

I hope to look into this soon but have some other things on my list as well, let's see who gets there first ;).

[drmontgomery]

This sounds very similar to what we encountered in https://github.com/akka/akka-grpc/issues/1459. As such, it should be fixed by https://github.com/akka/akka-grpc/pull/1462.

Just to confirm, I synced the akka-grpc-bug repo and added a quick Python gRPC-web client using Sonora. I was able to reproduce the bug when using akka-grpc-runtime v2.1.0, and able to successfully make requests when using akka-grpc-runtime built from head (currently 19307ec7 for me).

[UnsolvedCypher]

@drmontgomery, thank you very much for looking into this! I am happy to hear this has already been fixed.

@raboof, thank you for the quick reply! Would it be possible to get a release out pretty soon? I understand there are other open PRs, but this is a pretty major showstopper bug.

[raboof]

This sounds very similar to what we encountered in #1459. As such, it should be fixed by #1462.

Ah, of course, good point

Would it be possible to get a release out pretty soon?

Yes, that seems like a good idea. Since it looks like you have a reproducer ready, could you verify that the problem is indeed resolved for you on a development snapshot? Those are available from sonatype snapshots (https://oss.sonatype.org/content/repositories/snapshots/com/lightbend/akka/grpc/akka-grpc-maven-plugin/), 2.1.0-17-19307ec7-SNAPSHOT should have the fix. If you can confirm I'll cut the release.

[raboof]

I'm not too good with gRPC-web yet, but this looks good to me

While on 2.1.0 it indeed aborted:

I'll start a release, #1488

[raboof]

2.1.1 is now available on Maven Central, which should fix this issue. I'll close this for now, but feel free to re-open when the problem persists!

[fatemahasta]

@raboof Hello, I am trying to get gRPC-Web working with Akka gRPC, using WebHandler.grpcWebHandler(). I am sending a GET request using a http client, but somehow this is the error i see. I was able to get my application working with grpc web using Envoy, where the browser is able to send a request and receive a response. The native gRPC request-response is also working as expected. Can someone help with where I might be going wrong? Attaching proto file and other necessary files.

using WebHandler.grpcWebHandler()

        /**
        * Akka gRPC with gRPC Web requests
         */

         final Function<HttpRequest, CompletionStage<HttpResponse>> Service =
                HandlerFactory.create(new GrpcWebImpl());
        final Function<HttpRequest, CompletionStage<HttpResponse>> ServiceWeb =
                ServiceHandlerFactory.create(new AkkaGrpcWebImpl());

        final Function<HttpRequest, CompletionStage<HttpResponse>> webHandler =
                WebHandler.grpcWebHandler(Arrays.asList(Service , ServiceWeb), context.getSystem(), mat);

        Http.get(context.getSystem())
                .newServerAt("127.0.0.1", 1001)
                .bind(webHandler);

.proto file:

service Service{
  rpc EntityHsvtStats (Entity) returns (protobuf.stats.HsvtStats) {
  }
}

message Entity {
  string entity = 1;
}

Output:

curl http://127.0.0.1:1001/Service/EntityHsvtStats/dps6pa

Unsupported media type

I am using akka-http version 10.2.4 and for akka-http-cors_2.13 version 1.1.2 (akka-grpc version 2.0.0 )

Tagging a few other members too for help: @dwijnand @johanandren

[raboof]

akka-grpc version 2.0.0

The issue you're responding to was fixed in Akka gRPC 2.1.1, so it might be good to upgrade to that at least? Though I'm not sure it's related.

Shouldn't gRPC-Web requests have either the application/grpc-web or the application/grpc-web-text content type?

[fatemahasta]

Thank you for responding @raboof . I did upgrade to 2.1.1, but still the same error. I also did send the request along with setting the Content-Type header (if that's what your asking): Also, upon upgrading to 2.1.1, it broke my native gRPC that was already working with 2.0.0, to get it to work with 2.1.1 i had to upgrade akka-http to 10.2.7, but still cannot send http web request Also is it possible to send query parameters with this current implementation like i did in http://127.0.0.1:1001/Service/EntityHsvtStats/dps6pa, where "dps6pa" is essentially message Entity? or am i using the curl wrong?

[fatemahasta]

Thank you for responding @raboof . I did upgrade to 2.1.1, but still the same error. I also did send the request along with setting the Content-Type header (if that's what your asking): Also, upon upgrading to 2.1.1, it broke my native gRPC that was already working with 2.0.0, to get it to work with 2.1.1 i had to upgrade akka-http to 10.2.7, but still cannot send http web request Also is it possible to send query parameters with this current implementation like i did in http://127.0.0.1:1001/Service/EntityHsvtStats/dps6pa, where "dps6pa" is essentially message Entity? or am i using the curl wrong?

Any update for me @raboof ?

[raboof]

to get it to work with 2.1.1 i had to upgrade akka-http to 10.2.7

Good to hear you got it to work with recent versions.

is it possible to send query parameters with this current implementation like i did in http://127.0.0.1:1001/Service/EntityHsvtStats/dps6pa, where "dps6pa" is essentially message Entity? or am i using the curl wrong?

I don't think that's how gRPC-Web works

[fatemahasta]

Good to hear you got it to work with recent versions.

No wasn't able to get gRPC-Web working with the latest versions.

I don't think that's how gRPC-Web works

okay so essentially i cannot send mesages as parameters via grpc web curl call, like we can using envoy

[fatemahasta]

with envoy, i have been using it, this way:

service DtrnService { rpc QueryEntitySingleHsvtStats (EntityId) returns (protobuf.stats.HsvtStats) { option (google.api.http) = { get: "/v1/entities/{entity}" }; }

message EntityId { string entity = 1; }

and access the api using curl : http://localhost:port/v1/entities/dps6nd where dps6nd is essentially the entityId.

So i am looking at doing the same with grpc web using WebHandler.grpcWebHandler()

@raboof I dont see any documentation or examples stating the same, so not sure if this is even possible without envoy

[raboof]

option (google.api.http) = { get: "/v1/entities/{entity}" };

Aha, this is not gRPC-Web, this feature is called 'HTTP/JSON Transcoding'. Akka gRPC does not currently support this 'directly' - we would like to, and this is tracked in #1461, but there is no-one currently working on this AFAIK. You can, however, build this 'yourself' on top of Akka gRPC, and https://github.com/cloudstateio/cloudstate/blob/master/proxy/core/src/main/scala/io/cloudstate/proxy/HttpApi.scala might be a good starting point.