search

microsoft / reverse-proxy

A toolkit for developing high-performance HTTP reverse proxy applications.
https://microsoft.github.io/reverse-proxy
MIT License
8.43k stars 828 forks source link

gRPC headers are not proxied correctly #2520

Open bosh-3shape opened 3 months ago

bosh-3shape commented 3 months ago

Describe the bug

The setup is the following: a gRPC server that is behind YARP, and a client that calls the server. The server has a gRPC bidirectional streaming method exposed. Before the messages are sent on this call, the client and the server have to perform the header exchange:

  1. The client sends the headers to the server.
  2. The server sends the headers to the client.
  3. After the client receives the headers from the server, a regular message exchange starts (e.g., the client sends a message to the server, etc.).

What I observe is that step (3) never happens, as the client never receives the headers from the server.

If the client calls the server directly (without YARP), then step (3) happens, and the call procceeds.

To Reproduce

I've implemented a minimal repro here: https://github.com/bosh-3shape/YarpGrpcHeadersRepro

Further technical details

MihaZupan commented 2 months ago

Thank you for the great repro project!

This does appear to be a YARP issue where we're not flushing the response headers until we receive some content. Relevant section: https://github.com/microsoft/reverse-proxy/blob/8b53dac3d1a011a097883e9aa5eb504fe4c87426/src/ReverseProxy/Forwarder/HttpForwarder.cs#L269-L280