[Question] stream

[solarhell]

I have a connectrpc backend server that provides frontend access via Nginx. Now I'm finding some issues with one-way streaming. I checked the documentation suggesting that I should use apache or enovy to proxy the stream. Is it possible to use the stream method properly if access is provided directly by connectrpc's http.Server without any proxy server software? FYI, the one-way stream is like rpc StreamChat(StreamChatRequest) returns (stream StreamChatResponse); and the err code in chrome shows: net::ERR_HTTP2_PROTOCOL_ERROR 200 (OK)

[tahadostifam]

Hi, If you used reverse proxy with Nginx you need to specify somehow to use http2. I guess your problem because Nginx uses http1.1 by default and gRPC streams only work with http2.

And can i ask why you use envoy proxy? connectrpc has built-in json gateway it self and does not require external gateways like envoy... but if you want reverse proxy it's different...

[solarhell]

https://connectrpc.com/docs/faq/#how-do-i-proxy-the-connect-protocol-through-nginx

As the doc said that nginx can't proxy stream. I am using caddy now and it works fine. Next time i will try just use connectrpc's go http server to serve the stream.

[tahadostifam]

Good to see you fixed it, but you know caddy has really bad performance? If performance is important for your project you are in a big trouble.

Take a look at the litespeed. I don't know it has streaming features but offers good performance.

Good Luck👾🤌🏿

[jhump]

That error message looks like it may be related to the HTTP/2-related configuration of your nginx server, and not necessarily related to the use of a server-streaming RPC with nginx.

Streaming RPCs typically require end-to-end HTTP/2, which NGINX doesn't support.

IIUC, this statement in the FAQs refers to bidirectional streams, which typically cannot work with HTTP 1.1 since they require full-duplex communication (which is only possible when using HTTP/2 or web sockets). The issue with nginx is that it can only use HTTP/2 when communicating with the client, and only supports HTTP 1.1 when communicating with a backend server. So any incoming HTTP/2 request will be downgraded to HTTP 1.1 when proxied to a backend server.

But one-way streaming operations are half-duplex -- so they can still work with nginx because they can work with HTTP 1.1.

Is it possible to use the stream method properly if access is provided directly by connectrpc's http.Server without any proxy server software?

Yes, streaming definitely works if the client connects directly to the connect server w/out the use of a proxy. A proxy is not necessary for any functionality with connect.

Could you possibly provide more details on the nginx and backend server config? What client are you using -- are you using connect-web and, if so, what protocol is the client configured to use?

[solarhell]

here is my openresty config

server {
    listen 80 ; 
    listen 443 ssl http2 ; 
    server_name api.mydomain.com; 
    proxy_set_header Host $host; 
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 
    proxy_set_header X-Forwarded-Host $server_name; 
    proxy_set_header X-Real-IP $remote_addr; 
    proxy_http_version 1.1; 
    proxy_set_header Upgrade $http_upgrade; 
    proxy_set_header Connection "upgrade"; 
    include /www/sites/api.mydomain.com/proxy/*.conf; 
    if ($scheme = http) {
        return 301 https://$host$request_uri; 
    }
    ssl_certificate /www/sites/api.mydomain.com/ssl/fullchain.pem; 
    ssl_certificate_key /www/sites/api.mydomain.com/ssl/privkey.pem; 
    ssl_protocols TLSv1.3 TLSv1.2 TLSv1.1 TLSv1; 
    ssl_ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:!aNULL:!eNULL:!EXPORT:!DSS:!DES:!RC4:!3DES:!MD5:!PSK:!KRB5:!SRP:!CAMELLIA:!SEED; 
    ssl_prefer_server_ciphers on; 
    ssl_session_cache shared:SSL:10m; 
    ssl_session_timeout 10m; 
    error_page 497 https://$host$request_uri; 
    proxy_set_header X-Forwarded-Proto https; 
    add_header Strict-Transport-Security "max-age=31536000"; 
}

my vhost config

location ^~ / {
    proxy_pass http://127.0.0.1:23333; 
    proxy_set_header Host $host; 
    proxy_set_header X-Real-IP $remote_addr; 
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 
    proxy_set_header REMOTE-HOST $remote_addr; 
    proxy_set_header Upgrade $http_upgrade; 
    proxy_set_header Connection "upgrade"; 
    proxy_set_header X-Forwarded-Proto $scheme; 
    proxy_http_version 1.1; 
    add_header X-Cache $upstream_cache_status; 
    add_header Strict-Transport-Security "max-age=31536000"; 
}

and in my front-end code, i use connect-web

[jhump]

These lines in the config look suspicious:

    proxy_set_header Upgrade $http_upgrade; 
    proxy_set_header Connection "upgrade"; 

Do you also have web socket endpoints? IIRC, most browsers and most servers do not actually support web sockets over HTTP/2, so this could be the issue. The "upgrade" request header is not supported with HTTP/2.

[solarhell]

Yes, i also have websocket endpoints.

I will try to comment the nginx upgrade config and test the one-way stream.

[jhump]

@solarhell, if you also need to use web sockets, I'd recommend using plain HTTP 1.1 in the nginx config and not trying to use HTTP/2 at all. That should work fine for web sockets and for one-way streaming with Connect.

[tahadostifam]

Dear @jhump, The streaming documents for nodejs, connect-web is not complete so much on the site. If i work on them would connectrpc.com welcome that?

[jhump]

@tahadostifam, thank you for offering. I see you already have a PR over inhttps://github.com/connectrpc/connectrpc.com. I'll add comments over there.