search

facebook / proxygen

A collection of C++ HTTP libraries including an easy to use HTTP server.
Other
8.16k stars 1.5k forks source link

HTTP3 Server and Client from samples #472

Closed master-of-darkness closed 2 months ago

master-of-darkness commented 1 year ago

Hi there!

I'm trying to figure out why the HQ client from the samples can't connect to the HQ server (also from the samples). I am able to connect using HTTP/2 (HTTPS) from the browser, but QUIC doesn't seem to work at all. I noticed that hqParams.supportedAlpns includes hq-29. Could the issue be related to the ALPN or the certificate?

server run command:

sudo     ./proxygenhttp3_2serv    --mode=server \
        --h2port=8080 \
        --port=8443 \
        --protocol=hq-29 \
        --host=127.0.0.1

client run command: ./hq --mode=client --path=/ --host=127.0.0.1 --port=8443 --protocol=hq-29 server log:

I1101 23:22:18.819690 14063 HQServer.cpp:286] HQ server started at: 127.0.0.1:8443 E1101 23:26:13.730489 14073 HQSession.cpp:413] next protocol not supported: hq-29 sess=proto=, UA=, client CID=, server CID=4000000a27839153, downstream=127.0.0.1:49528, 127.0.0.1:8443=local, drain=none E1101 23:26:13.730515 14073 HQSession.cpp:492] Peer closed with error err=Connect failed msg=ALPN not supported proto=, UA=, client CID=, server CID=4000000a27839153, downstream=127.0.0.1:49528, 127.0.0.1:8443=local, drain=none

client log:

I1101 23:26:13.728768 14319 HQClient.cpp:48] HQClient connecting to 127.0.0.1:8443 E1101 23:26:13.730736 14319 HQClient.cpp:65] Failed to establish QUIC connection: Client closed by peer reason=Internal error E1101 23:26:13.730742 14319 HQClient.cpp:255] HQClient failed to connect, error=Internal Error, msg=Client closed by peer reason=Internal error E1101 23:26:13.730860 14319 QuicTransportBase.cpp:417] close threw exception Cannot encrypt (insufficient space for tag) client CID= server CID=4000000a27839153 peer address=127.0.0.1:8443

update 02.11.2023: HQClient works well with h3, but firefox and chrome still work only with http2 Also, in both browsers I have

alt-svc | h3-27=":443"; ma=3600

afrind commented 1 year ago

I usually skip the --protocol arg completely -- does that fix your issues? I know chrome -> hq server is working.

master-of-darkness commented 1 year ago

Unfortunately it doesn't work :(

afrind commented 1 year ago

If you just run

./hq --mode=server --cert=<your cert> --key=<your key> --port 8443 -v 4

And hit with chrome, what's the server output?

master-of-darkness commented 1 year ago

If you just run

./hq --mode=server --cert=<your cert> --key=<your key> --port 8443 -v 4

And hit with chrome, what's the server output?

E1109 12:34:45.815054  7856 main.cpp:44] Cannot open /tmp/logs
I1109 12:34:45.816812  7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.816824  7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.817651  7856 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.817658  7856 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.818532  7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.818539  7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN: 
I1109 12:34:45.818593  7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.818710  7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f6788017d40
I1109 12:34:45.818728  7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.819213  7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.819218  7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.819875  7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.819881  7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN: 
I1109 12:34:45.819924  7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.819931  7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67880bf360
I1109 12:34:45.819937  7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.820389  7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.820394  7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.821027  7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.821031  7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN: 
I1109 12:34:45.821075  7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.821081  7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67880cde90
I1109 12:34:45.821087  7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.821547  7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.821552  7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.822196  7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.822201  7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN: 
I1109 12:34:45.822264  7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.822273  7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67880dc390
I1109 12:34:45.822283  7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.822836  7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.822841  7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.823822  7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.823828  7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN: 
I1109 12:34:45.823876  7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.823884  7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67880eb2d0
I1109 12:34:45.823891  7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.824409  7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.824412  7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.825254  7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.825261  7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN: 
I1109 12:34:45.825313  7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.825321  7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67880f99d0
I1109 12:34:45.825330  7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.825836  7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.825839  7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.826503  7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.826508  7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN: 
I1109 12:34:45.826552  7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.826558  7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67880f8950
I1109 12:34:45.826565  7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.827070  7857 AeadTokenCipher.cpp:39] Updating token secrets
I1109 12:34:45.827075  7857 AeadTokenCipher.cpp:48] Updating token secrets, num=1
I1109 12:34:45.827760  7857 SSLContextManager.cpp:833] cert /etc/http32webserver/cert/localhost.crt Identity: localhost
I1109 12:34:45.827769  7857 SSLContextManager.cpp:836] cert /etc/http32webserver/cert/localhost.crt SAN: 
I1109 12:34:45.827828  7857 SSLContextManager.cpp:732] For vip vip_, setting sid_ctx localhost
I1109 12:34:45.827836  7857 SSLContextManager.cpp:740] On VipID=[::1]:6667 context=0x7f67881168a0
I1109 12:34:45.827843  7857 SSLContextManager.cpp:1091] Adding CN/Subject-alternative-name "localhost" for SNI search
I1109 12:34:45.828923  7866 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7a170 workerId=0 processId=0
I1109 12:34:45.828984  7867 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7a940 workerId=1 processId=0
I1109 12:34:45.829016  7868 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7b110 workerId=2 processId=0
I1109 12:34:45.829037  7869 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7b8e0 workerId=3 processId=0
I1109 12:34:45.829056  7870 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7c0d0 workerId=4 processId=0
I1109 12:34:45.829073  7872 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7c8e0 workerId=5 processId=0
I1109 12:34:45.829092  7873 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7d0d0 workerId=6 processId=0
I1109 12:34:45.829123  7874 QuicServer.cpp:274] No valid takenover fd found for address=[::1]:8443. binding on worker=0x2b7d8e0 workerId=7 processId=0
I1109 12:34:45.829139  7874 QuicServer.cpp:285] Initialized all workers in the eventbase
I1109 12:34:45.829190  7856 HQServer.cpp:285] HQ server started at: [::1]:8443
I1109 12:34:47.248520  7858 AsyncSSLSocket.cpp:340] actual destruction of AsyncSSLSocket(this=0x7f6780002550, evb=0x7f6780000d80, fd=folly::NetworkSocket(-1), state=3, sslState=9, events=0)
I1109 12:34:47.248592  7858 AsyncSocket.cpp:3169] this=0x7f6780003560, AsyncSocket::handleRead() got 517 bytes
I1109 12:34:47.255234  7858 AsyncSocket.cpp:3169] this=0x7f6780003560, AsyncSocket::handleRead() got 30 bytes
I1109 12:34:47.255358  7858 FizzAcceptorHandshakeHelper.cpp:153] Fizz handshake error with (peer=[::1]:45860, local=[::1]:6667) after 6 ms; 547 bytes received & 2120 bytes sent: fizz::FizzException: received alert: certificate_unknown, in state ExpectingFinished
I1109 12:34:47.255385  7858 Acceptor.cpp:548] Acceptor=0x7f6788007680 onEmpty()
I1109 12:34:48.002563  7859 AsyncSSLSocket.cpp:340] actual destruction of AsyncSSLSocket(this=0x7f67840021e0, evb=0x7f6784000d40, fd=folly::NetworkSocket(-1), state=3, sslState=9, events=0)
I1109 12:34:48.002631  7859 AsyncSocket.cpp:3169] this=0x7f6784002f00, AsyncSocket::handleRead() got 517 bytes
I1109 12:34:48.010644  7859 AsyncSocket.cpp:3169] this=0x7f6784002f00, AsyncSocket::handleRead() got 30 bytes
I1109 12:34:48.010754  7859 FizzAcceptorHandshakeHelper.cpp:153] Fizz handshake error with (peer=[::1]:45868, local=[::1]:6667) after 8 ms; 547 bytes received & 2120 bytes sent: fizz::FizzException: received alert: certificate_unknown, in state ExpectingFinished
I1109 12:34:48.010775  7859 Acceptor.cpp:548] Acceptor=0x7f678804b2d0 onEmpty()
master-of-darkness commented 1 year ago

On older version(10.16)

I1109 13:27:25.661306 67818 QuicServerWorker.cpp:1134] QuicServer readerr: AsyncSocketException: AsyncUDPSocket::getReadBuffer() returned empty buffer, type = Invalid arguments
I1109 13:27:25.661463 67811 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661550 67812 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661600 67813 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661643 67814 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661676 67815 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661716 67816 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661752 67817 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
I1109 13:27:25.661772 67818 QuicServerWorker.cpp:1384] QuicServer shutdown all connections. addressMap=0 connectionIdMap=0
afrind commented 1 year ago

Relevant log line:

I1109 12:34:47.255358 7858 FizzAcceptorHandshakeHelper.cpp:153] Fizz handshake error with (peer=[::1]:45860, local=[::1]:6667) after 6 ms; 547 bytes received & 2120 bytes sent: fizz::FizzException: received alert: certificate_unknown, in state ExpectingFinished

Is your cert unexpired and trusted by your browser?

master-of-darkness commented 1 year ago

Relevant log line:

I1109 12:34:47.255358 7858 FizzAcceptorHandshakeHelper.cpp:153] Fizz handshake error with (peer=[::1]:45860, local=[::1]:6667) after 6 ms; 547 bytes received & 2120 bytes sent: fizz::FizzException: received alert: certificate_unknown, in state ExpectingFinished

Is your cert unexpired and trusted by your browser?

My certificate is unexpired and added to my browser image

PooyaEimandar commented 1 year ago

I have the same issue on macOS, hq http3 not working with browsers but it works with HQClient.

Update: It works on FireFox nightly.

qtuancr261 commented 1 year ago

I'm working with proxygen <v2023.10.16.00>. HQServer works well with these parameters : --host=:: --port=443 --h2port==443 --protocol=h3 --quic_version=0. Google Chrome 119.0.6045.159, Mozilla Firefox 120.0, Microsoft Edge 119.0.2151.58 could setup a h3 connection to HQServer without problem. image

PooyaEimandar commented 1 year ago

I'm working with proxygen <v2023.10.16.00>. HQServer works well with these parameters : --host=:: --port=443 --h2port==443 --protocol=h3 --quic_version=0. Google Chrome 119.0.6045.159, Mozilla Firefox 120.0, Microsoft Edge 119.0.2151.58 could setup a h3 connection to HQServer without problem. image

I tested it on Google Chrome on macOS, but unfortunately, it didn't work. It seems that your test was conducted on Windows OS.

qtuancr261 commented 1 year ago

I'm working with proxygen <v2023.10.16.00>. HQServer works well with these parameters : --host=:: --port=443 --h2port==443 --protocol=h3 --quic_version=0. Google Chrome 119.0.6045.159, Mozilla Firefox 120.0, Microsoft Edge 119.0.2151.58 could setup a h3 connection to HQServer without problem. image

I tested it on Google Chrome on macOS, but unfortunately, it didn't work. It seems that your test was conducted on Windows OS. I built and tested on Rocky Linux 9 - not Windows, btw i didn't use the default TLS certificates in the sample code https://github.com/facebook/proxygen/blob/main/proxygen/httpserver/samples/hq/FizzContext.cpp, just imported the real TLS certificates from Let's Encrypt

afrind commented 2 months ago

What's the current status here - are folks able to make it work with chrome/firefox on various platforms?

PooyaEimandar commented 2 months ago

What's the current status here - are folks able to make it work with chrome/firefox on various platforms?

Hi, on MacBook Pro, Apple M2 Max as the client and Proxygen as the server on a linux machine.

  1. Firefox Nightly supports h3.
  2. Chrome supports h3.
  3. Safari does not receive h3 from Proxygen, although it does receive h3 from the Cloudflare asset server. (Should I downgrade QUIC version in proxygen? )
afrind commented 2 months ago

Can you open a separate issue for Safari + proxygen h3 server?

afrind commented 2 months ago

Are you using these args: --h2port==443 --protocol=h3 --quic_version=0 ?

These are important for some browsers, which require getting an Alt-Svc header from the h2 server before attempting QUIC. Please include the -v 4 debug logs from the server on a failed attempt.