cloudflare / quiche

🥧 Savoury implementation of the QUIC transport protocol and HTTP/3
https://docs.quic.tech/quiche/
BSD 2-Clause "Simplified" License
9.4k stars 709 forks source link

curl/libcurl not supporting --http3 #319

Closed tonyclemmey closed 4 years ago

tonyclemmey commented 4 years ago

Hello,

While I understand this may not be the most appropriate place or repo for my issue I am just seeking help with running --http3 command following the Cloudflare Quick & HTTP3 article (https://blog.cloudflare.com/http3-the-past-present-and-future/) and I'm hoping this is the fastest and most active way.

I have followed the instruction for "If you're running macOS, we've also made it easy to install an HTTP/3 equipped version of curl via Homebrew"

Step 1) Install Homebrew Curl

$ brew install --HEAD -s https://raw.githubusercontent.com/cloudflare/homebrew-cloudflare/master/curl.rb

Step 2) Added Homebrew curl path to bash profile and checked along with version.

$ curl -V
curl 7.69.0-DEV (x86_64-apple-darwin18.7.0) libcurl/7.69.0-DEV SecureTransport zlib/1.2.11 brotli/1.0.7 libidn2/2.3.0 librtmp/2.3
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS brotli IDN IPv6 Largefile libz NTLM NTLM_WB SSL UnixSockets
$ which curl
/usr/local/opt/curl/bin/curl
$ curl-config --version
libcurl 7.69.0-DEV

Step 4) THE ISSUE

When I run the example command it says libcurl version doesn't support this.

$ curl -I https://blog.cloudflare.com/ --http3
curl: option --http3: the installed libcurl version doesn't support this
curl: try 'curl --help' or 'curl --manual' for more information

I can find very little information on this online...So I'm hoping someone would be able to help.

Tony

tonyclemmey commented 4 years ago

I have also just run the following and noticed curl is showing OS version and not Homebrew version even though "which curl" and "curl -V" is showing and using Brew version. (https://github.com/Homebrew/brew/issues/3433)

$ brew config
HOMEBREW_VERSION: 2.2.2
ORIGIN: https://github.com/Homebrew/brew
HEAD: 13f508d0dc8ada1726ee09a750285d7447ac3df4
Last commit: 3 weeks ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 915cd48b0fdb4d333785be0dbef72243ce6a2b05
Core tap last commit: 6 hours ago
HOMEBREW_PREFIX: /usr/local
CPU: octa-core 64-bit haswell
Homebrew Ruby: 2.6.3 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3/bin/ruby
Clang: 10.0 build 1001
Git: 2.20.1 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 7.54.0 => /usr/bin/curl
Java: 1.8.0_232
macOS: 10.14.6-x86_64
CLT: 10.3.0.0.1.1562985497
Xcode: 11.3
XQuartz: 2.7.11 => /opt/X11

So extra step I have attempted this and refused:

$ brew link curl --force
Warning: Refusing to link macOS-provided software: curl
If you need to have curl first in your PATH run:
  echo 'export PATH="/usr/local/opt/curl/bin:$PATH"' >> ~/.bash_profile

For compilers to find curl you may need to set:
  export LDFLAGS="-L/usr/local/opt/curl/lib"
  export CPPFLAGS="-I/usr/local/opt/curl/include"

For pkg-config to find curl you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/curl/lib/pkgconfig"
tonyclemmey commented 4 years ago

In my .bash_profile

export PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"

# Homebrew
export PATH="/usr/local/sbin:$PATH"

# Homebrew Curl Cloudflare HTTP/3
export PATH="/usr/local/opt/curl/bin:$PATH"
junhochoi commented 4 years ago

@tonyclemmey Thanks. I am able to reproduce on my macbook (10.14) as well. I think this is because something is wrong with BoringSSL build (required by quiche and curl tls support both) and eventually curl is built without quiche, so http3 support is missing.

I'll take a look.

junhochoi commented 4 years ago

You can try #320 to see if it fixes the issue, using the following patch to curl.rb:

diff --git a/curl.rb b/curl.rb
index 04561a9..8abba98 100644
--- a/curl.rb
+++ b/curl.rb
@@ -39,7 +39,7 @@ class Curl < Formula
     system "./buildconf" if build.head?

     # build boringssl
-    system "git", "clone", "--recursive", "https://github.com/cloudflare/quiche"
+    system "git", "clone", "-b", "boringssl_update", "--recursive", "https://github.com/cloudflare/quiche"
     mkdir_p "quiche/deps/boringssl/build"
     cd "quiche/deps/boringssl/build" do
       system "cmake", "-DCMAKE_POSITION_INDEPENDENT_CODE=on", ".."
tonyclemmey commented 4 years ago

@junhochoi Many thanks, working for me now.

tonyclemmey commented 4 years ago

@junhochoi Would you happen to know how I can keep the http3 equipped version of curl in Homebrew and stop the Homebrew update/upgrade replacing curl. I believe I need to either "brew tap" the http3 equipped version of curl formulae? and/or "brew pin" the package and it's dependencies somehow? Does this sound correct and possible?

Currently, brew update/upgrade is causing my curl to be upgraded from "https://github.com/curl/curl.git" which I then have to uninstall and replace with CloudFlare Quic version.

Already up-to-date.
==> Upgrading 1 outdated package:
curl HEAD-be578ee -> HEAD-be578ee
==> Upgrading curl
==> Cloning https://github.com/curl/curl.git
Updating /Users/tony/Library/Caches/Homebrew/curl--git
==> Checking out branch master
Already on 'master'
Your branch is up to date with 'origin/master'.
HEAD is now at be578ee CMake: use check_symbol_exists also for inet_pton
==> ./buildconf
==> ./configure --disable-silent-rules --prefix=/usr/local/Cellar/curl/HEAD-be578ee --with-secure-transport --without-ca-bundle --without-ca-path
==> make install
==> make install -C scripts
Error: File exists @ dir_s_mkdir - /usr/local/opt/curl/.brew
Removing: /Users/tony/Library/Caches/Homebrew/cargo_cache... (787 files, 72.3MB)
Removing: /Users/tony/Library/Caches/Homebrew/go_cache... (15 files, 90.5KB)
tonyclemmey commented 4 years ago

@junhochoi I currently run into this

$ curl -I https://google.com/ --http3
curl: (7) quiche_conn_recv() == -3
junhochoi commented 4 years ago

@tonyclemmey

binarytrails commented 3 years ago
  HTTP3:            enabled (quiche)
  ECH:              no      (--enable-ech)
  Protocols:        DICT FILE FTP FTPS GOPHER HTTP HTTPS IMAP IMAPS LDAP LDAPS MQTT POP3 POP3S RTMP RTSP SMB SMBS SMTP SMTPS TELNET TFTP
  Features:         AsynchDNS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 NTLM NTLM_WB PSL SSL UnixSockets alt-svc brotli libz zstd

  WARNING:  HTTP3 enabled but marked EXPERIMENTAL. Use with caution!

but still:

$ ./src/.libs/curl -I https://cloudflare-quic.com --http3
curl: option --http3: the installed libcurl version doesn't support this
curl: try 'curl --help' or 'curl --manual' for more information```
$ ./src/.libs/curl  --help all | grep http3
     --http3         Use HTTP v3
$ ./src/.libs/curl  --version
curl 7.74.0-DEV (x86_64-pc-linux-gnu) libcurl/7.73.0 OpenSSL/1.1.1h zlib/1.2.11 zstd/1.4.5 libidn2/2.3.0 libpsl/0.21.1 (+libidn2/2.3.0) libssh2/1.9.0 nghttp2/1.41.0
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher http https imap imaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd

I followed https://github.com/curl/curl/blob/master/docs/HTTP3.md

jdunk commented 3 years ago

@binarytrails I have exactly the same issue. I followed the openssl installation route. Were you ever able to resolve this?

binarytrails commented 3 years ago

@jdunk No, I ended up using https://github.com/aiortc/aioquic/blob/main/examples/http3_client.py for the sake of time. Let me know if you find a solution! From what I understood/remember about curl in http3 all curl libraries needs to be aligned and I didn't want to break my curl by replacing them all or otherwise, pointing to the right ones which can get tedious.

jdunk commented 3 years ago

Thanks @binarytrails. In my case, my version of libcurl as reported by curl --version was not the right one (much older than the curl version), and while some people reported having to modify the ld load order in /etc/ld.so.conf, in my case I simply needed to sudo ldconfig -v and then curl --version reported the correct (matching) libcurl version and the --http3 option also then worked. curl's official guide definitely needs to add that step.

shinmai commented 1 year ago

curl's official guide definitely needs to add that step.

18 months later and the guide remains the same, but this fixed my issue, thanks!

GF-Huang commented 1 year ago

Any one help me?

$ curl --version
curl 7.74.0 (x86_64-pc-linux-gnu) libcurl/7.74.0 OpenSSL/1.1.1n zlib/1.2.11 brotli/1.0.9 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) libssh2/1.9.0 nghttp2/1.43.0 librtmp/2.3
Release-Date: 2020-12-09
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets
$ curl --http3 https://cloudflare-quic.com/
curl: option --http3: the installed libcurl version doesn't support this
curl: try 'curl --help' or 'curl --manual' for more information
luisalvarado commented 1 year ago

Just updating here:

luis@xtreme:~$ curl --version
curl 7.85.0 (x86_64-pc-linux-gnu) libcurl/7.85.0 OpenSSL/3.0.5 zlib/1.2.11 brotli/1.0.9 zstd/1.5.2 libidn2/2.3.3 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.49.0 librtmp/2.3
Release-Date: 2022-08-31
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd
luis@xtreme:~$ curl --http3
curl: option --http3: the installed libcurl version doesn't support this
curl: try 'curl --help' or 'curl --manual' for more information

And libcurl version is:

image

Do note that on the latest version (7.88 as of right now) they have --http3-only as a parameter in case --http3 needs a fallback way. More info at https://github.com/curl/curl/pull/10264