openwrt / packages

Community maintained packages for OpenWrt. Documentation for submitting pull requests is in CONTRIBUTING.md
GNU General Public License v2.0
4.04k stars 3.5k forks source link

curl add http/3 support by default #19382

Open user8446 opened 2 years ago

user8446 commented 2 years ago

Maintainer: @stangri

Hi,

cURL OpenWrt package currently does not support HTTP/3 support but should. This protocol is now widely used everywhere, is supported in all major browsers, and has many advantages over HTTPS/2 including speed and privacy.

For example https-dns-proxy can now use HTTP/3 for DNS resolvers that use it. DNS is extremely latency sensitive in which a faster protocol would benefit it and be noticeable on pageload and responsiveness on OpenWrt connected devices.

abelian424 commented 11 months ago

@stangri I've been trying to build other TLS libraries to look for something slimmer than OpenSSL - I saw that 'checking size of size_t' complete successfully when I was building cmake. What is your build environment? I've been using Alpine because OpenWrt on GCC 13 doesn't work in Debian-unstable. Here's the Dockerfile:

FROM alpine:latest

ENV GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01'
WORKDIR /workdir

RUN apk add \
    'argp-standalone' \
    'asciidoc' \
    'bash' \
    'bc' \
    'binutils' \
    'bzip2' \
    'cdrkit' \
    'coreutils' \
    'diffutils' \
    'elfutils-dev' \
    'findutils' \
    'flex' \
    'g++' \
    'gawk' \
    'gettext' \
    'git' \
    'grep' \
    'gzip' \
    'intltool' \
    'libxslt' \
    'linux-headers' \
    'musl-fts-dev' \
    'musl-libintl' \
    'musl-obstack-dev' \
    'ncurses-dev' \
    'openssl-dev' \
    'patch' \
    'perl' \
    'python3-dev' \
    'rsync' \
    'unzip' \
    'util-linux' \
    'zlib-dev' \
    'curl' 'build-base' 'wget' 'gnupg' 'perl-utils' 'tar' \
    'nano' 'expat' 'zsh' 'zsh-theme-powerlevel10k'

RUN curl -LO https://raw.githubusercontent.com/miyagawa/cpanminus/master/cpanm \
    && chmod +x cpanm \
    && ./cpanm App::cpanminus \
    && rm -fr ./cpanm /root/.cpanm

ENV PERL_CPANM_OPT --verbose --mirror https://cpan.metacpan.org --mirror-only
RUN cpanm Digest::SHA Module::Signature && rm -rf ~/.cpanm
ENV PERL_CPANM_OPT $PERL_CPANM_OPT --verify

RUN ln -s '/usr/lib/libncurses.so' '/usr/lib/libtinfo.so' && \
  addgroup 'buildbot' && \
  adduser -s '/bin/bash' -G 'buildbot' -D 'buildbot'

USER buildbot

#optional
#RUN mkdir -p ~/.local/share/zsh/plugins && \
#  ln -s /usr/share/zsh/plugins/powerlevel10k ~/.local/share/zsh/plugins/ && \
#  echo "alias make='make -j$(nproc)' && alias cp='cp -i'" >> ~/.zshrc \
# && ln -s /usr/lib/perl5/core_perl/ /workdir/{openwrt_dir}/staging_dir/hostpkg/usr/lib/perl5/{perl_ver}/

#ENV PERL5LIB='/workdir/{openwrt_dir}/staging_dir/hostpkg/usr/lib/perl5/{perl_ver}/'

You should symlink /usr/lib/perl5/core_perl/* to /workdir/{openwrt_dir}/staging_dir/hostpkg/usr/lib/perl5/{perl_ver}/ to bypass having to install perl inside the buildroot. If that doesn't work, prefixing PERL5LIB=/workdir/{openwrt_dir}/staging_dir/hostpkg/usr/lib/perl5/{perl_ver}/before make will do the trick (you might have to symlink into PERL5LIB directory any missing files).

Mind, I've gotten stuck on building ngtcp2 but that error is due to trying to build with libressl not being recognized as an openssl/quictls replacement. libcurl shows an error with ngtcp2 not being accepted due to lack of quic support. If you want you can wait until I verify that libcurl successully builds - I just need to figure out how to tell ngtcp2 to locate libressl in staging_dir/host/lib.

EDIT: You should hold off on trying to build with OpenSSL 3.2.

EDIT: Using quictls is still the best turnkey option for now. Libressl should be possible in the next few days or weeks with the release of version 1.2.0 of ngtcp2. Rustls+MbedTLS might be a good combo (Rustls doesn't allow self-signed certificates for even local networks). And h2o looks very interesting. I'll probably try to fit that in or WolfSSL if it proves too difficult.

stangri commented 11 months ago
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:    22.04
Codename:   jammy

I don't think the error I'm getting is perl-related. I've added/enabled all the perl modules in build and it still doesn't help.

abelian424 commented 11 months ago
$ lsb_release -a
No LSB modules are available.
Distributor ID:   Ubuntu
Description:  Ubuntu 22.04.3 LTS
Release:  22.04
Codename: jammy

I don't think the error I'm getting is perl-related. I've added/enabled all the perl modules in build and it still doesn't help.

Sorry, it isn't perl-related. Do you have gcc-multilib installed? I've successfully built an image with quictls 3.1.2 with ngtcp2 and nghttp3, and the only issue was building perl which I fixed by copying missing files into staging_dir from the host and prefacing make with PERL5LIB=/workdir/openwrt/staging_dir/hostpkg/usr/lib/perl5/5.38.2.

stangri commented 11 months ago

Do you have gcc-multilib installed?

I didn't, but I have installed it now and I'm still getting the checking size of size_t... configure: error: Failed to find size of size_t.

Any other suggestions?

abelian424 commented 11 months ago

@stangri This is stock Ubuntu 22.04.3 - which is based on Debian Bookworm? I didn't have any libc-related problems (which this is) with Debian-stable, but even so, why don't you try the dockerfile I posted? Without determining a clean chroot it's hard to say what package would fix this.

EDIT: I've made a repo so that you can easily look through the files I used.

stangri commented 11 months ago

EDIT: I've made a repo so that you can easily look through the files I used.

I've installed apline 3.19.0 in a VM and added all the packages from your docker file, however when I'm trying to build the toolchain (or curl), it fails with:

/home/stangri/SDK_x86-64_23.05.2/scripts/ipkg-build: line 150: 10817 Segmentation fault      mkdir "$tmp_dir"
abelian424 commented 11 months ago

@stangri That seems like it's crashing due to not having file/directory permissions.

EDIT: I just realized you might not have known about the wiki page for Docker image creation. Did you follow the other steps such as adding a .dockerignore file, etc.?

stangri commented 11 months ago

@abelian424 I've been using SDK before, but when I switched to building from source I was successful rebuilding all the necessary libraries and openssl-quictls-based libcurl in Alpine VM. Weirdly, the curl 8.5.0 building still fails, but I have install the binary from the OpenWrt repo, I just need the updated libraries.

abelian424 commented 11 months ago

@stangri Use these commands at the root of the openwrt directory:

make package/libs/openssl/compile && \
mkdir -p staging_dir/hostpkg/usr/lib/perl5/5.38.2 && \
cp -ruf /usr/lib/perl5/core_perl/* staging_dir/hostpkg/usr/lib/perl5/5.38.2/ && \
PERL5LIB=/workdir/openwrt/staging_dir/hostpkg/usr/lib/perl5/5.38.2/ make package/feeds/packages/perl/compile && \
make package/feeds/packages/nghttp2/compile && \
make package/feeds/packages/nghttp3/compile && \
make package/feeds/packages/ngtcp2/compile && \
make package/feeds/packages/curl/compile
stangri commented 11 months ago

@abelian424 thanks for the continuing feedback, after some experiments I came up with this README with the Alpine Linux VM instructions which work for me. My plan is to add the README to the OpenWrt curl package.

sbwml commented 11 months ago

In fact, you only need to change ngtcp2 and nghttp3 from cmake to automake to solve the problem. If you use cmake just to unify the style of Makefile, you don't have to. After all, his path and header files have affected the normal build.

In the existing openwrt v23.05.2, we usually only need two steps to implement the quic support of curl and nginx:

  1. ngtcp2 & nghttp3 switch to automake.
  2. add the quic-tls patch file in the openssl patches directory.

发自我的iPhone

在 2023年12月29日,04:35,Stan Grishin @.***> 写道:

 @abelian424 thanks for the continuing feedback, after some experiments I came up with this README with the Alpine Linux VM instructions which work for me. My plan is to add the README to the OpenWrt curl package.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

stangri commented 11 months ago

In fact, you only need to change ngtcp2 and nghttp3 from cmake to automake to solve the problem.

I don't have a problem building with cmake.

abelian424 commented 11 months ago

Hey @stangri I was attempting to build using the instructions on melmac.net but I'm stuck here:

/workdir/openwrt/staging_dir/toolchain-arm_cortex-a9_gcc-13.1.0_musl_eabi/lib/gcc/arm-openwrt-linux-muslgnueabi/13.1.0/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: /workdir/openwrt/staging_dir/host/lib/libssl.a: error adding symbols: file format not recognized
collect2: error: ld returned 1 exit status
make[5]: *** [Makefile:1696: libcurl.la] Error 1

I know that the build system has recently been having issues locating libraries, but is this problem related to those issues?

stangri commented 11 months ago

I've been meaning to amend the docs. To me it works if you build the world/toolchain only replacing the openssl lib with quictls and then updating other libs/packages/editing menuconfig and building curl package only.

abelian424 commented 11 months ago

No worries, the problem is related to the one reported here. If I remove the need for the fix reported there by changing out the downloaded file format for curl away from bzip2 format, I get this error:

/workdir/openwrt/staging_dir/toolchain-arm_cortex-a9_gcc-13.1.0_musl_eabi/lib/gcc/arm-openwrt-linux-muslgnueabi/13.1.0/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `ngtcp2_crypto_recv_retry_cb'
/workdir/openwrt/staging_dir/toolchain-arm_cortex-a9_gcc-13.1.0_musl_eabi/lib/gcc/arm-openwrt-linux-muslgnueabi/13.1.0/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `ngtcp2_crypto_recv_crypto_data_cb'
/workdir/openwrt/staging_dir/toolchain-arm_cortex-a9_gcc-13.1.0_musl_eabi/lib/gcc/arm-openwrt-linux-muslgnueabi/13.1.0/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `ngtcp2_crypto_decrypt_cb'
/workdir/openwrt/staging_dir/toolchain-arm_cortex-a9_gcc-13.1.0_musl_eabi/lib/gcc/arm-openwrt-linux-muslgnueabi/13.1.0/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `ngtcp2_crypto_update_key_cb'
/workdir/openwrt/staging_dir/toolchain-arm_cortex-a9_gcc-13.1.0_musl_eabi/lib/gcc/arm-openwrt-linux-muslgnueabi/13.1.0/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `ngtcp2_crypto_get_path_challenge_data_cb'
/workdir/openwrt/staging_dir/toolchain-arm_cortex-a9_gcc-13.1.0_musl_eabi/lib/gcc/arm-openwrt-linux-muslgnueabi/13.1.0/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `ngtcp2_crypto_encrypt_cb'
/workdir/openwrt/staging_dir/toolchain-arm_cortex-a9_gcc-13.1.0_musl_eabi/lib/gcc/arm-openwrt-linux-muslgnueabi/13.1.0/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `ngtcp2_crypto_delete_crypto_cipher_ctx_cb'
/workdir/openwrt/staging_dir/toolchain-arm_cortex-a9_gcc-13.1.0_musl_eabi/lib/gcc/arm-openwrt-linux-muslgnueabi/13.1.0/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `ngtcp2_crypto_client_initial_cb'
/workdir/openwrt/staging_dir/toolchain-arm_cortex-a9_gcc-13.1.0_musl_eabi/lib/gcc/arm-openwrt-linux-muslgnueabi/13.1.0/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `ngtcp2_crypto_quictls_configure_client_context'
/workdir/openwrt/staging_dir/toolchain-arm_cortex-a9_gcc-13.1.0_musl_eabi/lib/gcc/arm-openwrt-linux-muslgnueabi/13.1.0/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `ngtcp2_crypto_delete_crypto_aead_ctx_cb'
/workdir/openwrt/staging_dir/toolchain-arm_cortex-a9_gcc-13.1.0_musl_eabi/lib/gcc/arm-openwrt-linux-muslgnueabi/13.1.0/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `ngtcp2_crypto_hp_mask_cb'
collect2: error: ld returned 1 exit status
make[5]: *** [Makefile:1043: curl] Error 1

So I guess I'll just wait for a fix from upstream.

UPDATE: No fix yet, so I just skipped the ./scripts/feeds update -a step and only use ./scripts/feeds install -a

ryuuzake commented 8 months ago

UPDATE: No fix yet, so I just skipped the ./scripts/feeds update -a step and only use ./scripts/feeds install -a

Hey @abelian424, can you explain more about what you mean by skipping the feeds update? because after I tried the guide on melmac.net, I get the same error as you

abelian424 commented 8 months ago

It would only work if you have used ".scripts/feeds update -a" in the past before this issue popped up. It's far from a universal fix. What I suggest though is not adding more than just the packages needed - quictls, nghttp3, ngtcp2, and libcurl. You only need the built packages. You can use the image builder for the actual firmware image. BTW, could you state which version of quictls you are trying to build? Libcurl 8.6.0 does not build with 3.1 for me

On Tue, Mar 19, 2024, 10:57 PM Riza Nafis @.***> wrote:

UPDATE: No fix yet, so I just skipped the ./scripts/feeds update -a step and only use ./scripts/feeds install -a

Hey @abelian424 https://github.com/abelian424, can you explain more about what you mean by skipping the feeds update? because after I tried the guide on melmac.net, I get the same error as you

— Reply to this email directly, view it on GitHub https://github.com/openwrt/packages/issues/19382#issuecomment-2007684678, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJXIOPTREGO72YTVTJRPJQ3YZBU53AVCNFSM6AAAAAAQN6SEC2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBXGY4DINRXHA . You are receiving this because you were mentioned.Message ID: @.***>

ryuuzake commented 8 months ago

Wow, thank you for the swift reply. I didn't expect to be answered so soon

BTW, could you state which version of quictls you are trying to build? Libcurl 8.6.0 does not build with 3.1 for me

I don't know how to check that. But the openssl I'm trying to build is 3.0.12

PKG_NAME:=openssl

PKG_VERSION:=3.0.12
PKG_RELEASE:=quic1
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)-$(PKG_NAME)-$(PKG_VERSION)-$(PKG_RELEASE)
PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION)-$(PKG_RELEASE).tar.gz
PKG_SOURCE_URL:=https://codeload.github.com/quictls/openssl/tar.gz/$(PKG_NAME)-$(PKG_VERSION)-$(PKG_RELEASE)?
PKG_HASH:=ca0e148f5e50c38198a25559d52cc06781345f5bff15de526ff65b0cc623d1bc

For the libcurl/curl package I'm trying to build is 8.5.0

PKG_NAME:=curl
PKG_VERSION:=8.5.0
PKG_RELEASE:=1

PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.bz2
PKG_SOURCE_URL:=https://github.com/curl/curl/releases/download/curl-$(subst .,_,$(PKG_VERSION))/ \
        https://dl.uxnr.de/mirror/curl/ \
        https://curl.askapache.com/download/ \
        https://curl.se/download/
PKG_HASH:=ce4b6a6655431147624aaf582632a36fe1ade262d5fab385c60f78942dd8d87b

Both for the version I get is from the guide on melmac.net

I guess I'll try only updating the required packages and post an update of it

abelian424 commented 8 months ago

8.5.0 builds with openssl/quictls 3.0.12 and 3.1.4. I think your missing the LD_LIBRARY_PATH environment variable. Instructions are on my github repo: github.com/abelian424/openwrt-quictls

On Wed, Mar 20, 2024, 8:04 PM Riza Nafis @.***> wrote:

Wow, thank you for the swift reply. I didn't expect to be answered so soon

BTW, could you state which version of quictls you are trying to build? Libcurl 8.6.0 does not build with 3.1 for me

I don't know how to check that. But the openssl I'm trying to build is 3.0.12

PKG_NAME:=openssl

PKG_VERSION:=3.0.12 PKG_RELEASE:=quic1 PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)-$(PKG_NAME)-$(PKG_VERSION)-$(PKG_RELEASE) PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION)-$(PKG_RELEASE).tar.gz PKG_SOURCE_URL:=https://codeload.github.com/quictls/openssl/tar.gz/$(PKG_NAME)-$(PKG_VERSION)-$(PKG_RELEASE)? PKG_HASH:=ca0e148f5e50c38198a25559d52cc06781345f5bff15de526ff65b0cc623d1bc https://codeload.github.com/quictls/openssl/tar.gz/$(PKG_NAME)-$(PKG_VERSION)-$(PKG_RELEASE)?PKG_HASH:=ca0e148f5e50c38198a25559d52cc06781345f5bff15de526ff65b0cc623d1bc

For the libcurl/curl package I'm trying to build is 8.5.0

PKG_NAME:=curl PKG_VERSION:=8.5.0 PKG_RELEASE:=1

PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.bz2 PKG_SOURCEURL:=https://github.com/curl/curl/releases/download/curl-$(subst .,,$(PKG_VERSION))/ \ https://dl.uxnr.de/mirror/curl/ \ https://curl.askapache.com/download/ \ https://curl.se/download/ PKG_HASH:=ce4b6a6655431147624aaf582632a36fe1ade262d5fab385c60f78942dd8d87b https://curl.se/download/PKG_HASH:=ce4b6a6655431147624aaf582632a36fe1ade262d5fab385c60f78942dd8d87b

Both for the version I get is from the guide on melmac.net

I guess I'll try only updating the required packages and post an update of it

— Reply to this email directly, view it on GitHub https://github.com/openwrt/packages/issues/19382#issuecomment-2009652755, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJXIOPWVH62U6W332TBAOSDYZGJOHAVCNFSM6AAAAAAQN6SEC2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBZGY2TENZVGU . You are receiving this because you were mentioned.Message ID: @.***>

ryuuzake commented 8 months ago

Thank you @abelian424 , following your repo works.

If we want to make the melmac.net guide works for general. we needs to pin down the git commit were your ./script/feeds update -a still works and then pin it with the feeds.conf

sbwml commented 8 months ago

I may not fully understand why you have been facing various issues during the build process, but there's no need to worry. I have taken the initiative to create a build curl workflow for you.

Now anyone can build the libcurl library with support for the HTTP/3 protocol.

follow me 👉 https://github.com/sbwml/openwrt-curl-prebuilt

ipks 👉 https://github.com/sbwml/openwrt-curl-prebuilt/releases

stangri commented 8 months ago

@sbwml thank you, beautiful workflow!

liudongmiao commented 8 months ago

@sbwml Could you use latest curl 8.7.1+ with openssl 3.2 for http/3?

sbwml commented 8 months ago

@sbwml Could you use latest curl 8.7.1+ with openssl 3.2 for http/3?

Unfortunately, it is regrettable that OpenSSL 3.2's QUIC is not compatible with ngtcp2. The currently available versions that can be used are OpenSSL 1.1, 3.0, and 3.1.

liudongmiao commented 8 months ago

@sbwml Could you use latest curl 8.7.1+ with openssl 3.2 for http/3?

Unfortunately, it is regrettable that OpenSSL 3.2's QUIC is not compatible with ngtcp2. The currently available versions that can be used are OpenSSL 1.1, 3.0, and 3.1.

If curl 8.6.0+ use OpenSSL 3.2.0+, then forget ngtcp2: https://curl.se/docs/http3.html.

For curl 8.6.0, use configure or cmake with a patch, and for curl 8.7.1+, either configure or cmake would work. (I only test on macos.)

sbwml commented 8 months ago

If curl 8.6.0+ use OpenSSL 3.2.0+, then forget ngtcp2: https://curl.se/docs/http3.html.

For curl 8.6.0, use configure or cmake wirh a patch, and for curl 8.7.1+, either configure or cmake would work. (I only test on macos.)

Are you sure you are using openssl3.2 on macos and successfully made curl use http/3?

This is incredible. I want to see your information:

curl --version

# test
curl -I --http3-only https://cloudflare-quic.com
liudongmiao commented 8 months ago

@sbwml Yes, I'm very sure.

$ curl -q -s -v --http3-only https://cloudflare-quic.com -o /dev/null
* Host cloudflare-quic.com:443 was resolved.
* IPv6: (none)
* IPv4: 104.22.8.38, 104.22.9.38, 172.67.9.235
*   Trying 104.22.8.38:443...
* QUIC cipher selection: TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_CCM_SHA256
*  CAfile: /etc/ssl/cert.pem
*  CApath: /etc/ssl/certs
*  subjectAltName: host "cloudflare-quic.com" matched cert's "cloudflare-quic.com"
* Verified certificate just fine
* Connected to cloudflare-quic.com (104.22.8.38) port 443
* using HTTP/3
* [HTTP/3] [0] OPENED stream for https://cloudflare-quic.com/
* [HTTP/3] [0] [:method: GET]
* [HTTP/3] [0] [:scheme: https]
* [HTTP/3] [0] [:authority: cloudflare-quic.com]
* [HTTP/3] [0] [:path: /]
* [HTTP/3] [0] [user-agent: curl/8.7.1]
* [HTTP/3] [0] [accept: */*]
> GET / HTTP/3
> Host: cloudflare-quic.com
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/3 200 
< date: Tue, 02 Apr 2024 01:34:43 GMT
< content-type: text/html
< content-length: 125959
< server: cloudflare
< cf-ray: 86dd1401fba2094f-IAD
< alt-svc: h3=":443"; ma=86400
< 
{ [10994 bytes data]
* Connection #0 to host cloudflare-quic.com left intact
liudongmiao commented 8 months ago

@sbwml Sorry for missing curl version in previous comment.

$ curl --version
curl 8.7.1 (Darwin) libcurl/8.7.1 OpenSSL/3.2.1 zlib/1.2.11 nghttp2/1.60.0 nghttp3/1.2.0
Release-Date: 2024-03-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile libz NTLM SSL threadsafe TLS-SRP UnixSockets
liudongmiao commented 8 months ago

@sbwml Can you help to test?

openssl: https://github.com/brevent/openwrt-base/commit/2669accf4052fea9277b2d1ad4af98723ae45661 (base on openwrt-23.05 branch)

curl: https://github.com/brevent/openwrt-packages/commit/ba38467cbc4dbe9689f07ef6c7e7f296a04f5be3 (base on openwrt-23.05 branch)

For curl, please turn off LIBCURL_NGTCP2 and turn on LIBCURL_OPENSSL_QUIC.

sbwml commented 8 months ago

I also updated the repository:

https://github.com/sbwml/feeds_packages_net_curl/commits/main/ https://github.com/sbwml/package_libs_openssl/tree/openssl-3.2

But it won't go on because I saw:

image

sbwml commented 8 months ago

openssl 3.2 + curl enable --with-openssl-quic It does work, but it's experimental. And openssl 3.2 is not LTS either.

https://github.com/sbwml/feeds_packages_net_curl https://github.com/sbwml/package_libs_openssl/tree/openssl-3.2

[root@OpenWrt ~]# curl --version
curl 8.7.1 (x86_64-openwrt-linux-gnu) libcurl/8.7.1 OpenSSL/3.2.1 nghttp2/1.57.0 nghttp3/1.2.0
Release-Date: 2024-03-27
Protocols: file ftp ftps http https ipfs ipns mqtt
Features: alt-svc HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile SSL threadsafe UnixSockets
[root@OpenWrt ~]# 
[root@OpenWrt ~]# curl -I --http3-only https://tls.cooluc.com
HTTP/3 200 
server: nginx/1.25.4 (Red Hat Enterprise Linux release 9.3 (Plow))
date: Tue, 02 Apr 2024 02:39:07 GMT
content-type: text/html; charset=utf-8
content-length: 5633
last-modified: Fri, 16 Dec 2022 21:22:34 GMT
vary: accept-encoding
etag: "639ce19a-1601"
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
x-quic: h3
x-xss-protection: 1
x-xss-protection: mod=block
strict-transport-security: max-age=63072000
public-key-pins: pin-sha256="BbrVIhEYvvBL6FiyC7nzVKLLDU3GPYdqHWAfk0ev/80="; pin-sha256="amMeV6gb9QNx0Zf7FtJ19Wa/t2B7KpCF/1n2Js3UuSU="; max-age=31536000; includeSubDomains
accept-ranges: bytes
liudongmiao commented 8 months ago

@sbwml Let's think about this.

For ngtcp2, does it require a quic patch for openssl? In this page, it seems https://curl.se/docs/http3.html requires.

If curl + ngtcp2 require quicktls (openssl with quic patch), then continue.

quicktls (openssl with quic patch) is acceptable, while openssl (openssl with quic) is unacceptable.

For the curl, I think curl would remove experimental message in short future.

For openssl, wheter quicktls (openssl with quic patch), or openssl with quic, is a problem. I think we may ask / request openssl to release a lts version for quic / http3 support.

sbwml commented 8 months ago

@sbwml Let's think about this.

For ngtcp2, does it require a quic patch for openssl? In this page, it seems https://curl.se/docs/http3.html requires.

If curl + ngtcp2 requir quicktls (openssl with quic patch), then continue.

quicktls (openssl with quic patch) is acceptable, while openssl (openssl with quic) is unacceptable.

For the curl, I think curl would remove experimental message in short future.

For openssl, wheter quicktls (openssl with quic patch), or openssl with quic, is a problem. I think we may ask / request openssl to release a lts version for quic / http3 support.

No, OpenWrt has never supported curl http/3. Even if you use the openwrt main branch you cannot build a curl that supports http/3. 😄 It may be very difficult for you to get openwrt official to do this.

To put it simply, packages/net/curl h3 exists as a bug. because openwrt is using openssl 3.0.

But for individuals, no matter what openwrt does, we can still achieve our needs in our own way.

sbwml commented 8 months ago

@sbwml Let's think about this.

For ngtcp2, does it require a quic patch for openssl? In this page, it seems https://curl.se/docs/http3.html requires.

If curl + ngtcp2 require quicktls (openssl with quic patch), then continue.

quicktls (openssl with quic patch) is acceptable, while openssl (openssl with quic) is unacceptable.

For the curl, I think curl would remove experimental message in short future.

For openssl, wheter quicktls (openssl with quic patch), or openssl with quic, is a problem. I think we may ask / request openssl to release a lts version for quic / http3 support.

As long as we can convince ourselves to leave the official package, we can achieve everything we want: https://github.com/sbwml/openwrt-curl-prebuilt/releases/tag/8.7.1-openssl-quic

micfogas commented 6 months ago

8.5.0 builds with openssl/quictls 3.0.12 and 3.1.4. I think your missing the LD_LIBRARY_PATH environment variable. Instructions are on my github repo: github.com/abelian424/openwrt-quictls

Wouldn't it be LIBRARY_PATH, not LD_LIBRARY_PATH? Though I typically use CPPFLAGS, LDFLAGS, and LIBS so that the includes and library names are provided. When using QuicTLS as a drop-in for openssl, you can modify the VERSION.dat in quictls root and change SHLIB_VERSION= from "81.3" to "3". They added the 81 (which is hex for Q) to avoid versioning conflicts, but when you're replacing openssl entirely with quictls, this may help avoid some issues with any libraries or programs throwing a flag with the unexpected version number.

abelian424 commented 6 months ago

I don't know if they're interchangeable, but whatever works. IIRC, modifying the newer quictls version.dat from 81.3 to just 3 does not seem to work. You can edit the patch file yourself to see why curl can't use it.

On Mon, May 27, 2024, 2:40 PM technotic @.***> wrote:

8.5.0 builds with openssl/quictls 3.0.12 and 3.1.4. I think your missing the LD_LIBRARY_PATH environment variable. Instructions are on my github repo: github.com/abelian424/openwrt-quictls

Wouldn't it be LIBRARY_PATH, not LD_LIBRARY_PATH? Though I typically use CPPFLAGS, LDFLAGS, and LIBS so that the includes and library names are provided. When using QuicTLS as a drop-in for openssl, you can modify the VERSION.dat in quictls root and change SHLIB_VERSION= from "81.3" to "3". They added the 81 (which is hex for Q) to avoid versioning conflicts, but when you're replacing openssl entirely with quictls, this may help avoid some issues with any libraries or programs throwing a flag with the unexpected version number.

— Reply to this email directly, view it on GitHub https://github.com/openwrt/packages/issues/19382#issuecomment-2132964637, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJXIOPV5L4UWFROPXQHJIR3ZELWPRAVCNFSM6AAAAAAQN6SEC2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZSHE3DINRTG4 . You are receiving this because you were mentioned.Message ID: @.***>

stangri commented 2 months ago

Please tag/reassign to current curl maintainer.