curl add http/3 support by default

[user8446]

Maintainer: @stangri

Hi,

cURL OpenWrt package currently does not support HTTP/3 support but should. This protocol is now widely used everywhere, is supported in all major browsers, and has many advantages over HTTPS/2 including speed and privacy.

For example https-dns-proxy can now use HTTP/3 for DNS resolvers that use it. DNS is extremely latency sensitive in which a faster protocol would benefit it and be noticeable on pageload and responsiveness on OpenWrt connected devices.

[stangri]

@user8446 Thanks for bringing it up.

Looking at support for HTTP/3 in curl it is experimental and requires:

1. patched OpenSSL library.
2. two more 3rd party libraries: libngtcp2 and libnghttp3.

Which brings the following concerns to mind:

1. OpenSSL is rather large and by default curl now depends on mbedtls (used to on WolfSSL, used by wpad too) due to the space constraints on most devices OpenWrt is available for.
2. Also space concerns over two extra libraries.

So with the current state I don't think it would be accepted as a default curl build option to include support for HTTP/3.

However if you or anyone else wants to contribute the PR to enable building curl with HTTP/3 support as an option for any of the existing SSL libraries in OpenWrt, I'd gladly test it and advocate for merging said PR.

[user8446]

Hi @stangri thanks for the reply. Not sure if it would better but there is also LibreSSL.

I don't have the knowledge to add HTTP/3 so the best I could do is donate to fund adding it.

[sbwml]

quictls + libngtcp2 + libnghttp3, it works well

[user8446]

@sbwml That is great! How did you build this? Was hoping openssl3 in master has quic but it doesn't:

root@OpenWrt:~# openssl version
OpenSSL 3.0.10 1 Aug 2023 (Library: OpenSSL 3.0.10 1 Aug 2023)

[sbwml]

 -----------------------------------------------------
 OpenWrt 23.05.0-rc2, r23228-cd17d8df2a
 -----------------------------------------------------
[root@OpenWrt ~]# openssl version
OpenSSL 3.0.10+quic 1 Aug 2023 (Library: OpenSSL 3.0.10+quic 1 Aug 2023)
[root@OpenWrt ~]#
[root@OpenWrt ~]# openssl version -a
OpenSSL 3.0.10+quic 1 Aug 2023 (Library: OpenSSL 3.0.10+quic 1 Aug 2023)
built on: Mon Jun 26 11:20:39 2023 UTC
platform: linux-aarch64-openwrt
options:  bn(64,64)
compiler: aarch64-openwrt-linux-musl-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O3 -Wl,--gc-sections -pipe -march=armv8-a+crc+crypto -fno-caller-saves -fno-plt -fhonour-copts -ffunction-sections -fdata-sections -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -Ofast -DPIC -fPIC -pipe -march=armv8-a+crc+crypto -fno-caller-saves -fno-plt -fhonour-copts -ffunction-sections -fdata-sections -Wformat -Werror=format-security -fstack-protector -Ofast -fPIC -znow -zrelro -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -D_FORTIFY_SOURCE=1 -DPIC -DDEVRANDOM="\"/dev/hwrng\""
OPENSSLDIR: "/etc/ssl"
ENGINESDIR: "/usr/lib/engines-3"
MODULESDIR: "/usr/lib/ossl-modules"
Seeding source: os-specific
CPUINFO: OPENSSL_armcap=0x3d
[root@OpenWrt ~]#

[sbwml]

Because I built OpenWrt for myself, I don't need to follow the code specifications. I replaced openssl pkg with the quitls source.

[user8446]

Thank you for sharing!

[sbwml]

If you're willing to break the OpenWrt source code, you can do it.

# get openwrt source
git clone https://github.com/openwrt/openwrt --depth=1 -b v23.05.0-rc2
cd openwrt
./scripts/feeds update -a
./scripts/feeds install -a

# openssl -> quictls
rm -rf package/libs/openssl
git clone https://github.com/sbwml/package_libs_openssl package/libs/openssl

# nghttp3
git clone https://github.com/sbwml/package_libs_nghttp3 package/libs/nghttp3

# ngtcp2
git clone https://github.com/sbwml/package_libs_ngtcp2 package/libs/ngtcp2

# curl - http3/quic
rm -rf feeds/packages/net/curl
git clone https://github.com/sbwml/feeds_packages_net_curl feeds/packages/net/curl

Add this to .config

CONFIG_PACKAGE_curl=y
CONFIG_PACKAGE_libcurl=y
CONFIG_LIBCURL_NGHTTP3=y
CONFIG_LIBCURL_NGTCP2=y
CONFIG_LIBCURL_OPENSSL=y
# CONFIG_LIBCURL_GNUTLS is not set
# CONFIG_LIBCURL_MBEDTLS is not set
# CONFIG_LIBCURL_NOSSL is not set
# CONFIG_LIBCURL_WOLFSSL is not set

At this point, you can build a curl that supports h3/quic.

[user8446]

I haven't done this before but I will research. Thank you for the info!

[stangri]

If you're willing to break the OpenWrt source code, you can do it.

@sbwml Would you consider sending PRs for existing openssl and curl packages and the new libnghttp3 and libngtcp2 packages?

[sbwml]

If you're willing to break the OpenWrt source code, you can do it.

@sbwml Would you consider sending PRs for existing openssl and curl packages and the new libnghttp3 and libngtcp2 packages?

😛 It's too difficult for me, I don't know English and I'm not familiar with using git, these are the biggest obstacles

-- Translated from Youdao

[stangri]

@sbwml thanks for your prompt reply. Would you be OK with me submitting PRs crediting you for the code/testing?

[sbwml]

@sbwml thanks for your prompt reply. Would you be OK with me submitting PRs crediting you for the code/testing?

Sure 👍

[user8446]

Thank you @stangri for looking into it, http3 will definitely be a benefit with DNS being so sensitive to latency

[stangri]

Sorry, just noticed this uses a fork of OpenSSL, I'm pretty sure this will not be accepted to replace the official OpenSSL in OpenWrt. I'll try to reach to maintainer to see if this can be added as a variant.

[stangri]

@sbwml have you tested/made updates to work with 23.05.0-rc3 and curl 8.3.0?

[sbwml]

@sbwml have you tested/made updates to work with 23.05.0-rc3 and curl 8.3.0?

I am currently using OpenWrt-23.05-rc3 and curl/8.2.1.

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 23.05.0-rc3, r23389-5deed175a5
 -----------------------------------------------------
[root@OpenWrt ~]# curl --version
curl 8.2.1 (aarch64-openwrt-linux-gnu) libcurl/8.2.1 OpenSSL/3.0.10 nghttp2/1.51.0 ngtcp2/0.18.0 nghttp3/0.14.0
Release-Date: 2023-07-26
Protocols: file ftp ftps http https mqtt
Features: alt-svc HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile SSL threadsafe UnixSockets
[root@OpenWrt ~]#

[sbwml]

https://x86.cooluc.com/minimal/openwrt-23.05/v23.05.0-rc3/manifest.txt?preview

[stangri]

@sbwml following your instructions I get the error on building curl with ngtcp2 with 23.05.0-rc3. Any idea on how to overcome this?

checking for nghttp2_session_get_stream_local_window_size in -lnghttp2... yes
checking for nghttp2/nghttp2.h... yes
configure: Added /home/stangri/development/Builder/openwrt/staging_dir/target-x86_64_musl/usr/lib to CURL_LIBRARY_PATH
checking for libngtcp2 options with pkg-config... found
configure: -l is -lngtcp2
configure: -I is -I/home/stangri/development/Builder/openwrt/staging_dir/target-x86_64_musl/usr/include
configure: -L is -L/home/stangri/development/Builder/openwrt/staging_dir/target-x86_64_musl/usr/lib
checking for ngtcp2_conn_client_new_versioned in -lngtcp2... yes
checking for ngtcp2/ngtcp2.h... yes
configure: Added  to CURL_LIBRARY_PATH
checking for libngtcp2_crypto_quictls options with pkg-config... no
configure: error: --with-ngtcp2 was specified but could not find ngtcp2_crypto_quictls pkg-config file.
make[3]: *** [Makefile:192: /home/stangri/development/Builder/openwrt/build_dir/target-x86_64_musl/curl-8.2.1/.configured_73e8a40fec0d702ff96a3dc5ec6098da] Error 1
make[3]: Leaving directory '/home/stangri/development/Builder/openwrt/feeds/packages/net/curl'
time: package/feeds/packages/curl/compile#41.67#1.51#43.63
    ERROR: package/feeds/packages/curl failed to build.
make[2]: *** [package/Makefile:120: package/feeds/packages/curl/compile] Error 1
make[2]: Leaving directory '/home/stangri/development/Builder/openwrt'
make[1]: *** [package/Makefile:114: /home/stangri/development/Builder/openwrt/staging_dir/target-x86_64_musl/stamp/.package_compile] Error 2
make[1]: Leaving directory '/home/stangri/development/Builder/openwrt'
make: *** [/home/stangri/development/Builder/openwrt/include/toplevel.mk:232: world] Error 2

[sbwml]

configure: error: --with-ngtcp2 was specified but could not find ngtcp2_crypto_quictls pkg-config file.

ngtcp2 also needs to be based on quicssl header files and libraries. It cannot be built directly in the original openwrt source code because it requires https://github.com/quictls/openssl .

I was able to build it because of tampering with openwrt's openssl package. https://github.com/sbwml/package_libs_openssl

[sbwml]

There are also examples of using quickls from the ngtcp2 readme file. https://github.com/ngtcp2/ngtcp2#build-from-git

[stangri]

configure: error: --with-ngtcp2 was specified but could not find ngtcp2_crypto_quictls pkg-config file.

ngtcp2 also needs to be based on quicssl header files and libraries. It cannot be built directly in the original openwrt source code because it requires https://github.com/quictls/openssl .

I was able to build it because of tampering with openwrt's openssl package. https://github.com/sbwml/package_libs_openssl

Yes, I was using libs/curl from your repo as per your comment above: https://github.com/openwrt/packages/issues/19382#issuecomment-1685022455. I'm still getting an error when building. I wonder if there are steps you've left out in the instructions.

[sbwml]

Yes, I was using libs/curl from your repo as per your comment above: #19382 (comment). I'm still getting an error when building. I wonder if there are steps you've left out in the instructions.

Nothing was missing. from your error logs, it doesn't seem to use quictls when compiling ngtcp2, because it can't find libngtcp2_crypto_quictls.pc. I guess the source code of your build package is not cloned from https://github.com/openwrt/openwrt

It's more like using the OpenWrt sdk.

[sbwml]

If you use openwrt sdk to test compile it, you should replace openssl from feeds.

rm -rf feeds/base/package/libs/openssl
git clone https://github.com/sbwml/package_libs_openssl feeds/base/package/libs/openssl

[abelian424]

The only EDIT: patched curl and quictls are custom package needed now i.e.

# openssl -> quictls
rm -rf package/libs/openssl
git clone https://github.com/sbwml/package_libs_openssl package/libs/openssl

# curl - http3/quic
rm -rf feeds/packages/net/curl
git clone https://github.com/sbwml/feeds_packages_net_curl feeds/packages/net/curl

Unfortunately the default settings don't allow it to build. If you use git checkout dev and edit the Makefile you can successfully build openssl/quictls:

PKG_NAME:=openssl

PKG_VERSION:=3.0.12
PKG_RELEASE:=1
PKG_BUILD_FLAGS:=no-mips16 gc-sections no-lto

PKG_BUILD_PARALLEL:=1

PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
PKG_SOURCE_PROTO:=git
PKG_SOURCE_URL:=https://github.com/sbwml/openssl.git
PKG_SOURCE_VERSION:=9e001c32f3b0a22ce9968ef27850c30e8a8e3ee1
PKG_MIRROR_HASH:=f93c9e8edde5e9166119de31755fc87b4aa34863662f67ddfcba14d0b6b69b61

Leaving at default branch and changing out the quictls package source url to the official versions possibly works but I haven't tried. It's easy enough to get the sha256 hashes for each available release though.

[sbwml]

The ~only~ EDIT: patched curl and quictls are custom package needed now i.e.

I am currently patching the openssl package of openwrt to implement quic support. https://github.com/sbwml/r4s_build_script/tree/master/openwrt/patch/openssl/quic

Also I cannot directly use nghttp3 and ngtcp2 provided by openwrt feeds.

https://github.com/sbwml/r4s_build_script/blob/master/openwrt/scripts/00-prepare_base.sh#L206 https://github.com/sbwml/r4s_build_script/blob/master/openwrt/scripts/00-prepare_base.sh#L210

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 23.05.2, r23630-842932a63d
 -----------------------------------------------------
[root@OpenWrt ~]# 
[root@OpenWrt ~]# openssl version -a
OpenSSL 3.0.12+quic 24 Oct 2023 (Library: OpenSSL 3.0.12+quic 24 Oct 2023)
built on: Tue Nov 14 13:38:11 2023 UTC
platform: linux-aarch64-openwrt
options:  bn(64,64)
compiler: aarch64-openwrt-linux-musl-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O3 -Wl,--gc-sections -pipe -march=armv8-a+crc+crypto -fno-caller-saves -fno-plt -fhonour-copts -ffunction-sections -fdata-sections -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -Ofast -DPIC -fPIC -pipe -march=armv8-a+crc+crypto -fno-caller-saves -fno-plt -fhonour-copts -ffunction-sections -fdata-sections -Wformat -Werror=format-security -fstack-protector -Ofast -fPIC -znow -zrelro -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -D_FORTIFY_SOURCE=1 -DPIC -DDEVRANDOM="\"/dev/hwrng\""
OPENSSLDIR: "/etc/ssl"
ENGINESDIR: "/usr/lib/engines-3"
MODULESDIR: "/usr/lib/ossl-modules"
Seeding source: os-specific
CPUINFO: OPENSSL_armcap=0x3d
[root@OpenWrt ~]# 
[root@OpenWrt ~]# curl --version
curl 8.5.0 (aarch64-openwrt-linux-gnu) libcurl/8.5.0 quictls/3.0.12 nghttp2/1.57.0 ngtcp2/1.1.0 nghttp3/1.1.0
Release-Date: 2023-12-06
Protocols: file ftp ftps http https mqtt
Features: alt-svc HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile SSL threadsafe UnixSockets
[root@OpenWrt ~]# 
[root@OpenWrt ~]# curl -I --http3-only https://tls.cooluc.com
HTTP/3 200 
server: nginx/1.25.3 (Red Hat Enterprise Linux release 9.2 (Plow))
date: Sat, 16 Dec 2023 10:11:25 GMT
content-type: text/html; charset=utf-8
content-length: 5633
last-modified: Fri, 16 Dec 2022 21:22:34 GMT
vary: accept-encoding
etag: "639ce19a-1601"
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
x-quic: h3
x-xss-protection: 1
x-xss-protection: mod=block
strict-transport-security: max-age=63072000
public-key-pins: pin-sha256="BbrVIhEYvvBL6FiyC7nzVKLLDU3GPYdqHWAfk0ev/80="; pin-sha256="amMeV6gb9QNx0Zf7FtJ19Wa/t2B7KpCF/1n2Js3UuSU="; max-age=31536000; includeSubDomains
accept-ranges: bytes

[root@OpenWrt ~]#

[stangri]

Also I cannot directly use nghttp3 and ngtcp2 provided by openwrt feeds.

Why? Have you looked into patching the sources from openwrt feeds?

[sbwml]

Also I cannot directly use nghttp3 and ngtcp2 provided by openwrt feeds.

Why? Have you looked into patching the sources from openwrt feeds?

因为 cmake

[abelian424]

The ~only~ EDIT: patched curl and quictls are custom package needed now i.e.

I am currently patching the openssl package of openwrt to implement quic support. https://github.com/sbwml/r4s_build_script/tree/master/openwrt/patch/openssl/quic

Also I cannot directly use nghttp3 and ngtcp2 provided by openwrt feeds.

https://github.com/sbwml/r4s_build_script/blob/master/openwrt/scripts/00-prepare_base.sh#L206 https://github.com/sbwml/r4s_build_script/blob/master/openwrt/scripts/00-prepare_base.sh#L210

I find only superficial differences between curl, nghttp3, and ngtcp2:

https://github.com/sbwml/feeds_packages_net_curl/blob/main/Config.in https://github.com/openwrt/packages/blob/master/net/curl/Config.in

https://github.com/sbwml/package_libs_nghttp3/blob/main/Makefile https://github.com/openwrt/packages/blob/master/libs/nghttp3/Makefile

https://github.com/sbwml/package_libs_ngtcp2/blob/main/Makefile https://github.com/openwrt/packages/blob/master/libs/ngtcp2/Makefile

It is a case of OpenWrt catching up to your packages (and in the case of curl, outpacing by being 8.5.0 instead of 8.4.0)?

Also I cannot directly use nghttp3 and ngtcp2 provided by openwrt feeds.

Why? Have you looked into patching the sources from openwrt feeds?

因为 cmake

What is due to cmake?

I can just edit this file to download quictls:

PKG_BASE:=$(subst $(space),.,$(wordlist 1,2,$(subst .,$(space),$(PKG_VERSION))))
PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION)-quic1.tar.gz
PKG_SOURCE_URL:= \
        https://github.com/quictls/openssl/archive/refs/tags/

PKG_HASH:=ca0e148f5e50c38198a25559d52cc06781345f5bff15de526ff65b0cc623d1bc

and then use this patch by @sbwml to eliminate intentional versioning differences from the quictls team:

--- a/VERSION.dat
+++ b/VERSION.dat
@@ -4,4 +4,4 @@ PATCH=10
 PRE_RELEASE_TAG=
 RELEASE_DATE="1 Aug 2023"
-SHLIB_VERSION=81.3
+SHLIB_VERSION=3

I'm going to try to compile without this patch but with all these patches and see what happens.

[sbwml]

It is a case of OpenWrt catching up to your packages (and in the case of curl, outpacing by being 8.5.0 instead of 8.4.0)?

curl 的版本不是重点，我在GitHub Action 为自己构建 openwrt 固件时始终取自 openwrt/packages main 分支的 curl，它的版本是 8.5.0。

What is due to cmake?

在编译 curl 并启用 ngtcp2 库支持时，使用 cmake 编译 ngtcp2 & nghttp3，那么 curl 将会编译失败（我没有保留任何日志，因为已经过去三个月了，你可以自己试试），我第一次让 curl 支持 http3 时就是使用的 cmake 去编译 ngtcp2 & nghttp3 ，但是当时 curl 编译失败了，最后我使用默认的 automake 编译成功了。毕竟我的需求只是让 curl 实现 quic/http3 的支持，而不是使用什么方式去编译它，所以并没有进一步的去了解原因。

PS：来自手机的回复，没有任何方式将其翻译成英文

[sbwml]

What is due to cmake?

印象中好像是会让 curl 无法正确找到 ngtcp2 & nghttp3 的头文件（header file）

我确定文件被拷贝到 STAGING_DIR 中了，因为那时候我有定义 CMAKE_INSTALL:=1。

---

如果你把 openwrt feeds 中提供的 ngtcp2 & nghttp3 作为独立的库去构建，它是没有任何问题的，但是作为 curl 或其它软件的构建依赖，cmake 确实会让这个过程出现一点毛病

[sbwml]

I'm going to try to compile without this patch but with all these patches and see what happens.

这已经足够了！

--- a/VERSION.dat
+++ b/VERSION.dat
@@ -4,4 +4,4 @@ PATCH=10
 PRE_RELEASE_TAG=
 RELEASE_DATE="1 Aug 2023"
-SHLIB_VERSION=81.3
+SHLIB_VERSION=3

该补丁仅仅是改变了 shared library 的版本命名，因为 quictls 上游的仓库会将 SHLIB 版本更改为 81.3 （即 libssl.so.81.3），这样会导致 openwrt 其它依赖 openssl 库的程序出现问题，因为他们需要的是 libssl.so.3。

而 quic 这里的补丁并没有更改openssl 本身的版本信息，所以不需要它。

[abelian424]

What is due to cmake?

印象中好像是会让 curl 无法正确找到 ngtcp2 & nghttp3 的头文件（header file）

我确定文件被拷贝到 STAGING_DIR 中了，因为那时候我有定义 CMAKE_INSTALL:=1。

如果你把 openwrt feeds 中提供的 ngtcp2 & nghttp3 作为独立的库去构建，它是没有任何问题的，但是作为 curl 或其它软件的构建依赖，cmake 确实会让这个过程出现一点毛病

When compiling curl and enabling ngtcp2 library support, 
use cmake to compile ngtcp2 & nghttp3, then curl will fail to compile 
(I did not keep any logs, because it has been three months, you can try it yourself), 
the first time I let When curl supports http3, cmake is used to compile ngtcp2 & nghttp3, 
but curl compilation failed at that time. Finally, I used the default automake to compile successfully. 
After all, my requirement is just to enable curl to implement quic/http3 support, 
not how to compile it, so I don’t have any further understanding of the reason.

PS: Reply from mobile phone, no way to translate it into English

I see. I'm starting to suspect that the inclusion of quictls as a replacement for openssl in OpenWrt is the only thing really in the way. I've been unable to compile your version of libcurl BTW, probably due to the differences in Config.in - @stangri is possibly having the same issue.

[sbwml]

I can just edit this file to download quictls:

你不需要改变 openwrt 的 openssl ，只需要将这些补丁下载到 openssl 的patches 目录中，它就已经实现 quic 的支持 https://github.com/sbwml/r4s_build_script/blob/master/openwrt/scripts/00-prepare_base.sh#L151

[abelian424]

该补丁仅仅是改变了 shared library 的版本命名，因为 quictls 上游的仓库会将 SHLIB 版本更改为 81.3 （即 libssl.so.81.3），这样会导致 openwrt 其它依赖 openssl 库的程序出现问题，因为他们需要的是 libssl.so.3。

而 quic 这里的补丁并没有更改openssl 本身的版本信息，所以不需要它。

The shared library is only renamed because quictls changed the SHLIB version to 81.3 （即 libssl.so.81.3）, 
which will cause other openwrt programs that depend on openssl library to have problems because they require 是 libssl.so.3。

But couldn't I just symlink these libraries after a firmware update?

而 quic 这里的补丁并没有更改openssl 本身的版本信息，所以不需要它。

The patch here in quic does not change the version information of openssl itself, so it is not needed.

So these are cherry-picked commits from the quictls repo made by @tmshort one of the maintainers of quictls. Are you backporting commits of quictls versions > 3.0.12-quic1?

[sbwml]

而 quic 这里的补丁并没有更改 openssl 本身的版本信息，所以不需要它。

The patch here in quic does not change the version information of openssl itself, so it is not needed.

所以这些是 quictls 维护者之一 @tmshort 从 quictls 存储库中精心挑选的提交。您是否向后移植 quictls 版本的提交 > 3.0.12-quic1？

事实上它与 openssl-3.0.12-quic1 保持一致，只是移除了一些文档的说明（避免因为版本升级后文档的改变导致补丁失败）

[sbwml]

But couldn't I could just symlink these libraries after a firmware update?

它会影响你的一些依赖openssl 库的程序编译

[abelian424]

It's possible to build quictls using the official release tarballs:

PKG_NAME:=openssl
PKG_VERSION:=3.0.12-quic1
PKG_RELEASE:=1
PKG_BUILD_FLAGS:=no-mips16 gc-sections no-lto
PKG_BUILD_DIR:=$(BUILD_DIR_HOST)/openssl-openssl-$(PKG_VERSION)
PKG_BUILD_PARALLEL:=1

PKG_SOURCE:=openssl-$(PKG_VERSION).tar.gz
PKG_SOURCE_URL:=https://codeload.github.com/quictls/openssl/tar.gz/openssl-$(PKG_VERSION)?dummy=/
PKG_HASH:=ca0e148f5e50c38198a25559d52cc06781345f5bff15de526ff65b0cc623d1bc

Some caveats:

• Depending on openssl configuration the patches included in the patches directory may cause an error regarding devcrypto. A lot of the patches can be deleted, especially regarding ones about unused hardware encryption engines, but turning on NO_DEPRECATED option may lead to a compilation error. I edited 999-hack-version.patch so that I could use it with quicktls!=3.0.10 without having to edit it for every different version:

  --- a/VERSION.dat
  +++ b/VERSION.dat
  @@ -7,1 +7,1 @@
  -SHLIB_VERSION=81.3
  +SHLIB_VERSION=3

• curl took a lot of trial and error to compile. The built in toolchain doesn't have all the libraries to complete without error. Make sure gdb is turned on in menuconfig before trying so that the config.log is created; it has a lot of useful information about exactly which libraries are missing. I ended up running a Docker container and make only failed once for expat dependency that is not in the downloadable containerfile.

The takeaway here is that quictls is the only out-of-tree code. You can build curl with HTTP/3 support in order to select nghttp3 and ngtcp2 as dependencies.

[stangri]

@abelian424 thank you for posting this. Have you tried building it with quictls 3.1.4?

PS. Did you have to modify the OpenWrt's nghttp3 and ngtcp2 to disable cmake and use the regular make to complete the curl build?

[abelian424]

@stangri Yeah, I actually built with 3.1.4 but posted for 3.0.12 as that is the current version on OpenWrt. The sha256sum PKG_HASH for 3.1.4 is 4bf990243d6aa39b8befa0c399834415842912ef67f88bef98e74dc619469618. No, I did not modify nghttp3 or ngtcp2. It was actually libcurl that gave me issues but it was due to building for a 32-bit router on a 64-bit machine. The cmake issues reported were probably due to the particular version of glibc or musl on the machine. I switched to a Debian container and it built fine, no other modifications necessary apart from openssl.

[sbwml]

--- a/VERSION.dat
+++ b/VERSION.dat
@@ -7,1 +7,1 @@
-SHLIB_VERSION=81.3
+SHLIB_VERSION=3

How is this diff generated for a single line of content?

[abelian424]

@sbwml @@ -7,1 +7,1 @@ shows that line 7 and only line 7 is being patched and that the patch is also one line. In other words, the first pair of numbers is line number and number of lines being patched in a, and the second pair is the same for b.

[stangri]

• A lot of the patches can be deleted, especially regarding ones about unused hardware encryption engines, but turning on NO_DEPRECATED option may lead to a compilation error. I edited 999-hack-version.patch so that I could use it with quicktls!=3.0.10 without having to edit it for every different version:

Do you mind publishing the patches you kept on GitHub?

• The built in toolchain doesn't have all the libraries to complete without error. Make sure gdb is turned on in menuconfig before trying so that the config.log is created; it has a lot of useful information about exactly which libraries are missing.

Do you recall/can you compare configs to confirm which libraries solved the build problem for libcurl for you? I've successfully built HTTP/3 supporting curl before creating PRs/merging nghttp3/ngtcp2 libraries into the tree, but I've been unsuccessful since and would like to build it again to publish a recipe in the curl README.

[abelian424]

CONFIG_LIBCURL_COOKIES=y
CONFIG_LIBCURL_FILE=y
CONFIG_LIBCURL_FTP=y
CONFIG_LIBCURL_HTTP=y
CONFIG_LIBCURL_NGHTTP2=y
CONFIG_LIBCURL_NGHTTP3=y
CONFIG_LIBCURL_NGTCP2=y
CONFIG_LIBCURL_NO_SMB="!"
CONFIG_LIBCURL_OPENSSL=y
CONFIG_LIBCURL_PROXY=y
CONFIG_LIBCURL_CRYPTO_AUTH=y
CONFIG_LIBCURL_THREADED_RESOLVER=y
CONFIG_LIBCURL_TLS_SRP=y
CONFIG_LIBCURL_UNIX_SOCKETS=y
CONFIG_LIGHTTPD_CRYPTOLIB_NETTLE=y
CONFIG_LIGHTTPD_PCRE2=y
CONFIG_LIGHTTPD_SSL=y
CONFIG_LIGHTTPD_SSL_SELECT=m
CONFIG_OPENSSL_ENGINE=y
CONFIG_OPENSSL_SMALL_FOOTPRINT=y
CONFIG_OPENSSL_WITH_ASM=y
CONFIG_OPENSSL_WITH_CMS=y
CONFIG_OPENSSL_WITH_DEPRECATED=y
CONFIG_OPENSSL_WITH_DTLS=y
CONFIG_OPENSSL_WITH_ERROR_MESSAGES=y
CONFIG_OPENSSL_WITH_PSK=y
CONFIG_OPENSSL_WITH_SRP=y
CONFIG_OPENSSL_WITH_TLS13=y
CONFIG_PACKAGE_ath10k-board-qca988x=m
CONFIG_PACKAGE_block-mount=y
# CONFIG_PACKAGE_blockd is not set
CONFIG_PACKAGE_cgi-io=y
CONFIG_PACKAGE_curl=y
CONFIG_PACKAGE_libnghttp2=y
CONFIG_PACKAGE_libnghttp3=y
CONFIG_PACKAGE_libngtcp2=y
CONFIG_PACKAGE_libopenssl=y
CONFIG_PACKAGE_libopenssl-afalg_sync=y
CONFIG_PACKAGE_libopenssl-conf=y
CONFIG_OPENSSL_WITH_CHACHA_POLY1305=y
# CONFIG_OPENSSL_WITH_IDEA is not set
# CONFIG_OPENSSL_WITH_MDC2 is not set
# CONFIG_OPENSSL_WITH_SEED is not set
# CONFIG_OPENSSL_WITH_WHIRLPOOL is not set
CONFIG_PACKAGE_kmod-crypto-kpp=m

There is the diffconfig, but I would wager the failure of libcurl was more to do with the build environment. Using the Debian container linked with only expat as an addition allows it to build for me. However, I was unsuccessful with an Arch Linux environment, and Alpine Linux needs fixes/workarounds for its low support of perl.

As for patches, I only had to remove 100-Configure-afalg-support.patch but that's probably only because I use afalg_sync - using devcrypto might cause problems with other patches. In any case, I think the bare minimum of patches needed is 110-openwrt_targets.patch, 120-strip-cflags-from-binary.patch and 999-hack-version.patch.

[sbwml]

Have you tried building it with quictls 3.1.4?

Since it is separate from the version used by OpenWrt, why don't you just use openssl 3.2.0? This way you can support quic without making any changes to the source code. 😀

[abelian424]

Have you tried building it with quictls 3.1.4?

Since it is separate from the version used by OpenWrt, why don't you just use openssl 3.2.0? This way you can support quic without making any changes to the source code. 😀

That... is good to know. News to me that 3.2.0 has HTTP/3 support.

@stangri 3.2.0 openssl also builds fine. The only problem is that I pruned the dependency list too much. Some perl modules are needed to build libcurl- that is almost definitely the problem you were having. Perl doesn't have an obvious dependency system, but this is a known problem. I'm just going to install every perl package since not even every perlbase-* helps. I'm thinking these packages that get pulled sre dependencies of undeclared perl modules since a clean compile after these is successful:

 make[2] -C package/libs/zlib compile
 make[2] -C package/libs/libxml2 compile
 make[2] -C feeds/packages/libs/db47 compile
 make[2] -C feeds/packages/libs/gdbm

A somewhat shorter list of perl modules that will allow full build:

CONFIG_PACKAGE_perl-test-harness=y
CONFIG_PACKAGE_perl-www-curl=y
CONFIG_PACKAGE_perlbase-base=y
CONFIG_PACKAGE_perlbase-benchmark=y
CONFIG_PACKAGE_perlbase-bytes=y
CONFIG_PACKAGE_perlbase-class=y
CONFIG_PACKAGE_perlbase-config=y
CONFIG_PACKAGE_perlbase-cwd=y
CONFIG_PACKAGE_perlbase-db-file=y
CONFIG_PACKAGE_perlbase-errno=y
CONFIG_PACKAGE_perlbase-essential=y
CONFIG_PACKAGE_perlbase-fcntl=y
CONFIG_PACKAGE_perlbase-file=y
CONFIG_PACKAGE_perlbase-filehandle=y
CONFIG_PACKAGE_perlbase-gdbm-file=y
CONFIG_PACKAGE_perlbase-getopt=y
CONFIG_PACKAGE_perlbase-i18n=y
CONFIG_PACKAGE_perlbase-integer=y
CONFIG_PACKAGE_perlbase-io=y
CONFIG_PACKAGE_perlbase-list=y
CONFIG_PACKAGE_perlbase-locale=y
CONFIG_PACKAGE_perlbase-params=y
CONFIG_PACKAGE_perlbase-posix=y
CONFIG_PACKAGE_perlbase-re=y
CONFIG_PACKAGE_perlbase-scalar=y
CONFIG_PACKAGE_perlbase-selectsaver=y
CONFIG_PACKAGE_perlbase-selfloader=y
CONFIG_PACKAGE_perlbase-socket=y
CONFIG_PACKAGE_perlbase-symbol=y
CONFIG_PACKAGE_perlbase-tap=y
CONFIG_PACKAGE_perlbase-test=y
CONFIG_PACKAGE_perlbase-text=y
CONFIG_PACKAGE_perlbase-tie=y
CONFIG_PACKAGE_perlbase-unicore=y
CONFIG_PACKAGE_perlbase-utf8=y
CONFIG_PACKAGE_perlbase-xsloader=y

[stangri]

@sbwml thanks for the tip about OpenSSL 3.2.0, after switching to it from quictls library and with the in-tree ngtcp2 and nghttp3 I get the following when trying to build curl with QUIC support enabled and OpenSSL selected as the library:

...
checking size of size_t... configure: error: Failed to find size of size_t
make[2]: *** [Makefile:192: /SDK_x86-64_23.05.2/build_dir/target-x86_64_musl/curl-8.5.0/.configured_ee316af4a147e2242e111d0ebb731019] Error 1
make[2]: Leaving directory '/SDK_x86-64_23.05.2/feeds/packages/net/curl'
time: package/feeds/packages/curl/compile#45.61#3.60#48.82
    ERROR: package/feeds/packages/curl failed to build.
make[1]: *** [package/Makefile:128: package/feeds/packages/curl/compile] Error 1
make[1]: Leaving directory '/SDK_x86-64_23.05.2'
make: *** [/SDK_x86-64_23.05.2/include/toplevel.mk:225: package/curl/compile] Error 2

Any ideas @sbwml @abelian424 ?

[abelian424]

@stangri Oh, that's also a perl issue. Just install the whole perl language package and it will build. I need to find some macro to narrow down the exact dependencies needed, because I keep forgetting to clean files properly before rebuilding.

[stangri]

@stangri Oh, that's also a perl issue. Just install the whole perl language package and it will build. I need to find some macro to narrow down the exact dependencies needed, because I keep forgetting to clean files properly before rebuilding.

Thanks for your prompt reply, I'll try.

I remember one of the curl updates affected the perl-www-curl module, so that would be my primary suspect.

[stangri]

@stangri Oh, that's also a perl issue. Just install the whole perl language package and it will build. I need to find some macro to narrow down the exact dependencies needed, because I keep forgetting to clean files properly before rebuilding.

I've installed/enabled all perl packages that opkg lists:

    ./scripts/feeds install -dy perl perl-ack perl-authen-sasl perl-authen-sasl-xs perl-cgi perl-compress-bzip2 perl-dbi perl-device-serialport perl-device-usb perl-encode-locale perl-file-listing perl-file-next perl-file-rsyncp perl-file-sharedir-install perl-html-form perl-html-parser perl-html-tagset perl-html-tree perl-http-cookies perl-http-daemon perl-http-date perl-http-message perl-http-negotiate perl-http-server-simple perl-inline perl-inline-c perl-io-html perl-lockfile-simple perl-lwp-mediatypes perl-net-cidr-lite perl-net-dns perl-net-http perl-net-telnet perl-netaddr-ip 
    ./scripts/feeds install -dy perl-parse-recdescent perl-parse-yapp perl-sub-uplevel perl-test-harness perl-test-warn perl-text-csv_xs perl-try-tiny perl-uri perl-www perl-www-curl perl-www-mechanize perl-www-robotrules perl-xml-parser perlbase-anydbm-file perlbase-app perlbase-archive perlbase-arybase perlbase-attribute perlbase-attributes perlbase-autodie perlbase-autoloader perlbase-autosplit perlbase-autouse perlbase-b perlbase-base perlbase-benchmark perlbase-bigint perlbase-bignum perlbase-blib perlbase-bytes perlbase-charnames perlbase-class perlbase-compress perlbase-config perlbase-cpan perlbase-cwd perlbase-data perlbase-db perlbase-db-file perlbase-dbm-filter perlbase-devel perlbase-diagnostics perlbase-digest perlbase-dirhandle perlbase-dumpvalue perlbase-dumpvar perlbase-dynaloader perlbase-encode perlbase-encoding perlbase-english perlbase-env perlbase-errno perlbase-essential perlbase-experimental perlbase-extutils perlbase-fatal perlbase-fcntl perlbase-feature
    ./scripts/feeds install -dy perlbase-fields perlbase-file perlbase-filecache perlbase-filehandle perlbase-filetest perlbase-filter perlbase-findbin perlbase-gdbm-file perlbase-getopt perlbase-hash perlbase-http-tiny perlbase-i18n perlbase-if perlbase-integer perlbase-io perlbase-ipc perlbase-json-pp perlbase-less perlbase-list perlbase-locale perlbase-math perlbase-memoize perlbase-meta-notation perlbase-mime perlbase-module perlbase-mro perlbase-net perlbase-next perlbase-o perlbase-opcode perlbase-open perlbase-ops perlbase-ostype perlbase-params perlbase-perl5db perlbase-perlio perlbase-pod perlbase-posix perlbase-re perlbase-safe perlbase-scalar perlbase-sdbm-file perlbase-search perlbase-selectsaver perlbase-selfloader perlbase-sigtrap perlbase-socket perlbase-sort perlbase-storable perlbase-symbol perlbase-sys perlbase-tap perlbase-term perlbase-test perlbase-text perlbase-thread perlbase-threads perlbase-tie perlbase-time perlbase-unicode perlbase-unicore perlbase-universal perlbase-user perlbase-utf8 perlbase-version perlbase-xsloader 

And I still get the checking size of size_t... configure: error: Failed to find size of size_t error when building curl if I enable either HTTP/3 or QUIC in menuconfig. Without either, it builds fine even with openssl 3.2.0.

I've tried googling it and didn't find anything relevant to my case, I'm wondering if it may be the build environment.