ci: add Windows builds + tests

vszakats commented 10 months ago

Add automated Windows builds and tests.

Use curl-for-win with llvm + mingw64, and a minimal libcurl build with no external dependencies to build x64, ARM64 and x86 trurl.exe.

Boost build performance by not building the curl tool [EXPERIMENTAL]. Use a customized curl-for-win build with disabled TLS to further reduce footprint and build time.

Non-UNITY libcurl builds can make turl binaries about 120KB smaller, but they require 2x build times (4m vs. 2m), so opted not to use those here.

Also enable tests and fix issues along the way:

libcurl with IDN support cannot be used because trurl itself lacks UNICODE support and thus fails to accept non-ASCII strings via the command-line.
```
expected:
'https://xn--rksmrgs-5wao1o.se/\n'
got:
''
104: failed 'https://räksmörgås.se' -g '{puny:host}'
```
Ref: https://github.com/curl/trurl/actions/runs/6863796328/job/18664263891#step:3:4406
add test.py support for a runner like wine. Via --runner=<bin> option. This disables valgrind tests.
add test.py to override the default trurl binary to test. Via --trurl=<bin> option.
skip stderr tests when using a runner. (wine does trash stderr output)
fix to enable punycode2idn only when libcurl has IDN support.
delete line-ending spaces from test.json.
add --keep-port to 6 tests to avoid relying on libcurl builds with specific protocols enabled, such as HTTPS or FTP.
add a new test with default-port using http/80.
update 4 tests to use http/imap instead of https/imaps to make them work with no-TLS libcurl.
build libcurl with IMAP to make 'options' URL field extraction work in tests.

Fixes #109 Closes #249

vszakats commented 10 months ago

First successful build: https://github.com/curl/trurl/actions/runs/6814620035/job/18531898994#step:3:5308 Also for x86 and arm64.

vszakats commented 10 months ago

Second run (3m11s), with downloadable artifacts: https://github.com/curl/trurl/actions/runs/6814704059

bagder commented 10 months ago

Nice!

There is of course a lot of stuff in that downloadable, but we can potentially polish on that later.

vszakats commented 10 months ago

More details about these builds:

trurl binaries are built against libcurl DLL. IOW not standalone static builds. Building the latter is technically doable, but:
1. it's cumbersome to implement (like all static builds), so in curl-for-win I did not plan to do it. [UPDATE-3: I special cased it for -zero builds, which we use here. Still cumbersome, but less.]
2. it'd result in trurl binaries as large as the curl tool. Meaning 700KB–6MB+ depending on options. ~~This could probably be improved in curl, so that linking the functions necessary for trurl wouldn't pull in almost all curl code. This would be generally a good idea for all curl dependents but also a sisyphean work. [UPDATE-2: This isn't an issue with the trurl CI artifact, because curl tool isn't distributed and trurl + libcurl.dll is the same size as trurl static.]
UPDATE-1: This'd also need tweaking curl's unity build, to keep functions/sources required for trurl in a separate unity block. That is, after making these sources modular enough to not pull in unnecessary libcurl parts. For that, a slimmed-down curl_global_init()/-cleanup() is required at the minimum, then dealing with alloc/free, slist, curl_version_info(), curl_*printf(), curl_easy_[un]escape(), then curl_url_get().
They do not run make test. This is something to implement locally (not in curl-for-win). Switching to a Windows native CI host might make this easier, and it will need Python. Overall probably not a huge issue given that other CI jobs already run this and there isn't much (any?) platform dependent logic in it. UPDATE: It runs tests now.
They can be trivially extended to other platforms curl-for-win supports: Linux MUSL, Linux glibc and macOS.
They depend on curl-for-win Git repository head. Which may be temporarly broken. This happens rarely and it's covered by curl-for-win daily builds.
They also depend on curl Git repository master. Same caveat applies, but in practice it also builds fine almost always. Switching to use the latest stable curl is a trivial configuration option: -dev → -test in CW_CONFIG.

vszakats commented 10 months ago

@bagder: Yes, the artifact was tailored for curl. For trurl this can be re-done locally by picking out bin/turl.exe and bin/libcurl*.dll from the artifact. I'm thinking of an option to leave the necessary directories on disk (DONE → https://github.com/curl/curl-for-win/commit/67a04d17f5ec442548c71a942d0eac83c65fd1e2). (I was also thinking doing this for the standalone curl binaries for easier access that doesn't require any unpacking, though GHA will zip it anyway, so didn't pursue this just yet.)

vszakats commented 10 months ago

Improved artifacts with nothing else than necessary for trurl: https://github.com/curl/trurl/actions/runs/6815583239

curl-8.5.0-DEV_4fc402b9f2cefa597f58b9703293af2992988d68-win32-mingw-pico/bin/libcurl.dll
curl-8.5.0-DEV_4fc402b9f2cefa597f58b9703293af2992988d68-win32-mingw-pico/bin/trurl.exe
curl-8.5.0-DEV_4fc402b9f2cefa597f58b9703293af2992988d68-win64-mingw-pico/bin/libcurl-x64.dll
curl-8.5.0-DEV_4fc402b9f2cefa597f58b9703293af2992988d68-win64-mingw-pico/bin/trurl.exe
curl-8.5.0-DEV_4fc402b9f2cefa597f58b9703293af2992988d68-win64a-mingw-pico/bin/libcurl-arm64.dll
curl-8.5.0-DEV_4fc402b9f2cefa597f58b9703293af2992988d68-win64a-mingw-pico/bin/trurl.exe
urls.txt

vszakats commented 10 months ago

Now with a 'flat' package artifact:

curl-8.5.0-DEV_8197af4364d7c8402bc3a36f809801508ad2e96c-win32-mingw-zero/libcurl.dll
curl-8.5.0-DEV_8197af4364d7c8402bc3a36f809801508ad2e96c-win32-mingw-zero/trurl.exe
curl-8.5.0-DEV_8197af4364d7c8402bc3a36f809801508ad2e96c-win64-mingw-zero/libcurl-x64.dll
curl-8.5.0-DEV_8197af4364d7c8402bc3a36f809801508ad2e96c-win64-mingw-zero/trurl.exe
curl-8.5.0-DEV_8197af4364d7c8402bc3a36f809801508ad2e96c-win64a-mingw-zero/libcurl-arm64.dll
curl-8.5.0-DEV_8197af4364d7c8402bc3a36f809801508ad2e96c-win64a-mingw-zero/trurl.exe

vszakats commented 10 months ago

Made some finishing touches and implemented a flattening option in curl-for-win (so now it works for all platforms): https://github.com/curl/trurl/actions/runs/6828186933

vszakats commented 10 months ago

Made trurl.exe static (tested for Windows for now). The artifact now:

curl-8.5.0-DEV_3a2d22a9204d9e93e3820e275164e708dcd8552b-win32-mingw-zero/trurl.exe
curl-8.5.0-DEV_3a2d22a9204d9e93e3820e275164e708dcd8552b-win64-mingw-zero/trurl.exe
curl-8.5.0-DEV_3a2d22a9204d9e93e3820e275164e708dcd8552b-win64a-mingw-zero/trurl.exe

vszakats commented 10 months ago

When trying to run tests (after fixing a bunch of roadblocks), there are 10 tests still failing: https://github.com/curl/trurl/actions/runs/6864110037/job/18665177457#step:3:4226

These are test tests failing in the Windows libcurl + wine configuration:

20: passed  https://curl.se:22/ -s port=443
21: passed  https://curl.se:22/ -s port=443 --get '{url}'
23: passed  --default-port --url https://curl.se/we/are.html --get '{port}'
27: passed  --url 'imap://hello:secret;crazy@curl.se/we/are.html' --get '{options}'
35: passed  --url https://curl.se/we/are.html -g '{default:port}'
59: passed  https://hello:443/foo
60: passed  ftp://hello:21/foo
62: passed  ftp://hello:443/foo -s scheme=https
84: passed  https://curl.se --iterate 'port=80 81 443'
105: passed 'imaps://user:password;crazy@[ff00::1234%hello]:1234/path?a=b&c=d#fragment' --json

E.g.:

59: failed  https://hello:443/foo
--- stdout --- 
expected:
'https://hello/foo\n'
got:
'https://hello:443/foo\n'

Does anybody have an idea where the problem might be?

UPDATE-1: This replicates with a local macOS build, using curl 8.5.0-DEV. UPDATE-2: Also with curl 8.4.0 stable. UPDATE-3: Also with a local Linux MUSL build, using curl 8.5.0-DEV.

vszakats commented 10 months ago

Right, the solution is that I reduced the libcurl build to the absolute minimum, meaning no TLS, but, this also makes curl not recognize the HTTPS scheme anymore, thus not regarding 443 a default port for HTTPS. And since by default trurl hides default ports, it hides it with a libcurl built with a TLS backend and shows it with a libcurl without a TLS backend.

This extends to all schemes a curl build might recognize, e.g. FTP in above failures (which FWIW are also disabled in the libcurl here).

If anything, tests should not rely on such property of libcurl, so one good fix is to add --keep-port to these.

vszakats commented 10 months ago

Down to:

28: failed  --url '***curl.se/we/are.html' --get '{options}'
106: failed '***[ff00::1234%hello]:1234/path?a=b&c=d#fragment' --json

UPDATE: Fixed by enabling IMAP in libcurl and moving imaps to imap to work without TLS.

curl / trurl

ci: add Windows builds + tests #249