[Bug]: building with haproxy enabled fails (enabling HAProxy CI builds)

[chipitsine]

Contact Details

No response

Version

v5.5.4-stable

Description

I'm adding haproxy CI build for WolfSSL. so far, it fails

Reproduction steps

       autoreconf -iv
       ./configure --enable-all --prefix="${HOME}/opt"
        make all install

haproxy is built using:

USE_QUIC=1 USE_OPENSSL_WOLFSSL=1 SSL_LIB=${HOME}/opt/lib" SSL_INC=${HOME}/opt/include"

failed build log: https://github.com/chipitsine/haproxy/actions/runs/4206257921/jobs/7299503018

Relevant log output

CC      src/ssl_sample.o
In file included from /home/runner/opt/include/wolfssl/openssl/asn1.h:27,
                 from /home/runner/opt/include/wolfssl/ssl.h:4143,
                 from /home/runner/opt/include/wolfssl/openssl/conf.h:41,
                 from /home/runner/opt/include/wolfssl/openssl/crypto.h:44,
                 from include/haproxy/openssl-compat.h:11,
                 from src/ssl_sock.c:62:
/home/runner/opt/include/wolfssl/openssl/ssl.h:1610:41: error: static declaration of ‘wolfSSL_X509_STORE_get0_objects’ follows non-static declaration
 1610 | #define X509_STORE_get0_objects         wolfSSL_X509_STORE_get0_objects
      |                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/haproxy/openssl-compat.h:224:38: note: in expansion of macro ‘X509_STORE_get0_objects’
  224 | static inline STACK_OF(X509_OBJECT) *X509_STORE_get0_objects(X509_STORE *st)
      |                                      ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated due to -Wfatal-errors.
make: *** [Makefile:988: src/ssl_sock.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from /home/runner/opt/include/wolfssl/openssl/asn1.h:27,
                 from /home/runner/opt/include/wolfssl/ssl.h:4143,
                 from /home/runner/opt/include/wolfssl/openssl/conf.h:41,
                 from /home/runner/opt/include/wolfssl/openssl/crypto.h:44,
                 from include/haproxy/openssl-compat.h:11,
                 from include/haproxy/ssl_ckch-t.h:38,
                 from include/haproxy/ssl_ckch.h:26,
                 from src/ssl_ckch.c:35:
/home/runner/opt/include/wolfssl/openssl/ssl.h:1610:41: error: static declaration of ‘wolfSSL_X509_STORE_get0_objects’ follows non-static declaration
 1610 | #define X509_STORE_get0_objects         wolfSSL_X509_STORE_get0_objects
      |                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/haproxy/openssl-compat.h:224:38: note: in expansion of macro ‘X509_STORE_get0_objects’
  224 | static inline STACK_OF(X509_OBJECT) *X509_STORE_get0_objects(X509_STORE *st)
      |                                      ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated due to -Wfatal-errors.
make: *** [Makefile:988: src/ssl_ckch.o] Error 1
Error: Process completed with exit code 2.

[kareem-wolfssl]

Thanks for the report. I can reproduce this on master. Looking into this now.

[kareem-wolfssl]

As part of --enable-all, we are enabling other OpenSSL compatibility support, for example our QT support. This overrides OPENSSL_VERSION_NUMBER to a lower version than 0x1010000fL, which is required for HAProxy to build with wolfSSL. HAProxy is building for me using --enable-haproxy --enable-quic without --enable-all. Is there a reason you need --enable-all? We generally do not recommend using this option.

[chipitsine]

I first tried --enable-haproxy --enable-quic, but it did not pass automatic testing

here's another attempt: https://github.com/chipitsine/haproxy/commit/0d0b55b55c2e2dc0b16ff4faa3598a32fa9456df

build log (build is ok, but tests fail): https://github.com/chipitsine/haproxy/actions/runs/4206927500/jobs/7301056402

I guess other parts of WolfSSL should be enabled as well like "ocsp" and maybe few others. so, I came to an idea to try "enable-all"

it would be really nice if all required features are enabled altogether by --enable-haproxy

[chipitsine]

it is not that bad. 119 tests passed, 18 tests failed.

[chipitsine]

the most scary is signal 9, i.e. SEGFAULT

Error: # top TEST reg-tests/ssl/show_ssl_ocspresponse.vtc FAILED (10.011) signal=9

[kareem-wolfssl]

Let me look into this. We can bundle QUIC into --enable-haproxy, but QUIC seems to be an optional feature for HAProxy, we usually only include required features so users can enable optional features as desired.

[chipitsine]

if you want to run vtest locally, here's example

https://github.com/haproxy/haproxy/blob/master/.github/workflows/openssl-nodeprecated.yml

please see steps "Install VTest" and "Run VTest"

[tmael]

Hi @chipitsine, Can you test against the latest wolfssl commit? There have been a couple of fixes that are not part of 5.5.4 release that should help the haproxy tests.

[kareem-wolfssl]

Thank you both. Indeed as Tesfa said, I see less failures with master. I currently see six failures:

########################## Starting vtest ##########################
Testing with haproxy version: 2.8-dev4-5faf57-30
#    top  TEST reg-tests/http-messaging/srv_ws.vtc FAILED (0.230) exit=2
#    top  TEST reg-tests/ssl/ssl_client_auth.vtc FAILED (5.109) exit=2
#    top  TEST reg-tests/ssl/ssl_generate_certificate.vtc FAILED (3.124) exit=2
#    top  TEST reg-tests/ssl/ssl_client_samples.vtc FAILED (5.110) exit=2
#    top  TEST reg-tests/ssl/ssl_reuse.vtc FAILED (0.121) exit=2
#    top  TEST reg-tests/ssl/ssl_default_server.vtc FAILED (5.114) exit=2
6 tests failed, 20 tests skipped, 117 tests passed

Looking into what's going wrong. I see a couple of tests are not getting the headers set to what they expect, one is getting a different response code, and the rest are timing out. I will continue looking into fixing these.

[chipitsine]

thanks, I'll continue with master.

[chipitsine]

there's hint how to run single test in debug mode

make reg-tests VTEST_PROGRAM=../vtest/vtest -- --debug reg-tests/http-messaging/srv_ws.vtc

this particular test fails on

**   c2    === expect resp.http.x-alpn == "http/1.1"
---- c2    EXPECT resp.http.x-alpn () == "http/1.1" failed

Sirs, can you review https://github.com/haproxy/haproxy/blob/master/src/ssl_sample.c#L1371 ? seems alpn fetch does not work.

is it by design ? (if so, we can skip this test, but I'm afraid that people expect sample fetches to work..,..) should WolfSSL be built with some extra --enable ?

is it bug ?

[wlallemand]

There were some work from HAProxy side to make the CI work, some features are still missing so all tests can't work for now. I think you can close the ticket since there are more precise one that are open regarding missing features.