Building with OpenSSL 3.x leads to Core Dumps

ghost commented 4 months ago

Hey folks,

when I build NGINX from source using OpenSSL 3.x (3.2.1 tested) I run into core dump issues when I try to access an endpoint (location /test for example)that uses the ngx-http-auth-jwt-module. The compile process itself is working as it seems ... Any way to get this fixed? OpenSSL 1.1.1w is working fine.

Besides, OpenSSL 1.1.1 is deprecated, and should not be used anymore.

Thanks in advance

JoshMcCullough commented 4 months ago

Hey, which distro are you using that includes OpenSSL 3.2? Hopefully one that's Debian based so we can easily test it out!

ghost commented 4 months ago

@JoshMcCullough, I'm using the following docker image:

python:3.12.2-slim-bullseye.

I also tested against python:3.12.2-slim-bookworm, but here the compile process of libjwt fails, please see error below (this is still using OpenSSL 1.1.1w, so it appears to be a toolchain issue with newer versions of gcc, espcailly because it only happends on ARM64/aarch64):

#25 7.880 config.status: creating Makefile
#25 7.893 config.status: creating include/Makefile
#25 7.914 config.status: creating libjwt/Makefile
#25 7.939 config.status: creating examples/Makefile
#25 7.964 config.status: creating tests/Makefile
#25 7.986 config.status: creating libjwt/libjwt.pc
#25 8.014 config.status: creating dist/Makefile
#25 8.037 config.status: creating dist/libjwt.spec
#25 8.056 config.status: creating libjwt/config.h
#25 8.071 config.status: executing depfiles commands
#25 8.191 config.status: executing libtool commands
#25 8.261 Making all in include
#25 8.263 make[1]: Entering directory '/tmp/app/libjwt-1.17.0/include'
#25 8.263 make[1]: Nothing to be done for 'all'.
#25 8.263 make[1]: Leaving directory '/tmp/app/libjwt-1.17.0/include'
#25 8.264 Making all in libjwt
#25 8.267 make[1]: Entering directory '/tmp/app/libjwt-1.17.0/libjwt'
#25 8.270 make  all-am
#25 8.274 make[2]: Entering directory '/tmp/app/libjwt-1.17.0/libjwt'
#25 8.274 /bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.  -I../include    -Wall -DHAVE_OPENSSL  -I/usr/local/include     -D_GNU_SOURCE -g -O2 -MT libjwt_la-jwt.lo -MD -MP -MF .deps/libjwt_la-jwt.Tpo -c -o libjwt_la-jwt.lo `test -f 'jwt.c' || echo './'`jwt.c
#25 8.331 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -Wall -DHAVE_OPENSSL -I/usr/local/include -D_GNU_SOURCE -g -O2 -MT libjwt_la-jwt.lo -MD -MP -MF .deps/libjwt_la-jwt.Tpo -c jwt.c  -fPIC -DPIC -o .libs/libjwt_la-jwt.o
#25 8.888 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -Wall -DHAVE_OPENSSL -I/usr/local/include -D_GNU_SOURCE -g -O2 -MT libjwt_la-jwt.lo -MD -MP -MF .deps/libjwt_la-jwt.Tpo -c jwt.c -o libjwt_la-jwt.o >/dev/null 2>&1
#25 9.498 mv -f .deps/libjwt_la-jwt.Tpo .deps/libjwt_la-jwt.Plo
#25 9.501 /bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.  -I../include    -Wall -DHAVE_OPENSSL  -I/usr/local/include     -D_GNU_SOURCE -g -O2 -MT libjwt_la-base64.lo -MD -MP -MF .deps/libjwt_la-base64.Tpo -c -o libjwt_la-base64.lo `test -f 'base64.c' || echo './'`base64.c
#25 9.558 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -Wall -DHAVE_OPENSSL -I/usr/local/include -D_GNU_SOURCE -g -O2 -MT libjwt_la-base64.lo -MD -MP -MF .deps/libjwt_la-base64.Tpo -c base64.c  -fPIC -DPIC -o .libs/libjwt_la-base64.o
#25 9.640 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -Wall -DHAVE_OPENSSL -I/usr/local/include -D_GNU_SOURCE -g -O2 -MT libjwt_la-base64.lo -MD -MP -MF .deps/libjwt_la-base64.Tpo -c base64.c -o libjwt_la-base64.o >/dev/null 2>&1
#25 9.712 mv -f .deps/libjwt_la-base64.Tpo .deps/libjwt_la-base64.Plo
#25 9.715 /bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.  -I../include    -Wall -DHAVE_OPENSSL  -I/usr/local/include     -D_GNU_SOURCE -g -O2 -MT libjwt_la-jwt-openssl.lo -MD -MP -MF .deps/libjwt_la-jwt-openssl.Tpo -c -o libjwt_la-jwt-openssl.lo `test -f 'jwt-openssl.c' || echo './'`jwt-openssl.c
#25 9.766 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -Wall -DHAVE_OPENSSL -I/usr/local/include -D_GNU_SOURCE -g -O2 -MT libjwt_la-jwt-openssl.lo -MD -MP -MF .deps/libjwt_la-jwt-openssl.Tpo -c jwt-openssl.c  -fPIC -DPIC -o .libs/libjwt_la-jwt-openssl.o
#25 9.852 jwt-openssl.c: In function 'jwt_sign_sha_pem':
#25 9.852 jwt-openssl.c:282:17: warning: 'EVP_PKEY_get1_EC_KEY' is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
#25 9.852   282 |                 ec_key = EVP_PKEY_get1_EC_KEY(pkey);
#25 9.852       |                 ^~~~~~
#25 9.852 In file included from jwt-openssl.c:14:
#25 9.852 /usr/include/openssl/evp.h:1374:19: note: declared here
#25 9.852  1374 | struct ec_key_st *EVP_PKEY_get1_EC_KEY(EVP_PKEY *pkey);
#25 9.852       |                   ^~~~~~~~~~~~~~~~~~~~
#25 9.852 jwt-openssl.c:286:17: warning: 'EC_KEY_get0_group' is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
#25 9.852   286 |                 degree = EC_GROUP_get_degree(EC_KEY_get0_group(ec_key));
#25 9.852       |                 ^~~~~~
#25 9.852 In file included from /usr/include/openssl/x509.h:33,
#25 9.852                  from /usr/include/openssl/pem.h:23,
#25 9.852                  from jwt-openssl.c:17:
#25 9.852 /usr/include/openssl/ec.h:1034:39: note: declared here
#25 9.852  1034 | OSSL_DEPRECATEDIN_3_0 const EC_GROUP *EC_KEY_get0_group(const EC_KEY *key);
#25 9.852       |                                       ^~~~~~~~~~~~~~~~~
#25 9.852 jwt-openssl.c:288:17: warning: 'EC_KEY_free' is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
#25 9.852   288 |                 EC_KEY_free(ec_key);
#25 9.852       |                 ^~~~~~~~~~~
#25 9.852 /usr/include/openssl/ec.h:1003:28: note: declared here
#25 9.852  1003 | OSSL_DEPRECATEDIN_3_0 void EC_KEY_free(EC_KEY *key);
#25 9.852       |                            ^~~~~~~~~~~
#25 9.853 jwt-openssl.c: In function 'jwt_verify_sha_pem':
#25 9.853 jwt-openssl.c:433:17: warning: 'EVP_PKEY_get1_EC_KEY' is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
#25 9.853   433 |                 ec_key = EVP_PKEY_get1_EC_KEY(pkey);
#25 9.853       |                 ^~~~~~
#25 9.853 /usr/include/openssl/evp.h:1374:19: note: declared here
#25 9.853  1374 | struct ec_key_st *EVP_PKEY_get1_EC_KEY(EVP_PKEY *pkey);
#25 9.853       |                   ^~~~~~~~~~~~~~~~~~~~
#25 9.853 jwt-openssl.c:437:17: warning: 'EC_KEY_get0_group' is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
#25 9.853   437 |                 degree = EC_GROUP_get_degree(EC_KEY_get0_group(ec_key));
#25 9.853       |                 ^~~~~~
#25 9.853 /usr/include/openssl/ec.h:1034:39: note: declared here
#25 9.853  1034 | OSSL_DEPRECATEDIN_3_0 const EC_GROUP *EC_KEY_get0_group(const EC_KEY *key);
#25 9.853       |                                       ^~~~~~~~~~~~~~~~~
#25 9.853 jwt-openssl.c:439:17: warning: 'EC_KEY_free' is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
#25 9.853   439 |                 EC_KEY_free(ec_key);
#25 9.853       |                 ^~~~~~~~~~~
#25 9.853 /usr/include/openssl/ec.h:1003:28: note: declared here
#25 9.853  1003 | OSSL_DEPRECATEDIN_3_0 void EC_KEY_free(EC_KEY *key);
#25 9.853       |                            ^~~~~~~~~~~
#25 9.964 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -Wall -DHAVE_OPENSSL -I/usr/local/include -D_GNU_SOURCE -g -O2 -MT libjwt_la-jwt-openssl.lo -MD -MP -MF .deps/libjwt_la-jwt-openssl.Tpo -c jwt-openssl.c -o libjwt_la-jwt-openssl.o >/dev/null 2>&1
#25 10.18 mv -f .deps/libjwt_la-jwt-openssl.Tpo .deps/libjwt_la-jwt-openssl.Plo
#25 10.18 /bin/bash ../libtool  --tag=CC   --mode=link gcc -I/usr/local/include     -D_GNU_SOURCE -g -O2 -version-info 12:0:10    -no-undefined  -o libjwt.la -rpath /usr/local/lib libjwt_la-jwt.lo libjwt_la-base64.lo libjwt_la-jwt-openssl.lo  -L/usr/local/lib -ljansson  -lssl -lcrypto    
#25 10.28 libtool: link: gcc -shared  -fPIC -DPIC  .libs/libjwt_la-jwt.o .libs/libjwt_la-base64.o .libs/libjwt_la-jwt-openssl.o   -L/usr/local/lib -ljansson -lssl -lcrypto  -g -O2   -Wl,-soname -Wl,libjwt.so.2 -o .libs/libjwt.so.2.10.0
#25 10.35 libtool: link: (cd ".libs" && rm -f "libjwt.so.2" && ln -s "libjwt.so.2.10.0" "libjwt.so.2")
#25 10.36 libtool: link: (cd ".libs" && rm -f "libjwt.so" && ln -s "libjwt.so.2.10.0" "libjwt.so")
#25 10.37 libtool: link: ar cr .libs/libjwt.a  libjwt_la-jwt.o libjwt_la-base64.o libjwt_la-jwt-openssl.o
#25 10.41 libtool: link: ranlib .libs/libjwt.a
#25 10.45 libtool: link: ( cd ".libs" && rm -f "libjwt.la" && ln -s "../libjwt.la" "libjwt.la" )
#25 10.47 make[2]: Leaving directory '/tmp/app/libjwt-1.17.0/libjwt'
#25 10.47 make[1]: Leaving directory '/tmp/app/libjwt-1.17.0/libjwt'
#25 10.47 Making all in tests
#25 10.47 make[1]: Entering directory '/tmp/app/libjwt-1.17.0/tests'
#25 10.47 make[1]: Nothing to be done for 'all'.
#25 10.47 make[1]: Leaving directory '/tmp/app/libjwt-1.17.0/tests'
#25 10.47 Making all in dist
#25 10.48 make[1]: Entering directory '/tmp/app/libjwt-1.17.0/dist'
#25 10.48 make[1]: Nothing to be done for 'all'.
#25 10.48 make[1]: Leaving directory '/tmp/app/libjwt-1.17.0/dist'
#25 10.48 Making all in examples
#25 10.48 make[1]: Entering directory '/tmp/app/libjwt-1.17.0/examples'
#25 10.48 gcc -DHAVE_CONFIG_H -I. -I../libjwt  -I../include  -Wall -DKEYDIR="\"./keys\"" -D_GNU_SOURCE -g -O2 -MT main-gen.o -MD -MP -MF .deps/main-gen.Tpo -c -o main-gen.o main-gen.c
#25 10.57 mv -f .deps/main-gen.Tpo .deps/main-gen.Po
#25 10.57 /bin/bash ../libtool  --tag=CC   --mode=link gcc -Wall -DKEYDIR="\"./keys\"" -D_GNU_SOURCE -g -O2 -L../libjwt  -o jwtgen main-gen.o -ljwt 
#25 10.69 libtool: link: gcc -Wall -DKEYDIR=\"./keys\" -D_GNU_SOURCE -g -O2 -o .libs/jwtgen main-gen.o  -L../libjwt /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_true@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_set_alloc_funcs@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_dumps@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_key_to_iter@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_iter_value@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_integer@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_iter@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_del@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_loads@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_string@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_set_new@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_update@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_iter_next@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_false@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_deep_copy@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_string_value@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_delete@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_equal@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_get@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_clear@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_iter_key@JANSSON_4'
#25 10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_integer_value@JANSSON_4'
#25 10.74 collect2: error: ld returned 1 exit status
#25 10.74 make[1]: *** [Makefile:454: jwtgen] Error 1
#25 10.74 make[1]: Leaving directory '/tmp/app/libjwt-1.17.0/examples'
#25 10.74 make: *** [Makefile:493: all-recursive] Error 1
#25 ERROR: process "/bin/sh -c cd /tmp/app/libjwt-\"${RESTY_LIBJWT_VERSION}\" &&     autoreconf -i &&     ./configure &&     make all &&     make install" did not complete successfully: exit code: 2
------
 > [openresty_builder  8/12] RUN cd /tmp/app/libjwt-"1.17.0" &&     autoreconf -i &&     ./configure &&     make all &&     make install:
10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_delete@JANSSON_4'
10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_equal@JANSSON_4'
10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_get@JANSSON_4'
10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_clear@JANSSON_4'
10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_iter_key@JANSSON_4'
10.74 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_integer_value@JANSSON_4'
10.74 collect2: error: ld returned 1 exit status
10.74 make[1]: *** [Makefile:454: jwtgen] Error 1
10.74 make[1]: Leaving directory '/tmp/app/libjwt-1.17.0/examples'
10.74 make: *** [Makefile:493: all-recursive] Error 1
------
Dockerfile:275
--------------------
 274 |     # Installing LibJWT
 275 | >>> RUN cd /tmp/app/libjwt-"${RESTY_LIBJWT_VERSION}" && \
 276 | >>>     autoreconf -i && \
 277 | >>>     ./configure && \
 278 | >>>     make all && \
 279 | >>>     make install
 280 |     
--------------------

Don't get me wrong, I can build the module, but I can't use it, as soon as I hit a jwt_auth enabled endpoint/location we crash with core dump. I also tested this with 3.0.13 LTS of OpenSSL, same result. I'm building the module on top of OpenResty version 1.25.3.1.

The problem for me is that I absolutely rely on the ngx-http-auth-jwt-module , effectively blocking me from upgrading to a proper OpenSSL version

JoshMcCullough commented 4 months ago

I've been trying to get a working container w/debian:bookworm-slim and OpenSSL 3.2.1-2 but have not had luck yet...

Dockerfile

ARG IMAGE_TAG=bookworm-slim
ARG OPEN_SSL_VER

FROM debian:${IMAGE_TAG}
ARG OPEN_SSL_VER=3.2.1-2
RUN <<`
    set -e
    apt update
    apt install -y ca-certificates
    echo 'deb https://deb.debian.org/debian experimental main' > /etc/apt/sources.list.d/debian-experimental.list
    apt update
    apt upgrade
`
RUN <<`
    set -e
    apt install -y openssl=${OPEN_SSL_VER}
`

How are you getting openssl 3.2.1-2 on Debian? And, I have to ask, are you sure you can't use version 3.0.11-1 that comes with bookworm?

ghost commented 4 months ago

Naa, just build that stuff from source like a pro ;)


ARG RESTY_OPENSSL_VERSION="1.1.1w" # or 3.2.1

RUN     if [ ! -f ./openssl-"${RESTY_OPENSSL_VERSION}".tar.gz ]; then \
      curl -f -o openssl-"${RESTY_OPENSSL_VERSION}".tar.gz https://www.openssl.org/source/openssl-"${RESTY_OPENSSL_VERSION}".tar.gz; \
    fi && \
    if [ ! -d ./openssl-"${RESTY_OPENSSL_VERSION}" ]; then \
        tar -zxf openssl-"${RESTY_OPENSSL_VERSION}".tar.gz; \
    fi

RUN cd /tmp/app/openssl-"${RESTY_OPENSSL_VERSION}" && \
    ./config --prefix=/usr/local/openssl --openssldir=/usr/local/openssl shared zlib && \
    make && \
    make test && \
    make install_sw

JoshMcCullough commented 4 months ago

I was originally building from source but also got blocked. Will look at it again...

ghost commented 4 months ago

Merci, this one is for you ❤️

JoshMcCullough commented 4 months ago

We're using the debian image, and this is what I'm doing to build from source. It "works" but I get errors when running my built version:

docker run -it --rm debian-bullseye-slim:openssl-3.2.1 bash root@9192d5caa641:/# /usr/local/openssl/bin/openssl version /usr/local/openssl/bin/openssl: /lib/x86_64-linux-gnu/libssl.so.3: version OPENSSL_3.2.0' not found (required by /usr/local/openssl/bin/openssl) /usr/local/openssl/bin/openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: versionOPENSSL_3.2.0' not found (required by /usr/local/openssl/bin/openssl)

Dockerfile

FROM debian:bookworm-slim
ARG SRC_DIR=/tmp/openssl-src
ARG OUT_DIR=/usr/local/openssl
ARG OPEN_SSL_VER=3.2.1
RUN <<`
    set -e
    apt-get update
    apt-get install -y curl wget build-essential libz-dev
    apt-get clean
`
WORKDIR ${SRC_DIR}
RUN <<`
    set -e
    wget https://www.openssl.org/source/openssl-${OPEN_SSL_VER}.tar.gz
    tar -xf openssl-${OPEN_SSL_VER}.tar.gz --strip-components=1
`
RUN ./config --prefix=${OUT_DIR} shared zlib
RUN make
RUN <<`
    set -e
    make install
    echo "${OUT_DIR}/lib" > /etc/ld.so.conf.d/openssl-${OPEN_SSL_VER}.conf
    ldconfig
`
WORKDIR /
RUN rm -rf ${SRC_DIR}

ghost commented 4 months ago

@JoshMcCullough Please install the following package:

libssl-dev

I also install it like this:


# Installing OpenSSL
RUN cd /tmp/app/openssl-"${RESTY_OPENSSL_VERSION}" && \
    ./config --prefix=/usr/local/openssl --openssldir=/usr/local/openssl shared zlib && \
    make && \
    make test && \
    make install_sw

In the end it should really only be gcc, make... build-essentials etc. to make OpenSSL build, no idea why you aren't finding it ...

JoshMcCullough commented 4 months ago

I got it working, and will continue on this...

ghost commented 3 months ago

@JoshMcCullough Any update so far?

JoshMcCullough commented 3 months ago

Hey, I just need to find time to continue on it. I've made progress but not there yet.

JoshMcCullough commented 3 months ago

@venomone see this PR: https://github.com/TeslaGov/ngx-http-auth-jwt-module/pull/126

I was able to build the module without issue for SSL v1.1.1w, v3.0.11, and v3.2.1. The very base-est image that's used to build NGINX, the module, run NGINX, and run the tests is a custom-build Debian image for the version of SSL being tested.

Perhaps I missed something, but I don't see the problem you're running into here. Please advise, thanks.

ghost commented 3 months ago

@JoshMcCullough First of all, thanks for your Work. I really appriciate it ❤️. Please give me a couple of days. I will Provide a dockerfile to reproduce the issue.

ghost commented 3 months ago

@JoshMcCullough Have you tries building this on aarch64 ? As the error here basically is (using openssl 3.2.1):


#25 11.34 Making all in include
#25 11.34 make[1]: Entering directory '/tmp/app/libjwt-1.17.0/include'
#25 11.34 make[1]: Nothing to be done for 'all'.
#25 11.34 make[1]: Leaving directory '/tmp/app/libjwt-1.17.0/include'
#25 11.34 Making all in libjwt
#25 11.35 make[1]: Entering directory '/tmp/app/libjwt-1.17.0/libjwt'
#25 11.36 make  all-am
#25 11.36 make[2]: Entering directory '/tmp/app/libjwt-1.17.0/libjwt'
#25 11.36 /bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.  -I../include    -Wall -DHAVE_OPENSSL  -I/usr/local/include     -D_GNU_SOURCE -g -O2 -MT libjwt_la-jwt.lo -MD -MP -MF .deps/libjwt_la-jwt.Tpo -c -o libjwt_la-jwt.lo `test -f 'jwt.c' || echo './'`jwt.c
#25 11.44 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -Wall -DHAVE_OPENSSL -I/usr/local/include -D_GNU_SOURCE -g -O2 -MT libjwt_la-jwt.lo -MD -MP -MF .deps/libjwt_la-jwt.Tpo -c jwt.c  -fPIC -DPIC -o .libs/libjwt_la-jwt.o
#25 12.08 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -Wall -DHAVE_OPENSSL -I/usr/local/include -D_GNU_SOURCE -g -O2 -MT libjwt_la-jwt.lo -MD -MP -MF .deps/libjwt_la-jwt.Tpo -c jwt.c -o libjwt_la-jwt.o >/dev/null 2>&1
#25 12.76 mv -f .deps/libjwt_la-jwt.Tpo .deps/libjwt_la-jwt.Plo
#25 12.76 /bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.  -I../include    -Wall -DHAVE_OPENSSL  -I/usr/local/include     -D_GNU_SOURCE -g -O2 -MT libjwt_la-base64.lo -MD -MP -MF .deps/libjwt_la-base64.Tpo -c -o libjwt_la-base64.lo `test -f 'base64.c' || echo './'`base64.c
#25 12.86 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -Wall -DHAVE_OPENSSL -I/usr/local/include -D_GNU_SOURCE -g -O2 -MT libjwt_la-base64.lo -MD -MP -MF .deps/libjwt_la-base64.Tpo -c base64.c  -fPIC -DPIC -o .libs/libjwt_la-base64.o
#25 13.00 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -Wall -DHAVE_OPENSSL -I/usr/local/include -D_GNU_SOURCE -g -O2 -MT libjwt_la-base64.lo -MD -MP -MF .deps/libjwt_la-base64.Tpo -c base64.c -o libjwt_la-base64.o >/dev/null 2>&1
#25 13.11 mv -f .deps/libjwt_la-base64.Tpo .deps/libjwt_la-base64.Plo
#25 13.11 /bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.  -I../include    -Wall -DHAVE_OPENSSL  -I/usr/local/include     -D_GNU_SOURCE -g -O2 -MT libjwt_la-jwt-openssl.lo -MD -MP -MF .deps/libjwt_la-jwt-openssl.Tpo -c -o libjwt_la-jwt-openssl.lo `test -f 'jwt-openssl.c' || echo './'`jwt-openssl.c
#25 13.18 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -Wall -DHAVE_OPENSSL -I/usr/local/include -D_GNU_SOURCE -g -O2 -MT libjwt_la-jwt-openssl.lo -MD -MP -MF .deps/libjwt_la-jwt-openssl.Tpo -c jwt-openssl.c  -fPIC -DPIC -o .libs/libjwt_la-jwt-openssl.o
#25 13.34 jwt-openssl.c: In function 'jwt_sign_sha_pem':
#25 13.34 jwt-openssl.c:282:17: warning: 'EVP_PKEY_get1_EC_KEY' is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
#25 13.34   282 |                 ec_key = EVP_PKEY_get1_EC_KEY(pkey);
#25 13.34       |                 ^~~~~~
#25 13.34 In file included from jwt-openssl.c:14:
#25 13.34 /usr/include/openssl/evp.h:1374:19: note: declared here
#25 13.34  1374 | struct ec_key_st *EVP_PKEY_get1_EC_KEY(EVP_PKEY *pkey);
#25 13.34       |                   ^~~~~~~~~~~~~~~~~~~~
#25 13.34 jwt-openssl.c:286:17: warning: 'EC_KEY_get0_group' is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
#25 13.34   286 |                 degree = EC_GROUP_get_degree(EC_KEY_get0_group(ec_key));
#25 13.34       |                 ^~~~~~
#25 13.34 In file included from /usr/include/openssl/x509.h:33,
#25 13.34                  from /usr/include/openssl/pem.h:23,
#25 13.34                  from jwt-openssl.c:17:
#25 13.34 /usr/include/openssl/ec.h:1034:39: note: declared here
#25 13.34  1034 | OSSL_DEPRECATEDIN_3_0 const EC_GROUP *EC_KEY_get0_group(const EC_KEY *key);
#25 13.34       |                                       ^~~~~~~~~~~~~~~~~
#25 13.34 jwt-openssl.c:288:17: warning: 'EC_KEY_free' is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
#25 13.34   288 |                 EC_KEY_free(ec_key);
#25 13.34       |                 ^~~~~~~~~~~
#25 13.34 /usr/include/openssl/ec.h:1003:28: note: declared here
#25 13.34  1003 | OSSL_DEPRECATEDIN_3_0 void EC_KEY_free(EC_KEY *key);
#25 13.34       |                            ^~~~~~~~~~~
#25 13.34 jwt-openssl.c: In function 'jwt_verify_sha_pem':
#25 13.34 jwt-openssl.c:433:17: warning: 'EVP_PKEY_get1_EC_KEY' is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
#25 13.34   433 |                 ec_key = EVP_PKEY_get1_EC_KEY(pkey);
#25 13.34       |                 ^~~~~~
#25 13.34 /usr/include/openssl/evp.h:1374:19: note: declared here
#25 13.34  1374 | struct ec_key_st *EVP_PKEY_get1_EC_KEY(EVP_PKEY *pkey);
#25 13.34       |                   ^~~~~~~~~~~~~~~~~~~~
#25 13.34 jwt-openssl.c:437:17: warning: 'EC_KEY_get0_group' is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
#25 13.34   437 |                 degree = EC_GROUP_get_degree(EC_KEY_get0_group(ec_key));
#25 13.34       |                 ^~~~~~
#25 13.34 /usr/include/openssl/ec.h:1034:39: note: declared here
#25 13.34  1034 | OSSL_DEPRECATEDIN_3_0 const EC_GROUP *EC_KEY_get0_group(const EC_KEY *key);
#25 13.34       |                                       ^~~~~~~~~~~~~~~~~
#25 13.34 jwt-openssl.c:439:17: warning: 'EC_KEY_free' is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
#25 13.34   439 |                 EC_KEY_free(ec_key);
#25 13.34       |                 ^~~~~~~~~~~
#25 13.34 /usr/include/openssl/ec.h:1003:28: note: declared here
#25 13.34  1003 | OSSL_DEPRECATEDIN_3_0 void EC_KEY_free(EC_KEY *key);
#25 13.34       |                            ^~~~~~~~~~~
#25 13.47 libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -Wall -DHAVE_OPENSSL -I/usr/local/include -D_GNU_SOURCE -g -O2 -MT libjwt_la-jwt-openssl.lo -MD -MP -MF .deps/libjwt_la-jwt-openssl.Tpo -c jwt-openssl.c -o libjwt_la-jwt-openssl.o >/dev/null 2>&1
#25 13.70 mv -f .deps/libjwt_la-jwt-openssl.Tpo .deps/libjwt_la-jwt-openssl.Plo
#25 13.71 /bin/bash ../libtool  --tag=CC   --mode=link gcc -I/usr/local/include     -D_GNU_SOURCE -g -O2 -version-info 12:0:10    -no-undefined  -o libjwt.la -rpath /usr/local/lib libjwt_la-jwt.lo libjwt_la-base64.lo libjwt_la-jwt-openssl.lo  -L/usr/local/lib -ljansson  -lssl -lcrypto    
#25 13.85 libtool: link: gcc -shared  -fPIC -DPIC  .libs/libjwt_la-jwt.o .libs/libjwt_la-base64.o .libs/libjwt_la-jwt-openssl.o   -L/usr/local/lib -ljansson -lssl -lcrypto  -g -O2   -Wl,-soname -Wl,libjwt.so.2 -o .libs/libjwt.so.2.10.0
#25 13.94 libtool: link: (cd ".libs" && rm -f "libjwt.so.2" && ln -s "libjwt.so.2.10.0" "libjwt.so.2")
#25 13.95 libtool: link: (cd ".libs" && rm -f "libjwt.so" && ln -s "libjwt.so.2.10.0" "libjwt.so")
#25 13.97 libtool: link: ar cr .libs/libjwt.a  libjwt_la-jwt.o libjwt_la-base64.o libjwt_la-jwt-openssl.o
#25 14.02 libtool: link: ranlib .libs/libjwt.a
#25 14.07 libtool: link: ( cd ".libs" && rm -f "libjwt.la" && ln -s "../libjwt.la" "libjwt.la" )
#25 14.08 make[2]: Leaving directory '/tmp/app/libjwt-1.17.0/libjwt'
#25 14.08 make[1]: Leaving directory '/tmp/app/libjwt-1.17.0/libjwt'
#25 14.08 Making all in tests
#25 14.08 make[1]: Entering directory '/tmp/app/libjwt-1.17.0/tests'
#25 14.08 make[1]: Nothing to be done for 'all'.
#25 14.08 make[1]: Leaving directory '/tmp/app/libjwt-1.17.0/tests'
#25 14.08 Making all in dist
#25 14.09 make[1]: Entering directory '/tmp/app/libjwt-1.17.0/dist'
#25 14.09 make[1]: Nothing to be done for 'all'.
#25 14.09 make[1]: Leaving directory '/tmp/app/libjwt-1.17.0/dist'
#25 14.09 Making all in examples
#25 14.10 make[1]: Entering directory '/tmp/app/libjwt-1.17.0/examples'
#25 14.10 gcc -DHAVE_CONFIG_H -I. -I../libjwt  -I../include  -Wall -DKEYDIR="\"./keys\"" -D_GNU_SOURCE -g -O2 -MT main-gen.o -MD -MP -MF .deps/main-gen.Tpo -c -o main-gen.o main-gen.c
#25 14.26 mv -f .deps/main-gen.Tpo .deps/main-gen.Po
#25 14.27 /bin/bash ../libtool  --tag=CC   --mode=link gcc -Wall -DKEYDIR="\"./keys\"" -D_GNU_SOURCE -g -O2 -L../libjwt  -o jwtgen main-gen.o -ljwt 
#25 14.46 libtool: link: gcc -Wall -DKEYDIR=\"./keys\" -D_GNU_SOURCE -g -O2 -o .libs/jwtgen main-gen.o  -L../libjwt /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_true@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_set_alloc_funcs@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_dumps@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_key_to_iter@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_iter_value@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_integer@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_iter@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_del@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_loads@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_string@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_set_new@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_update@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_iter_next@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_false@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_deep_copy@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_string_value@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_delete@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_equal@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_get@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_clear@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_iter_key@JANSSON_4'
#25 14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_integer_value@JANSSON_4'
#25 14.52 collect2: error: ld returned 1 exit status
#25 14.52 make[1]: *** [Makefile:454: jwtgen] Error 1
#25 14.52 make[1]: Leaving directory '/tmp/app/libjwt-1.17.0/examples'
#25 14.52 make: *** [Makefile:493: all-recursive] Error 1
#25 ERROR: process "/bin/sh -c cd /tmp/app/libjwt-\"${RESTY_LIBJWT_VERSION}\" &&     autoreconf -i &&     ./configure &&     make all &&     make install" did not complete successfully: exit code: 2
------
 > [openresty_builder  8/12] RUN cd /tmp/app/libjwt-"1.17.0" &&     autoreconf -i &&     ./configure &&     make all &&     make install:
14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_delete@JANSSON_4'
14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_equal@JANSSON_4'
14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_get@JANSSON_4'
14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_clear@JANSSON_4'
14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_object_iter_key@JANSSON_4'
14.51 /usr/bin/ld: /tmp/app/libjwt-1.17.0/libjwt/.libs/libjwt.so: undefined reference to `json_integer_value@JANSSON_4'
14.52 collect2: error: ld returned 1 exit status
14.52 make[1]: *** [Makefile:454: jwtgen] Error 1
14.52 make[1]: Leaving directory '/tmp/app/libjwt-1.17.0/examples'
14.52 make: *** [Makefile:493: all-recursive] Error 1
------
Dockerfile:275
--------------------
 274 |     # Installing LibJWT
 275 | >>> RUN cd /tmp/app/libjwt-"${RESTY_LIBJWT_VERSION}" && \
 276 | >>>     autoreconf -i && \
 277 | >>>     ./configure && \
 278 | >>>     make all && \
 279 | >>>     make install
 280 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c cd /tmp/app/libjwt-\"${RESTY_LIBJWT_VERSION}\" &&     autoreconf -i &&     ./configure &&     make all &&     make install" did not complete successfully: exit code: 2

JoshMcCullough commented 3 months ago

That's a problem building libjwt. Not related to OpenSSL, right?

ghost commented 3 months ago

Yes, that's correct. But I'm a bit confused. If I install jasson like this:

apt-get install libjansson-dev

I have no problem building libjwt against OpenSSL version 3.2.1. Everything is fine on x86_64 and also aarch64, just as I expect it to be. But if I build jasson from source, my libjwt build fails under arrach64, where x86_64 on the other hand works...

This is how I build jansson it:


RUN cd /tmp/app/jansson-"${RESTY_JANSSON_VERSION}" && \
    cmake . -DJANSSON_BUILD_SHARED_LIBS=1 -DJANSSON_BUILD_DOCS=OFF && \
    make && \
    make check && \
    make install && \
    echo '/usr/local/lib' > /etc/ld.so.conf.d/usr_local.conf && \
    /sbin/ldconfig

I tried jansson 2.14 and the current master, both have exactly the same outcome.

Do you see any reason here why this might run into problems under aarch64 specifically ...? If not I guess the issue is somewhere else on my side, and I simply go with the package libjansson-dev

JoshMcCullough commented 3 months ago

If not I guess the issue is somewhere else on my side, and I simply go with the package libjansson-dev

Why are you building from source if you don't have to? If there is a binary already built and ready to go, just use that!

We haven't tested building libjansson or libjwt from source, nor have we targeted aarch64.

ghost commented 3 months ago

@JoshMcCullough Alrighty, then let's close this issue, at least from my side, when using a pre-build jansson package, it's working for me.

TeslaGov / ngx-http-auth-jwt-module

Building with OpenSSL 3.x leads to Core Dumps #124

Dockerfile

Dockerfile