SSL support is not compiled (libpq)

[manifest]

I'm getting an error from the application with Diesel dependency that should have compiled statically with SSL

[2019-01-22T06:11:14Z ERROR r2d2] sslmode value "require" invalid when SSL support is not compiled in

I've compiled application with the following command

OPENSSL_STATIC=yes OPENSSL_LIB_DIR=/musl/lib/ OPENSSL_INCLUDE_DIR=/musl/include cargo build --release

Entire Dockerfile

## -----------------------------------------------------------------------------
## Build
## -----------------------------------------------------------------------------
FROM clux/muslrust as build-stage

WORKDIR "/build"
COPY . .
RUN OPENSSL_STATIC=yes OPENSSL_LIB_DIR=/musl/lib/ OPENSSL_INCLUDE_DIR=/musl/include cargo build --release

## -----------------------------------------------------------------------------
## Package
## -----------------------------------------------------------------------------
FROM alpine

COPY --from=build-stage "/build/target/x86_64-unknown-linux-musl/release/example" "/app/example"

WORKDIR "/app"
ENTRYPOINT ["/app/example"]

App's Cargo.toml

[dependencies]
diesel = { version = "1.3.3", features = ["postgres", "uuid", "serde_json", "r2d2"] }
openssl = "*"

[patch.crates-io]
diesel = { git = "https://github.com/diesel-rs/diesel", rev = "59aa49b" }
diesel_derives = { git = "https://github.com/diesel-rs/diesel", rev = "59aa49b" }

[manifest]

The same error with the command:

OPENSSL_STATIC=yes OPENSSL_LIB_DIR=/musl/lib/ OPENSSL_INCLUDE_DIR=/musl/include PQ_LIB_STATIC=yes PQ_LIB_DIR=/musl/lib/ cargo build --release

And Cargo.toml

[dependencies]
diesel = { version = "1.3.3", features = ["postgres", "uuid", "serde_json", "r2d2"] }
openssl = "*"
pq-sys = "*"

[patch.crates-io]
diesel = { git = "https://github.com/diesel-rs/diesel", rev = "59aa49b" }
diesel_derives = { git = "https://github.com/diesel-rs/diesel", rev = "59aa49b" }

[manifest]

I've played around libpg compilation adding --with-openssl to postgresql`s configure script and start getting an error on compilation

  = note: /build/target/x86_64-unknown-linux-musl/release/deps/libpq_sys-784bf4f1583faa91.rlib(fe-secure-openssl.o): In function `pgtls_init':
          fe-secure-openssl.c:(.text+0xf48): undefined reference to `OPENSSL_config'
          collect2: error: ld returned 1 exit status

     cd postgresql-$PQ_VER && \
     CC="musl-gcc -fPIE -pie" LDFLAGS="-L$PREFIX/lib" CFLAGS="-I$PREFIX/include" ./configure \
+    --with-openssl \

[clux]

Hey, thanks for raising this! This should indeed be supported. curl goes through the same dance.

It's probably a bit annoying to get it to work though. I'll try to have a look on the weekend, but can't promise anything.

In the mean time, any chance you can commit what you have as a test case in a PR?

[manifest]

@clux I've published a test case for the issue here.

Steps to reproduce:

1. Create a database

   ## Database conenction URL must contain `?sslmode=require`
   ## query string parameter to reproduce the issue
   export DATABASE_URL='postgres://localhost:5432/diesel-muslrust-example?sslmode=require'
   diesel setup --database-url ${DATABASE_URL}

2. Build and run a docker container

   docker build -t diesel-muslrust-example -f docker/Dockerfile .
   docker run --rm -ti --entrypoint=/bin/sh diesel-muslrust-example

3. Run the example app from within the container

   ## The app reads 'DATABASE_URL' from environment variable, so it should be exported
   export DATABASE_URL=''
   RUST_LOG=diesel_muslrust_example=info /app/diesel-muslrust-example

4. You should see an error after connection will fail with timeout

   thread 'main' panicked at 'Error creating a database pool: Error(Some("sslmode value \"require\" invalid when SSL support is not compiled in\n"))', src/libcore/result.rs:999:5
   note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

[manifest]

@clux have you had time to take a look on this one?

[clux]

Ah, sorry not yet. Busy weeks :slightly_frowning_face:

[clux]

Had a bit of a go today and added failing tests from your code in https://github.com/clux/muslrust/commit/16774454b092795300f94c6c39d7f7233a121140

Unfortunately, no matter what permutations i tried libpq will build, but cargo build will fail to link due to missing SSL_ symbols. Leaving a comment for the future with current status.

Testing method now is:

• tweak Dockerfile
• make clean
• make build
• make test-dieselpgssl

There's quite a lot of flags in libpq's ./configure though:

./configure -h

```aconf Usage: ./configure [OPTION]... [VAR=VALUE]... To assign environment variables (e.g., CC, CFLAGS...), specify them as VAR=VALUE. See below for descriptions of some of the useful variables. Defaults for the options are specified in brackets. Configuration: -h, --help display this help and exit --help=short display options specific to this package --help=recursive display the short help of all the included packages -V, --version display version information and exit -q, --quiet, --silent do not print `checking ...' messages --cache-file=FILE cache test results in FILE [disabled] -C, --config-cache alias for `--cache-file=config.cache' -n, --no-create do not create output files --srcdir=DIR find the sources in DIR [configure dir or `..'] Installation directories: --prefix=PREFIX install architecture-independent files in PREFIX [/usr/local/pgsql] --exec-prefix=EPREFIX install architecture-dependent files in EPREFIX [PREFIX] By default, `make install' will install all the files in `/usr/local/pgsql/bin', `/usr/local/pgsql/lib' etc. You can specify an installation prefix other than `/usr/local/pgsql' using `--prefix', for instance `--prefix=$HOME'. For better control, use the options below. Fine tuning of the installation directories: --bindir=DIR user executables [EPREFIX/bin] --sbindir=DIR system admin executables [EPREFIX/sbin] --libexecdir=DIR program executables [EPREFIX/libexec] --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] --datarootdir=DIR read-only arch.-independent data root [PREFIX/share] --datadir=DIR read-only architecture-independent data [DATAROOTDIR] --infodir=DIR info documentation [DATAROOTDIR/info] --localedir=DIR locale-dependent data [DATAROOTDIR/locale] --mandir=DIR man documentation [DATAROOTDIR/man] --docdir=DIR documentation root [DATAROOTDIR/doc/postgresql] --htmldir=DIR html documentation [DOCDIR] --dvidir=DIR dvi documentation [DOCDIR] --pdfdir=DIR pdf documentation [DOCDIR] --psdir=DIR ps documentation [DOCDIR] System types: --build=BUILD configure for building on BUILD [guessed] --host=HOST cross-compile to build programs to run on HOST [BUILD] Optional Features: --disable-option-checking ignore unrecognized --enable/--with options --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) --enable-FEATURE[=ARG] include FEATURE [ARG=yes] --disable-integer-datetimes obsolete option, no longer supported --enable-nls[=LANGUAGES] enable Native Language Support --disable-rpath do not embed shared library search path in executables --disable-spinlocks do not use spinlocks --disable-atomics do not use atomic operations --disable-strong-random do not use a strong random number source --enable-debug build with debugging symbols (-g) --enable-profiling build with profiling enabled --enable-coverage build with coverage testing instrumentation --enable-dtrace build with DTrace support --enable-tap-tests enable TAP tests (requires Perl and IPC::Run) --enable-depend turn on automatic dependency tracking --enable-cassert enable assertion checks (for debugging) --disable-thread-safety disable thread-safety in client libraries --disable-largefile omit support for large files --disable-float4-byval disable float4 passed by value --disable-float8-byval disable float8 passed by value Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) --with-extra-version=STRING append STRING to version --with-template=NAME override operating system template --with-includes=DIRS look for additional header files in DIRS --with-libraries=DIRS look for additional libraries in DIRS --with-libs=DIRS alternative spelling of --with-libraries --with-pgport=PORTNUM set default port number [5432] --with-blocksize=BLOCKSIZE set table block size in kB [8] --with-segsize=SEGSIZE set table segment size in GB [1] --with-wal-blocksize=BLOCKSIZE set WAL block size in kB [8] --with-wal-segsize=SEGSIZE set WAL segment size in MB [16] --with-CC=CMD set compiler (deprecated) --with-icu build with ICU support --with-tcl build Tcl modules (PL/Tcl) --with-tclconfig=DIR tclConfig.sh is in DIR --with-perl build Perl modules (PL/Perl) --with-python build Python modules (PL/Python) --with-gssapi build with GSSAPI support --with-krb-srvnam=NAME default service principal name in Kerberos (GSSAPI) [postgres] --with-pam build with PAM support --with-bsd-auth build with BSD Authentication support --with-ldap build with LDAP support --with-bonjour build with Bonjour support --with-openssl build with OpenSSL support --with-selinux build with SELinux support --with-systemd build with systemd support --without-readline do not use GNU Readline nor BSD Libedit for editing --with-libedit-preferred prefer BSD Libedit over GNU Readline --with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp) --with-ossp-uuid obsolete spelling of --with-uuid=ossp --with-libxml build with XML support --with-libxslt use XSLT support when building contrib/xml2 --with-system-tzdata=DIR use system time zone data in DIR --without-zlib do not use Zlib --with-gnu-ld assume the C compiler uses GNU ld [default=no] Some influential environment variables: CC C compiler command CFLAGS C compiler flags LDFLAGS linker flags, e.g. -L if you have libraries in a nonstandard directory LIBS libraries to pass to the linker, e.g. -l CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I if you have headers in a nonstandard directory CPP C preprocessor PKG_CONFIG path to pkg-config utility PKG_CONFIG_PATH directories to add to pkg-config's search path PKG_CONFIG_LIBDIR path overriding pkg-config's built-in search path ICU_CFLAGS C compiler flags for ICU, overriding pkg-config ICU_LIBS linker flags for ICU, overriding pkg-config LDFLAGS_EX extra linker flags for linking executables only LDFLAGS_SL extra linker flags for linking shared libraries only Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. ```

There might be one in there that'll fix it. Otherwise, there might be some magic in https://github.com/sgrif/pq-sys/blob/master/build.rs that we can use.

[clux]

oh, and saw this once during compilation of libpq:

configure: WARNING: openssl/ssl.h: accepted by the compiler, rejected by the preprocessor!
configure: WARNING: openssl/ssl.h: proceeding with the compiler's result

got rid of this warning by passing:

    --with-libraries=$PREFIX/lib \
    --with-includes=$PREFIX/include \

to ./configure. didn't seem to help any further though.

[clux]

Another diff that may or may not help bring this closer:

diff --git a/Dockerfile b/Dockerfile
index 9899a4f..46448c1 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -45,7 +45,7 @@ RUN curl https://sh.rustup.rs -sSf | \
 ENV SSL_VER="1.0.2q" \
     CURL_VER="7.64.0" \
     ZLIB_VER="1.2.11" \
-    PQ_VER="10.6" \
+    PQ_VER="9.6.8" \
     SQLITE_VER="3270200" \
     CC=musl-gcc \
     PREFIX=/musl \
@@ -91,10 +91,14 @@ RUN curl -sSL https://curl.haxx.se/download/curl-$CURL_VER.tar.gz | tar xz && \

 # Build libpq
 # TODO: fix so that --with-openssl works with pqssl tests
-RUN curl -sSL https://ftp.postgresql.org/pub/source/v$PQ_VER/postgresql-$PQ_VER.tar.gz | tar xz && \
-    cd postgresql-$PQ_VER && \
+RUN curl -sSL https://ftp.postgresql.org/pub/source/v$PQ_VER/postgresql-$PQ_VER.tar.gz | tar xz
+RUN cd postgresql-$PQ_VER && \
     CC="musl-gcc -fPIE -pie" LDFLAGS="-L$PREFIX/lib" CFLAGS="-I$PREFIX/include" ./configure \
     --without-readline \
+    --with-openssl \
+    --with-libraries=$PREFIX/lib \
+    --with-includes=$PREFIX/include \
+    --oldincludedir=/usr/include \
     --prefix=$PREFIX --host=x86_64-unknown-linux-musl && \
     cd src/interfaces/libpq make -s -j$(nproc) all-static-lib && make -s install install-lib-static && \
     cd ../../bin/pg_config && make -j $(nproc) && make install && \
@@ -121,13 +125,19 @@ ENV PATH=$PREFIX/bin:$PATH \
     PKG_CONFIG_ALLOW_CROSS=true \
     PKG_CONFIG_ALL_STATIC=true \
     PQ_LIB_STATIC_X86_64_UNKNOWN_LINUX_MUSL=true \
+    PQ_LIB_STATIC=true \
+    PQ_LIB_DIR=$PREFIX/lib/ \
+    OPENSSL_LIB_DIR=$PREFIX/lib/ \
+    OPENSSL_INCLUDE_DIR=$PREFIX/include \
+    DEP_OPENSSL_INCLUDE=$PREFIX/include/ \
     PKG_CONFIG_PATH=$PREFIX/lib/pkgconfig \
     PG_CONFIG_X86_64_UNKNOWN_LINUX_GNU=/usr/bin/pg_config \
     OPENSSL_STATIC=true \
     OPENSSL_DIR=$PREFIX \
     SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt \
     SSL_CERT_DIR=/etc/ssl/certs \
-    LIBZ_SYS_STATIC=1
+    LIBZ_SYS_STATIC=1 \
+    TARGET=musl

 # Allow ditching the -w /volume flag to docker run
 WORKDIR /volume

[clux]

the issue might be libpq configuration based on this output from its detection:

checking for CRYPTO_new_ex_data in -lcrypto... yes
checking for SSL_new in -lssl... yes
checking for SSL_get_current_compression... yes
checking for OPENSSL_init_ssl... no
checking for BIO_get_data... no
checking for BIO_meth_new... no
checking for ASN1_STRING_get0_data... no
checking for RAND_OpenSSL... no
checking for CRYPTO_lock... yes

but maybe that's just openssl 1.1 stuff it's testing for :face_with_head_bandage:

[emk]

Hello! I'm the maintainer of rust-musl-builder, and we ran into this problem lately. You might be interested in the following issues:

• https://github.com/emk/rust-musl-builder/issues/69
• https://github.com/sgrif/pq-sys/issues/25

TL;dr: It's possible to work around this by adding this to main.rs:

extern crate openssl;
#[macro_use]
extern crate diesel;

...but the workaround might break with certain projects or releases of Rust. I've filed a bug upstream to investigate further: https://github.com/sgrif/pq-sys/issues/25

[clux]

Oh wow, thanks a lot @emk ! It does look like the order of the includes actually fixes this!

Had a quick check compiling pg with --with-openssl and make test-dieselpgssl now both links and runs in this repo without ssl errors.

[clux]

Have made openssl required now, since there's now way of having the support without pulling in the crate, but it's documented in the main README. Will keep an eye on pq-sys#25 for a potential future improvement on this.