`--with-opt-dir` doesn't prevent Ruby from seeing OpenSSL 3 headers/lib

[dgholz]

Description

This does also happen when compiling Ruby manually, but I see in the Ruby MacOS CI steps that they use --with-openssl-dir instead.

Recently, Homebrew updated loads of formulæ to use OpenSSL 3, and I see the latest version of their OpenSSL package (3.1.1_1) get installed when I updated some frequently-used tools (like curl) . When I tried compiling Ruby afterwards, I was surprised to see some OpenSSL 3 errors appear in the output.

I traced it to how Ruby figures out where to find OpenSSL, and how the paths in the --with-opt-dir don't preclude it from making its own search. It then configures ~/src/ruby-3.1.4/ext/openssl/Makefile like:

[...]
INCFLAGS = -I. -I$(arch_hdrdir) -I$(hdrdir) -I$(srcdir) -I/opt/homebrew/Cellar/openssl@3/3.1.1_1/include
DEFS     =
CPPFLAGS = -DRUBY_EXTCONF_H=\"$(RUBY_EXTCONF_H)\" -I/opt/homebrew/opt/openssl@1.1/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/libyaml/include -I/opt/homebrew/opt/gdbm/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT $(DEFS) $(cppflags)
CXXFLAGS = $(CCDLFLAGS) -fdeclspec  $(ARCH_FLAG)
ldflags  = -L. -fstack-protector-strong -L/opt/homebrew/opt/openssl@1.1/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/libyaml/lib -L/opt/homebrew/opt/gdbm/lib -L/opt/homebrew/Cellar/openssl@3/3.1.1_1/lib
[...]

I noticed that specifying the path to OpenSSL explicitly let it work just fine:

$ ruby-install 3.1.4 -- --with-openssl-dir="$(brew --prefix openssl@1.1)"
[...]
>>> Successfully installed ruby 3.1.4 into /Users/danielholz/.rubies/ruby-3.1.4

Steps To Reproduce

Steps to reproduce the bug:

1. brew update && brew upgrade
   • or brew install openssl
   • or brew upgrade openssl
2. ruby-install 3.1.4

Expected Behavior

No weird mentions of OpenSSL 3 in compilation errors.

Actual Behavior

ossl_ts.c:829:5: error: incomplete definition of type 'struct TS_verify_ctx'
    TS_VERIFY_CTX_set_certs(ctx, x509inter);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./openssl_missing.h:215:46: note: expanded from macro 'TS_VERIFY_CTX_set_certs'
#  define TS_VERIFY_CTX_set_certs(ctx, crts) TS_VERIFY_CTS_set_certs(ctx, crts)
                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./openssl_missing.h:195:52: note: expanded from macro 'TS_VERIFY_CTS_set_certs'
#  define TS_VERIFY_CTS_set_certs(ctx, crts) ((ctx)->certs=(crts))
                                              ~~~~~^
/opt/homebrew/Cellar/openssl@3/3.1.1_1/include/openssl/ts.h:407:16: note: forward declaration of 'struct TS_verify_ctx'
typedef struct TS_verify_ctx TS_VERIFY_CTX;
               ^
1 warning and 1 error generated.
make[2]: *** [ossl_ts.o] Error 1
make[1]: *** [ext/openssl/all] Error 2
make: *** [build-ext] Error 2
!!! Compiling ruby 3.1.4 failed!
ERROR: Unable to install ruby version: 3.1.4

Environment

$ ruby-install --version
ruby-install: 0.9.0 # which matches https://github.com/postmodern/ruby-install/blob/v0.9.1/share/ruby-install/ruby-install.sh#L5

$ uname -a
Darwin daniel-holz-laptop 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun  8 22:22:20 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T6000 arm64`

$ cc --version
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

$ openssl version
OpenSSL 3.1.1 30 May 2023 (Library: OpenSSL 3.1.1 30 May 2023)

[dgholz]

I also noticed that unpinning OpenSSL in the ruby dependencies let Ruby compile just fine against OpenSSL 3.1.1.

I appreciate that strictly this looks like Ruby is not detecting the path to OpenSSL correctly, and I'll file a bug upstream if you think it's better suited there.

[dgholz]

Ah, this seems extremely similar to https://github.com/postmodern/ruby-install/issues/412#issuecomment-1118178152

Code-digging the same ditch(?), I see that the handling of --with-openssl-dir was changed so pkg-config won't be consulted if that option is passed.

[todd-a-jacobs]

I've been running into this, too. As a work-around, and assuming you are using Homebrew on an M1/M2 pointing to the /opt directory rather than installing into /usr/local, the following works for me:

LDFLAGS="-L/opt/homebrew/opt/openssl@1.1/lib" \
    CPPFLAGS="-I/opt/homebrew/opt/openssl@1.1/include" \
    PKG_CONFIG_PATH="/opt/homebrew/opt/openssl@1.1/lib/pkgconfig" \
    PATH="/opt/homebrew/opt/openssl@1.1/bin:$PATH" \
    ruby-install -c ruby 3.2.1 -- --enable-yjit

Note that these flags may be necessary regardless of the Ruby version you're installing if you need OpenSSL 1.1 and have both openssl@1 and openssl@3 installed. If your Homebrew is putting things under /usr/local you can adjust the paths above accordingly. Call brew info openssl@1.1 to find the right values for your system.

Until/unless Ruby can compile against OpenSSL 3, or against the macOS native or Homebrew versions of LibreSSL, this likely will be an ongoing pain point for some people. This includes those who may need rustc and/or a patched OpenSSL to build TruffleRuby-GraalVM from source instead of using Homebrew's GraalVM cask, which is another time-saver I highly recommend for those with SSL or Java build problems.

Hope this helps!

[todd-a-jacobs]

@dgholz said:

I also noticed that unpinning OpenSSL in the ruby dependencies let Ruby compile just fine against OpenSSL 3.1.1.

I think compiling against the latest stable OpenSSL or LibreSSL (whether statically or dynamically) is better for security than compiling against specific versions, since that allows for people to keep their systems patched. To be honest, though, I'm not sure whether the way Homebrew installs OpenSSL allows for the latest version on the system to be used, or if it compiles in the OpenSSL version that was present when Ruby was built by ruby-install. That's kind of an important question in my mind, and I don't currently know how that works with ruby-install.

[dgholz]

Thanks for the tips, I'll try them out.

To be honest, though, I'm not sure whether the way Homebrew installs OpenSSL allows for the latest version on the system to be used, or if it compiles in the OpenSSL version that was present when Ruby was built by ruby-install. That's kind of an important question in my mind, and I don't currently know how that works with ruby-install.

Ruby itself doesn't directly link to OpenSSL. It gets linked to via the openssl gem, which is a Ruby extension & also included and built as a default gem. By default, that extension links to whatever OpenSSL library you have installed when the gem was built. I can see it when I inspect the openssl.bundle with otool:

$ otool -L ~/.rubies/ruby-3.1.4/lib/ruby/3.1.0/arm64-darwin22/openssl.bundle 
/Users/danielholz/.rubies/ruby-3.1.4/lib/ruby/3.1.0/arm64-darwin22/openssl.bundle:
    /opt/homebrew/opt/openssl@1.1/lib/libssl.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
    /opt/homebrew/opt/openssl@1.1/lib/libcrypto.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)

If I update openssl@1.1 & Homebrew puts newer files in those locations, the openssl gem will load them when it's required. So it automatically uses updates. The gem won't switch from 1.1 to 3 without being recompiled, since the shared libraries will have different paths.

[chadhs]

Has anyone figured out a fix for this yet? Was just trying to install ruby with ruby-install on a new Mac and I'm getting the same compile error.

[skaes]

--with-opt-dir= is the wrong approach to select the openssl version. The openssl gem looks for pkg-config information first and ignores that only when --with-openssl-dir=... is specified. I know, because I contributed the patch: https://github.com/ruby/openssl/commit/b23fa75aa3a514e83e40d5220339f992f947befd.

Note that the option only works for ruby versions 3.1.1 and higher. Anything smaller can not be compiled without applying the patch first.

[skaes]

Version 3.1.1 and higher can be installed with a single command:

ruby-install ruby 3.2.2 --with-openssl-dir=/opt/homebrew/opt/openssl@1.1

[semaperepelitsa]

> ruby-install ruby 3.2.2 --with-openssl-dir=/opt/homebrew/opt/openssl@1.1
ruby-install: unrecognized option --with-openssl-dir=/opt/homebrew/opt/openssl@1.1

Should be:

ruby-install ruby 3.2.2 -- --with-openssl-dir=/opt/homebrew/opt/openssl@1.1

[chadhs]

thank you everyone! what about for versions 2.7.6 and under?