search

savonet / ocaml-ssl

OCaml SSL bindings.
http://liquidsoap.info/ocaml-ssl/
Other
57 stars 48 forks source link

Add support for openssl@3.3 on macOS #153

Closed smorimoto closed 7 months ago

smorimoto commented 7 months ago

The current codebase only supports up to openssl@3.0, but currently the default installation of openssl with brew is @3.3, which seems to break some builds.

oemmerson commented 7 months ago

I'm experiencing this issue also.

An external web request in my Dream app fails. This is the relevant section of the log:

...
...
30.04.24 02:23:14.889    dream.logger  WARN REQ 22 Aborted by: SSL connection() error: error:14:000086:SSL routines::certificate verify failed
30.04.24 02:23:14.889    dream.logger  WARN Raised by primitive operation at Ssl in file "src/ssl.ml", line 785, characters 14-33
30.04.24 02:23:14.889    dream.logger  WARN Called from Lwt_ssl.wrap_call in file "src/lwt_ssl.ml", line 47, characters 4-8
30.04.24 02:23:14.889    dream.logger  WARN Re-raised at Lwt_ssl.wrap_call in file "src/lwt_ssl.ml", line 57, characters 7-14
30.04.24 02:23:14.889    dream.logger  WARN Called from Lwt_unix.retry_syscall in file "src/unix/lwt_unix.cppo.ml", line 509, characters 13-24
...
...

Installing OpenSSL 3.0 from Homebrew and linking to that makes the request work.

anmonteiro commented 7 months ago

What are the build errors?

smorimoto commented 7 months ago

https://github.com/ocsigen/eliom/actions/runs/8880329780/job/24380120879

anmonteiro commented 7 months ago

I'm packaging OpenSSL 3.3 for Nix here: https://github.com/NixOS/nixpkgs/pull/308082

I don't see any failures there, which leads me to believe this could be a bug in the discover.ml script just for homebrew?

anmonteiro commented 7 months ago

I don't have a Homebrew installation, so I'll need a little help here. The current discover.ml script looks for a set of hardcoded paths on macOS. Could you check if these are still present for OpenSSL 3.3, or whether they have been changed for this release?

https://github.com/savonet/ocaml-ssl/blob/05a3e8e241d17ec04eabe15b20812745915f0bd6/src/config/discover.ml#L6-L13

anmonteiro commented 7 months ago

Alternatively, could this be a homebrew packaging issue? e..g https://github.com/Homebrew/homebrew-core/pull/169721 https://github.com/Homebrew/homebrew-core/issues/169728

I'd be interested in checking whether builds are still failing after brew update?

anmonteiro commented 7 months ago

https://github.com/ocsigen/eliom/actions/runs/8880329780/job/24380120879

@smorimoto could you point me to where OpenSSL 3.3 is being used here? The build looks like it's installing OpenSSL 3.0

smorimoto commented 7 months ago

As far as I can see here: https://github.com/ocsigen/eliom/actions/runs/8880329780/job/24380120879#step:5:390 The path is here: /opt/homebrew/Cellar/openssl@3/3.3.0

anmonteiro commented 7 months ago

Do you know if multiple version of the package can coexist? because the step "Reinstall OpenSSL on macOS" is installing 3.0

image
cuihtlauac commented 7 months ago

https://github.com/ocsigen/eliom/actions/runs/8880329780/job/24380120879

We are seeing the same issues when CI building ocaml.org (Issue: https://github.com/ocaml/ocaml.org/issues/2395). Here is the relevant part of the log:

#=== ERROR while compiling ssl.0.7.0 ==========================================#
# context     2.1.5 | macos/arm64 | ocaml-base-compiler.4.14.1 | git+https://github.com/ocaml/opam-repository#8cc107f96e33a4601f7c39346eb19fbbe46486d3
# path        ~/work/ocaml.org/ocaml.org/_opam/.opam-switch/build/ssl.0.7.0
# command     ~/work/ocaml.org/ocaml.org/_opam/bin/dune build -p ssl -j 3 @install
# exit-code   1
# env-file    ~/.opam/log/ssl-7020-69c5ca.env
# output-file ~/.opam/log/ssl-7020-69c5ca.out
### output ###
# (cd _build/default/src && /usr/bin/cc -O2 -fno-strict-aliasing -fwrapv -pthread -D_FILE_OFFSET_BITS=64 -I/opt/homebrew/Cellar/openssl@3/3.3.0/include -g -I /Users/runner/work/ocaml.org/ocaml.org/_opam/lib/ocaml -o ssl_stubs.o -c ssl_stubs.c)
# ssl_stubs.c:1142:5: warning: 'DH_free' is deprecated [-Wdeprecated-declarations]
#     DH_free(dh);
#     ^
# /opt/homebrew/Cellar/openssl@3/3.3.0/include/openssl/dh.h:211:1: note: 'DH_free' has been explicitly marked deprecated here
# OSSL_DEPRECATEDIN_3_0 void DH_free(DH *dh);
# ^
# /opt/homebrew/Cellar/openssl@3/3.3.0/include/openssl/macros.h:194:49: note: expanded from macro 'OSSL_DEPRECATEDIN_3_0'
# #   define OSSL_DEPRECATEDIN_3_0                OSSL_DEPRECATED(3.0)
#                                                 ^
# /opt/homebrew/Cellar/openssl@3/3.3.0/include/openssl/macros.h:62:52: note: expanded from macro 'OSSL_DEPRECATED'
# #     define OSSL_DEPRECATED(since) __attribute__((deprecated))
#                                                    ^
# ssl_stubs.c:1168:10: warning: 'EC_KEY_new_by_curve_name' is deprecated [-Wdeprecated-declarations]
#   ecdh = EC_KEY_new_by_curve_name(nid);
#          ^
# /opt/homebrew/Cellar/openssl@3/3.3.0/include/openssl/ec.h:1017:1: note: 'EC_KEY_new_by_curve_name' has been explicitly marked deprecated here
# OSSL_DEPRECATEDIN_3_0 EC_KEY *EC_KEY_new_by_curve_name(int nid);
# ^
# /opt/homebrew/Cellar/openssl@3/3.3.0/include/openssl/macros.h:194:49: note: expanded from macro 'OSSL_DEPRECATEDIN_3_0'
# #   define OSSL_DEPRECATEDIN_3_0                OSSL_DEPRECATED(3.0)
#                                                 ^
# /opt/homebrew/Cellar/openssl@3/3.3.0/include/openssl/macros.h:62:52: note: expanded from macro 'OSSL_DEPRECATED'
# #     define OSSL_DEPRECATED(since) __attribute__((deprecated))
#                                                    ^
# ssl_stubs.c:1176:5: warning: 'EC_KEY_free' is deprecated [-Wdeprecated-declarations]
#     EC_KEY_free(ecdh);
#     ^
# /opt/homebrew/Cellar/openssl@3/3.3.0/include/openssl/ec.h:1022:1: note: 'EC_KEY_free' has been explicitly marked deprecated here
# OSSL_DEPRECATEDIN_3_0 void EC_KEY_free(EC_KEY *key);
# ^
# /opt/homebrew/Cellar/openssl@3/3.3.0/include/openssl/macros.h:194:49: note: expanded from macro 'OSSL_DEPRECATEDIN_3_0'
# #   define OSSL_DEPRECATEDIN_3_0                OSSL_DEPRECATED(3.0)
#                                                 ^
# /opt/homebrew/Cellar/openssl@3/3.3.0/include/openssl/macros.h:62:52: note: expanded from macro 'OSSL_DEPRECATED'
# #     define OSSL_DEPRECATED(since) __attribute__((deprecated))
#                                                    ^
# ssl_stubs.c:2031:9: warning: 'PEM_read_bio_DHparams' is deprecated [-Wdeprecated-declarations]
#   ret = PEM_read_bio_DHparams(bio, NULL, NULL, NULL);
#         ^
# /opt/homebrew/Cellar/openssl@3/3.3.0/include/openssl/pem.h:473:21: note: 'PEM_read_bio_DHparams' has been explicitly marked deprecated here
# DECLARE_PEM_rw_attr(OSSL_DEPRECATEDIN_3_0, DHparams, DH)
#                     ^
# /opt/homebrew/Cellar/openssl@3/3.3.0/include/openssl/macros.h:194:49: note: expanded from macro 'OSSL_DEPRECATEDIN_3_0'
# #   define OSSL_DEPRECATEDIN_3_0                OSSL_DEPRECATED(3.0)
#                                                 ^
# /opt/homebrew/Cellar/openssl@3/3.3.0/include/openssl/macros.h:62:52: note: expanded from macro 'OSSL_DEPRECATED'
# #     define OSSL_DEPRECATED(since) __attribute__((deprecated))
#                                                    ^
# 4 warnings generated.
# File "src/dune", line 1, characters 0-259:
#  1 | (library
#  2 |  (name ssl)
#  3 |  (public_name ssl)
# ....
# 12 |   (:include c_library_flags.sexp))
# 13 |  (instrumentation
# 14 |   (backend bisect_ppx)))
# (cd _build/default && /Users/runner/work/ocaml.org/ocaml.org/_opam/bin/ocamlmklib.opt -g -o src/ssl_stubs src/ssl_stubs.o -ldopt -L/opt/homebrew/Cellar/openssl@3/3.3.0 -ldopt -lssl -ldopt -lcrypto)
# ld: warning: -undefined suppress is deprecated
# ld: warning: -undefined suppress is deprecated
# ld: library 'ssl' not found
# clang: error: linker command failed with exit code 1 (use -v to see invocation)

<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫 
┌─ The following actions failed
│ λ build ssl 0.7.0
└─
anmonteiro commented 7 months ago

I’m not a home brew user so I’d appreciate some help figuring this out.

Would someone seeing this issue try to build ssl locally with OpenSSL 3.3 and paste the contents of the .sexp files with flags?

tonyfettes commented 7 months ago

c_library_flags.sexp:

("-L/opt/homebrew/Cellar/openssl@3/3.3.0" "-lssl" "-lcrypto")

c_flags.sexp;

("-I/opt/homebrew/Cellar/openssl@3/3.3.0/include")

I tried to lookup the directory, and it seems the correct directory is located at /opt/homebrew/Cellar/openssl@3/3.3.0/lib on my machine.

So as a temporary measurement to install the ssl spam package I'm using

export LIBRARY_PATH=$LIBRARY_PATH:/opt/homebrew/Cellar/openssl@3/3.3.0/lib

Update: The issue disappeared after I reinstalled openssl@3.3.0 using brew.

anmonteiro commented 7 months ago

Isn’t that https://github.com/Homebrew/homebrew-core/issues/169728 then, as I pasted above?

tonyfettes commented 7 months ago

Oh sorry, I missed that. Looks pretty much exactly the issue.

anmonteiro commented 7 months ago

or, I guess that issue was for libcrypto, but we need a similar one for libssl?

smorimoto commented 7 months ago

Adding the following step certainly fixed the issue:

- name: Re-install OpenSSL on macOS
  if: runner.os == 'macOS'
  run: brew update && brew reinstall openssl@3
anmonteiro commented 7 months ago

Thanks for confirming that it was a homebrew issue indeed.