Support OpenSSL 3.0

[rhenium]

OpenSSL 3.0 is scheduled to be released later this year. It is a major version bump from 1.1 and contains architecture changes that affect Ruby/OpenSSL.

From https://www.openssl.org/policies/releasestrat.html:

The following alpha and beta releases for OpenSSL 3.0 are currently scheduled. Note that these dates are subject to change and alpha or beta releases may be inserted or removed as required:

alpha1, 2020-03-31: Basic functionality plus basic FIPS module
alpha2, 2020-04-21: Complete external provider support (serialization, support for new algs, support for providers which only include operations in a class)
alpha3, 2020-05-21: Aiming to test the API completeness before beta1 freezes it)
beta1, 2020-06-02: Code complete (API stable, feature freeze)
betaN: Other beta releases TBD
Final: 2020 early Q4

The design is outlined in the web page:

https://www.openssl.org/docs/OpenSSL300Design.html

Unlike OpenSSL 1.0 -> 1.1, not so many changes are required to make it just compile, but a lot of deprecation warnings are generated while compiling and many test cases are currently failing when compiled against OpenSSL's master.

• OpenSSL::HMAC needs a rewrite with the EVP API as it currently uses the low-level HMAC_*() functions.
• Subclasses of OpenSSL::PKey, such as RSA or DSA, provide access to those low-level functions. The following methods need rewrite:
  • PKey::*#generate (and an overload of .new)
    • This can probably be implemented in pure-Ruby with PKey.generate_parameters and PKey.generate_key.
  • Low-level sign/verify methods that take prehashed values.
    • RSA#{private,public}_{encrypt,decrypt}
    • DSA#syssign and #sysverify
    • EC#dsa_sign_asn1 and #dsa_verify_asn1
• The ENGINE API is deprecated in favor of "Provider"s.

[rhenium]

Updated TODO list as of alpha14:

• [x] OpenSSL::HMAC, Use the EVP API

  • No incompatibilities expected for Ruby programs.
  • Done by #371.

• [x] OpenSSL::PKey, Parameters/key generation with the EVP API

  • No incompatibilities expected.
  • Done by #397.
    • Added OpenSSL::PKey.generate_key and OpenSSL::PKey.generate_parameters.
    • Rewritten OpenSSL::PKey::{RSA,DSA,DH}.{new,generate}.

• [x] OpenSSL::PKey::*, Low-level methods for signature/encryption/key agreement

  • RSA#{private,public}_{encrypt,decrypt}, DSA#syssign, DSA#sysverify, EC#dsa_sign_asn1, and EC#dsa_verify_asn1.
  • No incompatibilities expected.
  • Done by #382.
    • Added OpenSSL::PKey.{encrypt,decrypt,sign_raw,verify_raw,verify_recover}.

• [ ] OpenSSL::PKey::*, Getters for parameters/key components

  • RSA#{n,e,d,p,...}, etc.
  • No incompatibilities expected.
  • Needs a rewrite with EVP_PKEY_get_params() family (only exists in 3.0)

• [x] OpenSSL::PKey::*, Setters for parameters/key components

  • {RSA,DSA,DH}#set_* and EC#{private_key=,public_key=,group=}
  • Feature removed without replacement. Keys are now immutable once created - all components must be specified at once.
  • Use cases definitely exist, a new interface is required.
    • EVP_PKEY_fromdata() requires the caller to specify what the pkey is: parameters only, public key only, or private key?
    • OpenSSL::PKey.new_private_key("RSA", n: 123, e: 456, d: 789)?

• [ ] OpenSSL::PKey::{RSA,DSA}, Private/public key decoders for unpopular/non-standard formats

  • OpenSSL::PKey::DSA.new accepts PEM encoded "DSAPublicKey" format ("DSA PUBLIC KEY" header).
  • OpenSSL::PKey::RSA.new accepts DER/PEM encoded "RSAPublicKey" format ("RSA PUBLIC KEY" header).
  • ruby-openssl has no ability to export keys in such formats as of [Bug #4422].
  • Would it be OK to drop them? We need to write our decoder using OpenSSL::ASN1.decode if we want to keep it.

• [x] OpenSSL::PKey::*, Methods that need a rewrite, which can be done with existing (<= 1.1.1) API

  • #to_text
  • #public_key (the method to create a copy with private components removed)
  • DH#params_ok? and EC#check_key
  • Done in #436.

• [ ] OpenSSL::PKey::*, Methods that need a rewrite, which can be done in a compatible way, but requires new functions in 3.0.0

  • #params
  • #initialize_copy - annoying because EVP_PKEY_dup() only exists in OpenSSL 3.0.
  • #private? and #public?

• [x] OpenSSL::PKey::{DH,EC}, #generate_key!

  • Feature removed without replacement. Keys are now immutable once created.
  • Print deprecation warning and suggest using OpenSSL::PKey.generate_key(pkey_obj_without_key_components).

• [ ] OpenSSL::PKey::EC::Group and OpenSSL::PKey::EC::Point

  • Effectively removed without replacement, though most of the functions are not explicitly marked as deprecated.
  • Because the low-level key generation is deprecated and the EVP API can't create keys using an EC_GROUP object, there is no use for this class.
  • Information that used to be obtained through EC_GROUP object (curve's name or parameters) are directly obtained from EVP_PKEY object with EVP_PKEY_get_params().

• [x] OpenSSL::Engine

  • The entire ENGINE API is deprecated in favor of Providers, but it's new and is a completely different concept.
  • ENGINE should work if we ignore the deprecation warnings, but I doubt if there will be any ENGINE implementations that work with OpenSSL 3.0 anyway.

• [ ] OpenSSL::SSL, tmp_dh_callback

  • The callback used on the server side for ephemeral DH. Called during the handshake, but there is no reason for it to be a callback (anymore).
  • Replaced by SSL_CTX_set_tmp_dh() or SSL_CTX_set_dh_auto().
  • Added SSLContext#tmp_dh= by #459.

[zzak]

This is awesome, thanks for your hard work @rhenium!

[pvalena]

I've tried to apply PR #399 on top of Ruby 3.0.1, and I'm experiencing a segfault on running test suite (failed when reading key). Any ideas how to fix that? Is that still expected?

I'll appreciate any help.

[MSP-Greg]

@rhenium

I just updated two items:

First, the below test is skipped for OpenSSL 3. I removed the 'pend', and it passes with the three Ruby builds using OpenSSL 3, Windows mswin, Ubuntu 22.04 3.1 & head.

https://github.com/ruby/openssl/blob/8752d9eb27dc41d845270b6351f736501ebe0273/test/openssl/test_hmac.rb#L23-L29

Secondly, I updated the Actions test-openssls jobs, 1.1.1l to 1.1.1q, and 3.0.1 to 3.0.5. They also passed. JFYI, 3.0.1 testing froze before I updated these.

Not sure if you want PR's, three lines of code, also added timeouts to the test steps.

Thanks for your work on this. A lot of non-trivial work...

[rhenium]

First, the below test is skipped for OpenSSL 3.

Yes, we can remove it now. The problem was apparently fixed by OpenSSL 3.0.2 (https://github.com/openssl/openssl/commit/9a4e7d863fa5a65b0efef96c1c4891864aac036f).

Secondly, I updated the Actions test-openssls jobs, 1.1.1l to 1.1.1q, and 3.0.1 to 3.0.5. They also passed. JFYI, 3.0.1 testing froze before I updated these.

Not sure if you want PR's, three lines of code, also added timeouts to the test steps.

Yes, please!

[MSP-Greg]

Yes, please!

Not sure if you wanted one or two, see PR's #528 & #529

[simi]

Hello, I have found out some difference in between 1 and 3 behaviour.

OpenSSL::PKey::EC.new('prime256v1').to_pem

raises can't export - no public key set (OpenSSL::PKey::ECError) with OpenSSL 1, but segfaults with OpenSSL 3 with following stacktrace (using GDB)

(gdb) bt
#0  EC_POINT_point2oct (group=0x555555e2cd40, point=0x0, form=POINT_CONVERSION_UNCOMPRESSED, buf=0x0, len=0, ctx=0x0) at crypto/ec/ec_oct.c:82
#1  0x00007fffe6b37a05 in i2o_ECPublicKey (a=a@entry=0x555555e3be20, out=out@entry=0x0) at crypto/ec/ec_asn1.c:1169
#2  0x00007fffe6b37bf6 in eckey_pub_encode (pk=0x555555e499b0, pkey=<optimized out>) at crypto/ec/ec_ameth.c:81
#3  0x00007fffe6c3768c in i2d_PUBKEY (a=0x555555e49780, pp=0x0) at crypto/x509/x_pubkey.c:560
#4  0x00007fffe6bcde7a in PEM_ASN1_write_bio (i2d=0x7fffe6c37630 <i2d_PUBKEY>, name=name@entry=0x7fffe6d4edf5 "PUBLIC KEY", bp=bp@entry=0x555555e44600, x=x@entry=0x555555e49780, 
    enc=enc@entry=0x0, kstr=kstr@entry=0x0, klen=0, callback=0x0, u=0x0) at crypto/pem/pem_lib.c:340
#5  0x00007fffe6bce631 in PEM_write_bio_PUBKEY (out=out@entry=0x555555e44600, x=x@entry=0x555555e49780) at crypto/pem/pem_all.c:226
#6  0x00007fffe6f691f0 in ossl_pkey_export_spki (self=<optimized out>, to_der=0) at ossl_pkey.c:780
#7  0x0000555555778f77 in vm_call_cfunc_with_frame (ec=0x5555559a48b0, reg_cfp=0x7ffff7dc2f90, calling=<optimized out>) at /home/retro/src/ruby-3.1.2/vm_insnhelper.c:3037
#8  0x000055555578b483 in vm_sendish (method_explorer=<optimized out>, block_handler=<optimized out>, cd=<optimized out>, reg_cfp=<optimized out>, ec=<optimized out>)
    at /home/retro/src/ruby-3.1.2/vm_callinfo.h:349
#9  vm_exec_core (ec=0x555555e2cd40, initial=0) at /home/retro/src/ruby-3.1.2/insns.def:778
#10 0x000055555577bb63 in rb_vm_exec (ec=0x5555559a48b0, mjit_enable_p=true) at vm.c:2211
#11 0x00005555555832e5 in rb_ec_exec_node (ec=ec@entry=0x5555559a48b0, n=n@entry=0x7fffe69f8d30) at eval.c:280
#12 0x00005555555884e8 in ruby_run_node (n=0x7fffe69f8d30) at eval.c:321
#13 0x0000555555582f7b in main (argc=<optimized out>, argv=<optimized out>) at ./main.c:47

[dbussink]

Found a regression in EC private key parsing and have submitted a fix in https://github.com/ruby/openssl/pull/535

[simi]

Another problem I have found out.

OpenSSL::HMAC.digest(OpenSSL::Digest.new('sha1'), '', 'A')

OpenSSL 1

irb(main):001:0> OpenSSL::HMAC.digest(OpenSSL::Digest.new('sha1'), '', 'A')
=> "\xA6\xFD\xA2\xA6\xAC\xDDtc\v \xAA\xC0\xC6\x87\x16\x04\x8E\xCD\x033"

OpenSSL 3

irb(main):050:0> OpenSSL::HMAC.digest(OpenSSL::Digest.new('sha1'), '', 'A')
/home/retro/.rubies/ruby-3.1.2/lib/ruby/3.1.0/openssl/hmac.rb:36:in `initialize': EVP_PKEY_new_mac_key: malloc failure (OpenSSL::HMACError)
        from /home/retro/.rubies/ruby-3.1.2/lib/ruby/3.1.0/openssl/hmac.rb:36:in `new'
        from /home/retro/.rubies/ruby-3.1.2/lib/ruby/3.1.0/openssl/hmac.rb:36:in `digest'
        from /home/retro/code/work/oss/rubygems.org/app/models/concerns/user_multifactor_methods.rb:50:in `otp_verified?'
        from /home/retro/code/work/oss/rubygems.org/app/models/concerns/user_multifactor_methods.rb:58:in `otp_verified?'
        from /home/retro/code/work/oss/rubygems.org/test/unit/user_test.rb:363:in `block (4 levels) in <class:UserTest>'
        from /home/retro/code/work/oss/rubygems.org/test/unit/user_test.rb:370:in `instance_exec'
        from /home/retro/code/work/oss/rubygems.org/test/unit/user_test.rb:370:in `block in create_test_from_should_hash'
        from /home/retro/.gem/ruby/3.1.2/gems/minitest-5.16.3/lib/minitest/test.rb:98:in `block (3 levels) in run'
        from /home/retro/.gem/ruby/3.1.2/gems/minitest-5.16.3/lib/minitest/test.rb:195:in `capture_exceptions'
        from /home/retro/.gem/ruby/3.1.2/gems/minitest-5.16.3/lib/minitest/test.rb:95:in `block (2 levels) in run'
        from /home/retro/.gem/ruby/3.1.2/gems/minitest-5.16.3/lib/minitest.rb:296:in `time_it'
        from /home/retro/.gem/ruby/3.1.2/gems/minitest-5.16.3/lib/minitest/test.rb:94:in `block in run'
        from /home/retro/.gem/ruby/3.1.2/gems/minitest-5.16.3/lib/minitest.rb:391:in `on_signal'
        from /home/retro/.gem/ruby/3.1.2/gems/minitest-5.16.3/lib/minitest/test.rb:243:in `with_info_handler'
        from /home/retro/.gem/ruby/3.1.2/gems/minitest-5.16.3/lib/minitest/test.rb:93:in `run'
        from /home/retro/.gem/ruby/3.1.2/gems/minitest-5.16.3/lib/minitest.rb:1059:in `run_one_method'

[simi]

Btw. the problem mentioned above ^ breaks the https://github.com/mdp/rotp gem. Feel free to ping me if that's expected behaviour and I can try to fix it in the gem itself.

[rhenium]

OpenSSL::PKey::EC.new('prime256v1').to_pem

@no6v also reported this at https://github.com/ruby/openssl/pull/527#issuecomment-1220504524. This is actually an older bug unrelated to OpenSSL 3.0: it will crash with OpenSSL 1.1 too.

[rhenium]

OpenSSL::HMAC.digest(OpenSSL::Digest.new('sha1'), '', 'A')

This appears to be a bug in the current OpenSSL 3.0.x releases. EVP_PKEY_new_mac_key() is incorrectly handling zero-length key.

ruby/openssl actually should use EVP_PKEY_new_raw_private_key() in place of the now-deprecated EVP_PKEY_new_mac_key() and it doesn't seem the issue.

Regardless, I will also report it upstream.

[ariccio]

FYI: I think everybody might benefit from some better docs on how to replace code that uses OpenSSL::PKey::, Setters for parameters/key components. It's apparently a problem in several software packages that many of us depend on, and the fix is not entirely* clear. As a non-cryptographer, I greatly hesitate to touch any cryptography handling code, as doing so is the cause of many a security nightmare.

Examples of places where people have hit this problem: https://github.com/googleapis/google-auth-library-ruby/issues/381 https://github.com/net-ssh/net-ssh/issues/657 https://github.com/nov/json-jwt/issues/100 https://www.reddit.com/r/metasploit/comments/w2no56/metasploit_just_updated_to_version_626_but_it_is/ & https://github.com/rapid7/metasploit-framework/issues/16767

[akostadinov]

Can the fix also be backported to Ruby 3.0.x because I can't compile it on latest Fedora?

UPDATE: eventually I used this approach https://gist.github.com/yob/08d53a003181aa0fcce9812b1b533870?permalink_comment_id=4334396#gistcomment-4334396

[eregon]

@akostadinov That's not the right place to ask that. It's been asked and the answer is basically no. You can use ruby-build to install Ruby <=3.0 easily on system with a libssl 3.

[pvalena]

Can the fix also be backported to Ruby 3.0.x because I can't compile it on latest Fedora?

Or simply install Ruby using DNF: https://developer.fedoraproject.org/tech/languages/ruby/ruby-installation.html

[ojab]

Not sure if it's a known issue, but

require 'openssl'

ec_key = OpenSSL::PKey::EC.generate('prime256v1')
cipher = OpenSSL::Cipher.new('aes-128-cbc-hmac-sha256')
ec_key.export(cipher, 'key_password')

leads to

$ ruby /tmp/2.rb 
/tmp/2.rb:5:in `export': PEM_write_bio_PrivateKey_traditional: cipher operation failed (OpenSSL::PKey::PKeyError)
    from /tmp/2.rb:5:in `<main>'

on Fedora-37 (openssl-3.0.5) and openssl gem from master (1ddbf28dcedf4a9aae118b738ebdde236f44e951)

[akostadinov]

@ojab , shouldn't you be using an EC algorithm when using an EC key?

[ojab]

@akostadinov Nope, aes-128-cbc-hmac-sha256 is encrypting generic string (it's EC key in the example above, just an extract from the existing code). Could be as well:

require 'openssl'

data = "Very, very confidential data"
cipher = OpenSSL::Cipher.new('aes-128-cbc-hmac-sha256')
cipher.encrypt

key = cipher.random_key
iv = cipher.random_iv

encrypted = cipher.update(data)
encrypted += cipher.final
p encrypted

which also fails with cipher operation failed on cipher#update

[ojab]

But that's a good point, because

require 'openssl'

ec_key = OpenSSL::PKey::EC.generate('prime256v1')
cipher = OpenSSL::Cipher.new('aes-256-gcm')
pem = ec_key.export(cipher, 'key_password')

p OpenSSL::PKey.read(pem, 'key_password')

fails with

/tmp/3.rb:7:in `read': Could not parse PKey: bad decrypt (OpenSSL::PKey::PKeyError)
    from /tmp/3.rb:7:in `<main>'

Maybe it's covered by unchecked points above, but dunno.

EDIT: okay, gcm is hard and I can't get it working even with openssl-1.

[collimarco]

Do you have any suggestions to update the Ruby web-push library to use OpenSSL v3?

I can replace OpenSSL::PKey::EC.new with OpenSSL::PKey::EC.generate, but then I find it hard to update this file to v3:

https://github.com/pushpad/web-push/blob/master/lib/web_push/vapid_key.rb

The gem has good test coverage and is already updated to use the openssl gem v3, but tests pass only on with the C library v1.1 (and not with C library v3).

Any suggestions?

[collimarco]

# input: the base64 string of the public key and the base64 string of private key
# output: an OpenSSL::PKey::EC with that keys

public_key = OpenSSL::PKey::EC::Point.new(group, to_big_num(public_key_base64))
private_key = to_big_num(private_key_base64)

# ... then how do you generate the OpenSSL::PKey::EC from that (in OpenSSL 3)?

This is the only workaround that we have found: https://github.com/pushpad/web-push/pull/2/files#diff-e8e36e8a5282b2fbf5503e699e222f3d5413c86cdbb94ecdd13148ea0c59e5f5R90

Which is quite complex compared to the straightforward assignment that you could do with OpenSSL 1.1.

Is there a simpler alternative to create an OpenSSL::PKey::EC from existing keys (in base64) in OpenSSL 3?

[collimarco]

OpenSSL::PKey::, Setters for parameters/key components {RSA,DSA,DH}#set_ and EC#{private_key=,public_key=,group=} Feature removed without replacement. Keys are now immutable once created - all components must be specified at once.

Would it make sense to replace these methods with a new method like set_keys or new_from_keys that assigns both keys at the same time (since they are immutable)? Would you consider adding it to Ruby? Or what is the new recommended way for assigning the existing keys?

[rhenium]

Would it make sense to replace these methods with a new method like set_keys or new_from_keys that assigns both keys at the same time (since they are immutable)?

555 is working on it, which provides access to EVP_PKEY_fromdata() added in OpenSSL 3.0. It currently (as of openssl gem v3.1) has to be done by making the ASN.1 encoding as in this PR:

This is the only workaround that we have found: https://github.com/pushpad/web-push/pull/2/files#diff-e8e36e8a5282b2fbf5503e699e222f3d5413c86cdbb94ecdd13148ea0c59e5f5R90

[junaruga]

As this is related to supporting OpenSSL 3, I just put the issue tickets and PR to support FIPS mode on OpenSSL 3

• https://github.com/ruby/openssl/issues/605, PR
• https://github.com/ruby/openssl/issues/603

[mlarraz]

FYI OpenSSL 1.1.1 (the last version before 3.0) is set to be EOL in less than 2 months.

I first want to say I appreciate all the work that has gone into adding 3.0 support so far (as well as all the other work on this gem).

From my limited perspective, it looks like full support is unlikely to be completed before the EOL deadline. Is this a reasonable assumption? If so, it seems like something many users will want to know ahead of time.

[hlein]

Although the occasional OpenSSL-3.x issue is still spotted and fixed, according to https://github.com/ruby/openssl/blob/master/History.md OpenSSL 3.0 has been supported since v3.0.0 in Dec 24, 2021.

Meanwhile projects that consume this library are looking at this issue still being open as a reason they can't support OpenSSL 3.x yet. Can this issue be marked closed?

[thomthom]

We also got confused by this open issue and the incomplete 3.0 milestone: https://github.com/ruby/openssl/milestone/2 We didn't move to OpenSSL 3 because we got the impression it wasn't ready for production yet.

[eregon]

Agreed, @rhenium are you OK to close this issue and the milestone? And if there any remaining TODO for OpenSSL 3.0 then I would suggest to file a new issue with just that.