Fix openssl3 unsafe legacy renegotiation disabled error

[adfoster-r7]

Relates to: https://github.com/rapid7/metasploit-framework/issues/16954

This PR fixes the following error in Metasploit-framework when the host machine has OpenSSL 3:

msf6 auxiliary(scanner/http/title) > run https://www.courts.ie/

[*] Error: 137.191.225.100: OpenSSL::SSL::SSLError SSL_connect returned=1 errno=0 peeraddr=137.191.225.100:443 state=error: unsafe legacy renegotiation disabled
[*] Scanned 1 of 1 hosts (100% complete)
[*] Auxiliary module execution completed

OpenSSL 1.x.x

Default SSLContext flags - used for Net HTTP

2.7.6 :027 > flags = OpenSSL::SSL::SSLContext::DEFAULT_PARAMS[:options]; pp OpenSSL::SSL.constants.select { |constant| constant.to_s.start_with?('OP_') }.map { |c| [c, flags & OpenSSL::SSL.const_get(c) > 0] }.select(&:last); nil
[[:OP_ALL, true],
 [:OP_LEGACY_SERVER_CONNECT, true],
 [:OP_TLSEXT_PADDING, true],
 [:OP_SAFARI_ECDHE_ECDSA_BUG, true],
 [:OP_NO_COMPRESSION, true],
 [:OP_CRYPTOPRO_TLSEXT_BUG, true]]
 => nil 

OP_ALL - used internally by rex-socket

3.0.2 :007 > flags = OpenSSL::SSL::OP_ALL; pp OpenSSL::SSL.constants.select { |constant| constant.to_s.start_with?('OP_') }.map { |c| [c, flags & OpenSSL::SSL.const_get(c) > 0] }.select(&:last); nil
[[:OP_DONT_INSERT_EMPTY_FRAGMENTS, true],
 [:OP_CRYPTOPRO_TLSEXT_BUG, true],                                                                                                                                                                     
 [:OP_ALL, true],
 [:OP_LEGACY_SERVER_CONNECT, true],
 [:OP_TLSEXT_PADDING, true],
 [:OP_SAFARI_ECDHE_ECDSA_BUG, true]]

OpenSSL 3

Default SSLContext flags - used for Net HTTP - missing OP_LEGACY_SERVER_CONNECT

>> flags = OpenSSL::SSL::SSLContext::DEFAULT_PARAMS[:options]; pp OpenSSL::SSL.constants.select { |constant| constant.to_s.start_with?('OP_') }.map { |c| [c, flags & OpenSSL::SSL.const_get(c) > 0] }.select(&:last); nil
[[:OP_ALL, true],
 [:OP_TLSEXT_PADDING, true],                                                                                                                                                                                               
 [:OP_SAFARI_ECDHE_ECDSA_BUG, true],
 [:OP_NO_COMPRESSION, true],
 [:OP_CRYPTOPRO_TLSEXT_BUG, true],
 [:OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION, true]]

OP_ALL - used internally by rex-socket

irb(main):003:0> flags = OpenSSL::SSL::OP_ALL; pp OpenSSL::SSL.constants.select { |constant| constant.to_s.start_with?('OP_') }.map { |c| [c, flags & OpenSSL::SSL.const_get(c) > 0] }.select(&:last); nil
[[:OP_TLSEXT_PADDING, true],
 [:OP_SAFARI_ECDHE_ECDSA_BUG, true],                                                                                                                                                                       
 [:OP_DONT_INSERT_EMPTY_FRAGMENTS, true],                                                 
 [:OP_CRYPTOPRO_TLSEXT_BUG, true],                                                        
 [:OP_ALL, true]]                                                                         
=> nil

Without this change rex-socket connections break:

It's also possible to set OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION - which is technically what the error suggests to configure

Raw example:

require 'socket'
require 'net/http'
require 'uri'

# Flag which was removed from OP_ALL in OpenSSL 3
# OpenSSL::SSL::SSLContext::DEFAULT_PARAMS[:options] |= OpenSSL::SSL::OP_LEGACY_SERVER_CONNECT
# Alternatively - enabling the flag which appears in the error
# OpenSSL::SSL::SSLContext::DEFAULT_PARAMS[:options] |= OpenSSL::SSL::OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION

puts Net::HTTP.get(URI('https://www.courts.ie/'))

Without changes the following error occurs:

/var/lib/gems/3.0.0/gems/net-protocol-0.1.3/lib/net/protocol.rb:46:in `connect_nonblock': SSL_connect returned=1 errno=0 peeraddr=137.191.225.100:443 state=error: unsafe legacy renegotiation disabled (OpenSSL::SSL::SSLError)
        from /var/lib/gems/3.0.0/gems/net-protocol-0.1.3/lib/net/protocol.rb:46:in `ssl_socket_connect'
        from /usr/lib/ruby/3.0.0/net/http.rb:1038:in `connect'
        from /usr/lib/ruby/3.0.0/net/http.rb:970:in `do_start'
        from /usr/lib/ruby/3.0.0/net/http.rb:959:in `start'
        from /usr/lib/ruby/3.0.0/net/http.rb:621:in `start'
        from /usr/lib/ruby/3.0.0/net/http.rb:496:in `get_response'
        from /usr/lib/ruby/3.0.0/net/http.rb:467:in `get'
        from foo.rb:6:in `<main>'

[jmartin-tech]

This looks reasonable as long as this constant is available in all the project's supported Ruby versions.

[adfoster-r7]

Looks like it was introduced 5 years ago - https://github.com/ruby/openssl/commit/b44ab7f7e7e1c5f0cf618a347579d090b390103d#diff-09f822c26289f5347111795ca22ed7ed1cfadd6ebd28f987991d1d414eef565aR2674

And is pretty supported :+1:

$ docker run -it --rm -w $(pwd) -v $(pwd):$(pwd) ruby:2.5-alpine /bin/sh -c 'ruby -r openssl -e "puts OpenSSL::SSL::OP_LEGACY_SERVER_CONNECT.inspect"'
4