Failure in SSL verification requires hack

[ronaldtse]

In packed-mn, we had to make this hack: https://github.com/metanorma/packed-mn/blob/c1904865ca7e5375ecb4acbf2e5e86fe682ebb9f/bin/metanorma#L178-L186

Net::HTTP.class_eval do
  alias _use_ssl= use_ssl=

  def use_ssl= boolean
    self.ca_file = cert_file_path
    self.verify_mode = OpenSSL::SSL::VERIFY_PEER
    self._use_ssl = boolean
  end
end

And supply our own CA file (from cURL which uses Mozilla's: https://curl.se/ca/cacert.pem).

This should not be necessary given that we already have this patch: https://github.com/metanorma/ruby-packer/blob/master/.patches/ruby/0004-ssl-certs.patch

Without this hack, we get these errors:

https://github.com/metanorma/packed-mn/runs/4454422319?check_suite_focus=true

2021-12-08T08:12:37.7764700Z Downloading font "source" from https://github.com/fontist/source-fonts/releases/download/v1.0/source-fonts-1.0.zip
2021-12-08T08:12:37.8281670Z Invalid URL: https://github.com/fontist/source-fonts/releases/download/v1.0/source-fonts-1.0.zip. Error: #<Down::SSLError: SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate)>.
2021-12-08T08:12:37.8318500Z /__ruby_packer_memfs__/local/vendor/bundle/ruby/2.6.0/gems/fontist-1.13.0/lib/fontist/font_installer.rb:68:in `download_file': Invalid URL: https://github.com/fontist/source-fonts/releases/download/v1.0/source-fonts-1.0.zip. Error: #<Down::SSLError: SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate)>. (Fontist::Errors::InvalidResourceError)
2021-12-08T08:12:37.8321240Z    from /__ruby_packer_memfs__/local/vendor/bundle/ruby/2.6.0/gems/fontist-1.13.0/lib/fontist/font_installer.rb:41:in `extract'

https://github.com/metanorma/packed-mn/runs/4454422376?check_suite_focus=true

2021-12-08T08:11:57.6276950Z #<Thread:0x00007ff4bf7a0650@/__ruby_packer_memfs__/local/vendor/bundle/ruby/2.6.0/gems/relaton-1.9.5/lib/relaton/workers_pool.rb:10 run> terminated with exception (report_on_exception is true):
2021-12-08T08:11:57.6285680Z /__ruby_packer_memfs__/lib/ruby/2.6.0/net/protocol.rb:44:in `connect_nonblock': SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate) (OpenSSL::SSL::SSLError)
2021-12-08T08:11:57.6288470Z    from /__ruby_packer_memfs__/lib/ruby/2.6.0/net/protocol.rb:44:in `ssl_socket_connect'
2021-12-08T08:11:57.6290150Z    from /__ruby_packer_memfs__/lib/ruby/2.6.0/net/http.rb:996:in `connect'
2021-12-08T08:11:57.6291650Z    from /__ruby_packer_memfs__/lib/ruby/2.6.0/net/http.rb:930:in `do_start'
2021-12-08T08:11:57.6292890Z    from /__ruby_packer_memfs__/lib/ruby/2.6.0/net/http.rb:919:in `start'
2021-12-08T08:11:57.6293980Z    from /__ruby_packer_memfs__/lib/ruby/2.6.0/net/http.rb:605:in `start'
2021-12-08T08:11:57.6295140Z    from /__ruby_packer_memfs__/lib/ruby/2.6.0/net/http.rb:481:in `get_response'
2021-12-08T08:11:57.6296600Z    from /__ruby_packer_memfs__/local/vendor/bundle/ruby/2.6.0/gems/relaton-ietf-1.9.2/lib/relaton_ietf/scrapper.rb:125:in `get_page'
2021-12-08T08:11:57.6298430Z    from /__ruby_packer_memfs__/local/vendor/bundle/ruby/2.6.0/gems/relaton-ietf-1.9.2/lib/relaton_ietf/scrapper.rb:111:in `rfc_item'

[alexeymorozov]

It occurred 0004-ssl-certs.patch contains only certs for the gem install to work.

Specifying the SSL_CERT_FILE environment variable for rubyc helps with the issue, so I see the following options:

1. Save the cert in a binary, find a place where SSL_CERT_FILE is used and provide a path there
2. Save the cert in a binary and implement the same hack (Net::HTTP.ca_file), but in ruby-packer

[ronaldtse]

It would certainly be better to use SSL_CERT_FILE to run this. Maybe when we start up the application we just set ENV['SSL_CERT_FILE'] to point to a file on the memfs?

[alexeymorozov]

Found a third solution: to provide --openssldir when compiling openssl, but seems it doesn't work.

Let's set the SSL_CERT_FILE variable in the main patch.

[alexeymorozov]

Though the openssldir option works, and it changes the OpenSSL::X509::DEFAULT_CERT_FILE constant properly, but it still didn't work.

The same with SSL_CERT_FILE. It doesn't work with paths from memfs. Seems that openssl calls to file system should be patched to fix this issue.

Finally it works when the cert is read from memfs with Ruby and provided for the openssl lib as a string. It is done by using OpenSSL::X509::Certificate.

So now calls with Net::HTTP work properly:

$ RUBY_PACKER_USE_ORIGINAL_RUBY=1 ./rubyc-linux-x64 -e 'require "net/http"; puts Net::HTTP.get(URI("https://github.com/fontist/source-fonts/releases/download/v1.0/source-fonts-1.0.zip"))'

[ronaldtse]

That's excellent -- @alexeymorozov if this code can be merged that would be great, and this issue can be closed. Thanks!