jruby / jruby-openssl

JRuby's OpenSSL gem
http://www.jruby.org
Other
45 stars 79 forks source link

OpenSSL::Buffering#read_nonblock(..., exception: false) can still throw exceptions #271

Open fxposter opened 1 year ago

fxposter commented 1 year ago

Hello. I am not sure whether it's a jruby-openssl's or jruby's bug, but opening the issue here for now. Also, I don't have a way to reproduce, but it does reproduce with jruby-openssl 0.12.2 and 0.14.0 on jruby 9.2.21.0 and 9.3.9.0.

We're using httprb library, which has the following code (https://github.com/httprb/http/blob/main/lib/http/timeout/per_operation.rb#L41):

result = @socket.read_nonblock(size, buffer, :exception => false)

Which in my case leads to OpenSSL::Buffering#read_nonblock method. As you can see, it asks the method not to return the exception, but sometimes we get an exception from this method with class "OpenSSL::SSL::SSLErrorWaitReadable" and message "read would block" with this backtrace:

org/jruby/ext/openssl/SSLSocket.java:883:in `sysread_nonblock'
/opt/jruby/lib/ruby/stdlib/openssl/buffering.rb:205:in `read_nonblock'
/usr/local/bundle/gems/http-4.4.1/lib/http/timeout/per_operation.rb:63:in `block in readpartial'
org/jruby/RubyKernel.java:1442:in `loop'
/usr/local/bundle/gems/http-4.4.1/lib/http/timeout/per_operation.rb:62:in `readpartial'

As I said - I don't have neither a minimal code that reproduces the bug, nor an understanding on when it happens. For us it happened when we tried to connect to AWS-managed kubernetes apiserver (which means I can't get any info about what is happening there), but I assume that it should be pretty easy to understand where the problem lies for the person familiar with jruby-openssl and jruby itself.

cc @headius

fxposter commented 1 year ago

for now the "fix" for us is to monkey-patch the HTTP::Timeout::PerOperation#readpartial method, replacing the line mentioned above with this:

begin
  result = @socket.read_nonblock(size, buffer, :exception => false)
rescue => e
  if e.class == OpenSSL::SSL::SSLErrorWaitReadable && e.message == "read would block"
    result = :wait_readable
  else
    raise e
  end
end
fxposter commented 1 year ago

to be fair - it does look like that it only reproduces with this java version:

openjdk version "1.8.0_352"
OpenJDK Runtime Environment (Temurin)(build 1.8.0_352-b08)
OpenJDK 64-Bit Server VM (Temurin)(build 25.352-b08, mixed mode)

And doesn't reproduce on

openjdk version "1.8.0_345"
OpenJDK Runtime Environment (Temurin)(build 1.8.0_345-b01)
OpenJDK 64-Bit Server VM (Temurin)(build 25.345-b01, mixed mode)

at least I have never been able to catch see this "read would block" error on the latter java (ie: I have built docker images with my app with FROM jruby:9.2-jdk in Dockerfile, and previously it was 8u345 and now it seems to be 8u352, and on 352 the problematic behaviour does happen in production).

fxposter commented 1 year ago

confirmed that with the same jruby version and the same code, but with different java versions - the issue is present in 8u352 b08 and absent in 8u345 b01...

fxposter commented 1 year ago

@headius maybe you know how this can be solved? We would really like to update to Java 11, but with java 11.0.17 we get "read would block" errors, where ":exception => false" is set for socket.read_nonblock :(

kares commented 1 day ago

optimistically calling this fixed with 0.15.0 (which is part of JRuby 9.4.8.0)

fxposter commented 14 hours ago

Just checked - we still get "read would block" errors with JRuby 9.4.8.0.