search

DeepLcom / deepl-rb

Official Ruby library for the DeepL language translation API.
https://www.deepl.com
MIT License
8 stars 2 forks source link

Deluge of errors when making concurrent translate requests #6

Open clemens1483 opened 1 month ago

clemens1483 commented 1 month ago

We have been using the gem on version 2.5.3 without any issues. However, we are now trying to update to the newest version, 3.0.2 and are facing quite a few issues

When making many ~10+ simultaneous translation requests via DeepL.translate from within sidekiq workers running in parallel a lot of the requests fail and raise an error with either

This was NOT an issue on 2.5.3

VERSION: 3.0.2 API CALL: `DeepL.translate("some string", nil, "FR", tag_handling: 'html')

2024-10-10T14:50:19.914Z pid=19199 tid=29nz WARN: IOError: stream closed in another thread
2024-10-10T14:50:19.914Z pid=19199 tid=29nz WARN: /gems/ruby/3.1.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:229:in `wait_readable'
/gems/ruby/3.1.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:229:in `rbuf_fill'
/gems/ruby/3.1.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:199:in `readuntil'
/gems/ruby/3.1.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:209:in `readline'
/usr/local/lib/ruby/3.1.0/net/http/response.rb:42:in `read_status_line'
/usr/local/lib/ruby/3.1.0/net/http/response.rb:31:in `read_new'
/usr/local/lib/ruby/3.1.0/net/http.rb:1609:in `block in transport_request'
/usr/local/lib/ruby/3.1.0/net/http.rb:1600:in `catch'
/usr/local/lib/ruby/3.1.0/net/http.rb:1600:in `transport_request'
/usr/local/lib/ruby/3.1.0/net/http.rb:1573:in `request'
/usr/local/lib/ruby/3.1.0/net/http.rb:1566:in `block in request'
/usr/local/lib/ruby/3.1.0/net/http.rb:985:in `start'
/usr/local/lib/ruby/3.1.0/net/http.rb:1564:in `request'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/base.rb:54:in `block in execute_request_with_retries'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/base.rb:53:in `loop'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/base.rb:53:in `execute_request_with_retries'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/translate.rb:44:in `request'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl.rb:77:in `translate'
2024-10-10T14:50:28.408Z pid=19199 tid=28kj WARN: FrozenError: can't modify frozen OpenSSL::SSL::SSLContext: #<OpenSSL::SSL::SSLContext:0x0000ffff848e46b0 @verify_mode=1, @verify_hostname=true, @max_proto_version=nil, @min_proto_version=769, @cert_store=#<OpenSSL::X509::Store:0x0000ffff8cb26ec0 @verify_callback=nil, @error=nil, @error_string=nil, @chain=nil, @time=nil>, @session_new_cb=#<Proc:0x0000ffff848e4318 /usr/local/lib/ruby/3.1.0/net/http.rb:1061>>
2024-10-10T14:50:28.408Z pid=19199 tid=28kj WARN: /gems/ruby/3.1.0/gems/openssl-3.2.0/lib/openssl/ssl.rb:148:in `options='
/gems/ruby/3.1.0/gems/openssl-3.2.0/lib/openssl/ssl.rb:148:in `set_params'
/usr/local/lib/ruby/3.1.0/net/http.rb:1054:in `connect'
/usr/local/lib/ruby/3.1.0/net/http.rb:995:in `do_start'
/usr/local/lib/ruby/3.1.0/net/http.rb:984:in `start'
/usr/local/lib/ruby/3.1.0/net/http.rb:1564:in `request'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/base.rb:54:in `block in execute_request_with_retries'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/base.rb:53:in `loop'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/base.rb:53:in `execute_request_with_retries'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/translate.rb:44:in `request'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl.rb:77:in `translate'
JanEbbing commented 1 month ago

Thanks for the report! There is some discussion on this in https://github.com/DeepLcom/deepl-rb/pull/3 . I don't think the keepalive support I added in 3.0 is possible without such errors with Net::HTTP which we currently use, so if you need to make concurrent requests I'm afraid you will need to keep using 2.5.3 for now - I will look into a HTTP library we can use that supports both.