net_http_args nil class exception

ncs1 commented 1 year ago

Hi,

Using (latest versions):

    parallel (1.22.1)
    connection_pool (2.4.0)
    net-http-persistent (4.0.1)

Using simple:

Parallel.each([], in_processes: 10) do |it|
  # ...
end

results in:

net-http-persistent-4.0.1/lib/net/http/persistent/pool.rb:14:in `checkin': undefined method `[]' for nil:NilClass (NoMethodError)
stack = Thread.current[@key][net_http_args] ||= []

I think it traces back to commit 428c06f on connection_pool repository

Opened the issue concurrently on parallel gem too, but from reviewing (shallowly) parallel code it may be more appropriate here.

Stack trace:

__RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/net-http-persistent-4.0.1/lib/net/http/persistent/pool.rb:14:in `checkin': undefined method `[]' for nil:NilClass (NoMethodError)                                                                                                                                                                                

    stack = Thread.current[@key][net_http_args] ||= []                                                                                                                                 
                                ^^^^^^^^^^^^^^^                                                                                                                                        
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:56:in `block in after_fork'                                     
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:52:in `each'                                                    
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:52:in `after_fork'                                              
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.0/lib/connection_pool.rb:72:in `_fork'                                                   
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/parallel-1.22.1/lib/parallel.rb:528:in `fork'                                                                
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/parallel-1.22.1/lib/parallel.rb:528:in `worker'
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/parallel-1.22.1/lib/parallel.rb:519:in `block in create_workers'                                             
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/parallel-1.22.1/lib/parallel.rb:518:in `each'                                                                        
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/parallel-1.22.1/lib/parallel.rb:518:in `each_with_index'                                                     
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/parallel-1.22.1/lib/parallel.rb:518:in `create_workers'                                                      
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/parallel-1.22.1/lib/parallel.rb:457:in `work_in_processes'                                                   
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/parallel-1.22.1/lib/parallel.rb:294:in `map'                                                                 
        from __RBPATH__/3.2.1/lib/ruby/gems/3.2.0/gems/parallel-1.22.1/lib/parallel.rb:238:in `each'

Thank you very much.

mperham commented 1 year ago

@casperisfine

casperisfine commented 1 year ago

I'm able to repro with:

begin
  require "bundler/inline"
rescue LoadError => e
  $stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler"
  raise e
end

gemfile(true) do
  source "https://rubygems.org"
  gem "net-http-persistent"
end

require "net/http/persistent"
http = Net::HTTP::Persistent.new name: 'my_app_name'
p http.request(URI("https://example.com"))
pid = fork do
  p http.request(URI("https://example.com"))

end
Process.wait(pid)

I'll dig a bit more, but at first sight I'd say the problem is on net-http-persistent side (they inherit from ConnectionPool).

casperisfine commented 1 year ago

Fix is here: https://github.com/drbrain/net-http-persistent/pull/144

ncs1 commented 1 year ago

@casperisfine thank you very much ! Much appreciated.

mperham / connection_pool

net_http_args nil class exception #172