search

drbrain / net-http-persistent

Thread-safe persistent connections with Net::HTTP
http://seattlerb.rubyforge.org/net-http-persistent
339 stars 117 forks source link

Leaking Proc instances (memory leak) #96

Closed brauliobo closed 5 years ago

brauliobo commented 6 years ago

After finishing all SQS messages, my daemon which uses mechanize + net-http-persistent have a much higher number of Proc instances, and most of them created by net-http-persistent, see below

~/P/o/Review-collector ❯❯❯ bundle exec rbtrace -p 9743 -e 'ObjectSpace.each_object.with_object(Hash.new 0){ |x,h| h[x.class] += 1 }.to_a.sort_by{ |x| -x[1] }.first(30)'           ⏎master ◼
*** run `sudo sysctl kernel.msgmnb=1048576` to prevent losing events (currently: 16384 bytes)
*** attached to process 9743
>> ObjectSpace.each_object.with_object(Hash.new 0){ |x,h| h[x.class] += 1 }.to_a.sort_by{ |x| -x[1] }.first(30)                                                                             
=> [[String, 307191], [Hash, 40345], [Array, 39571], [ActiveSupport::Multibyte::Unicode::Codepoint, 30593], [ReviewCollector::Review, 10203], [Class, 6920], [Set, 5202], [Proc, 4996], [Symbol, 3376], [MIME::Type::Columnar, 1964], [Regexp, 1830], [Time, 1553], [Thread::Mutex, 1355], [Seahorse::Client::HandlerListEntry, 1153], [Range, 1047], [Nokogiri::XML::SyntaxError, 1028],
[Net::HTTP, 1004], [TCPSocket, 991], [Net::HTTP::Persistent::Connection, 984], [Net::BufferedIO, 978], [Thread::ConditionVariable, 878], [Net::HTTP::Persistent::TimedStackMulti, 859], [Net::HTTP::Persistent::Pool, 859], [Net::HTTP::Persistent, 859], [ActiveSupport::JSON::Encoding::JSONGemEncoder::EscapedString, 825], [Module, 774], [Gem::Requirement, 426], [OpenSSL::SSL::SSLSocket, 372], [OpenSSL::SSL::SSLContext, 372], [OpenSSL::SSL::Session, 371]]
*** detached from process 9743
~/P/o/Review-collector ❯❯❯ bundle exec rbtrace -p 9743 -e 'ObjectSpace.each_object.with_object(Hash.new 0){ |x,h| h[x.source_location] += 1 if x.is_a? Proc }.to_a.sort_by{ |x| -x[1] }.first(30)'
*** run `sudo sysctl kernel.msgmnb=1048576` to prevent losing events (currently: 16384 bytes)
*** attached to process 9743
>> ObjectSpace.each_object.with_object(Hash.new 0){ |x,h| h[x.source_location] += 1 if x.is_a? Proc }.to_a.sort_by{ |x| -x[1] }.first(30)                                                   
=> [[["/home/braulio/.rvm/gems/ruby-2.5.1/gems/net-http-persistent-3.0.0/lib/net/http/persistent.rb", 525], 1820], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/net-http-persistent-3.0.0/lib/net/http/persistent/timed_stack_multi.rb", 7], 910], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/net-http-persistent-3.0.0/lib/net/http/persistent/pool.rb", 29], 909], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/delegate.rb", 347], 142], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/site_ruby/2.5.0/bundler/current_ruby.rb", 80], 90], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.33/lib/seahorse/client/http/response.rb", 13], 62], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.33/lib/seahorse/client/net_http/handler.rb",
79], 40], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.33/lib/aws-sdk-core/plugins/param_converter.rb", 20], 40], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.33/lib/seahorse/client/http/response.rb", 164], 30], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/site_ruby/2.5.0/bundler/errors.rb", 6], 30], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/timeout.rb", 83], 28], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/timeout.rb", 80], 28], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/countries-2.1.4/lib/countries/country.rb", 8], 27], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/site_ruby/2.5.0/bundler/feature_flag.rb", 20], 26], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/cgi/core.rb", 437], 26], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.33/lib/seahorse/client/http/response.rb", 145], 23], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/addressable-2.5.2/lib/addressable/uri.rb", 796], 22], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/sequel-5.10.0/lib/sequel/dataset/dataset_module.rb", 29], 21], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/mime-types-3.1/lib/mime/type/columnar.rb", 25], 21], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-resources-2.11.33/lib/aws-sdk-resources/services/sqs/queue_poller.rb", 514], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.33/lib/seahorse/client/response.rb", 17], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/newrelic_rpm-4.4.0.336/lib/new_relic/agent/configuration/default_source.rb", 13], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.33/lib/seahorse/client/base.rb", 204], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/oni-4.1.2/lib/oni/daemon.rb", 177], 20], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/ostruct.rb", 178], 20], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/ostruct.rb",
177], 20], [["/home/braulio/Projects/olery/Review-collector/lib/review_collector/http_client.rb", 294], 19], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-resources-2.11.33/lib/aws-sdk-resources/operation_methods.rb", 18], 19], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/oni-4.1.2/lib/oni/daemon.rb", 118], 17], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb", 181], 16]]
*** detached from process 9743
brauliobo commented 6 years ago

Moving to 2.9.4 appears to solve the issue

~/P/o/Review-collector ❯❯❯ bundle exec rbtrace -p 3646 -e 'ObjectSpace.each_object.with_object(Hash.new 0){ |x,h| h[x.source_location] += 1 if x.is_a? Proc }.to_a.sort_by{ |x| -x[1] }.first(30)'
*** run `sudo sysctl kernel.msgmnb=1048576` to prevent losing events (currently: 16384 bytes)
*** attached to process 3646
>> ObjectSpace.each_object.with_object(Hash.new 0){ |x,h| h[x.source_location] += 1 if x.is_a? Proc }.to_a.sort_by{ |x| -x[1] }.first(30)
=> [[["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/delegate.rb", 347], 142], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/site_ruby/2.5.0/bundler/current_ruby.rb", 80], 90], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.82/lib/seahorse/client/http/response.rb", 13], 62], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.82/lib/aws-sdk-core/plugins/param_converter.rb", 20], 42], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.82/lib/seahorse/client/net_http/handler.rb", 79], 41], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.82/lib/seahorse/client/http/response.rb", 164], 32], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/timeout.rb", 83], 31], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/timeout.rb", 80], 31], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/site_ruby/2.5.0/bundler/errors.rb", 6], 30], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/countries-2.1.4/lib/countries/country.rb", 8], 27], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/site_ruby/2.5.0/bundler/feature_flag.rb", 20], 26], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/cgi/core.rb", 437], 26], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.82/lib/seahorse/client/http/response.rb", 145], 24], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb", 165], 23], [["/home/braulio/Projects/olery/Review-collector/lib/review_collector/http_client.rb", 294], 21], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.82/lib/seahorse/client/response.rb", 17], 21], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/sequel-5.10.0/lib/sequel/dataset/dataset_module.rb", 29], 21], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/mime-types-3.1/lib/mime/type/columnar.rb", 25], 21], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-resources-2.11.82/lib/aws-sdk-resources/services/sqs/queue_poller.rb", 514], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb", 181], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb", 175], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb", 164], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/mechanize-2.7.6/lib/mechanize/http/auth_store.rb", 25], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb", 590], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb", 589], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/oni-4.1.2/lib/oni/daemon.rb", 177], 20], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/ostruct.rb", 178], 20], [["/home/braulio/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/ostruct.rb", 177], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/aws-sdk-core-2.11.82/lib/seahorse/client/base.rb", 204], 20], [["/home/braulio/.rvm/gems/ruby-2.5.1/gems/newrelic_rpm-4.4.0.336/lib/new_relic/agent/configuration/default_source.rb", 13], 20]]
*** detached from process 3646