FIX: severe memory leak in WeakValueMap

SamSaffron commented 9 years ago

entries were not being cleaned up correctly from the backing store. Impact is huge:

ENV['RAILS_ENV'] = 'production'
require 'objspace'
require 'memory_profiler'
require File.expand_path("../../config/environment", __FILE__)

def rss
 `ps -eo pid,rss | grep #{Process.pid} | awk '{print $2}'`.to_i
end

PrettyText.cook("hello world")

# MemoryProfiler has a helper that runs the GC multiple times to make
#  sure all objects that can be freed are freed.
MemoryProfiler::Helpers.full_gc
puts "rss: #{rss} live objects #{GC.stat[:heap_live_slots]}"

20.times do

  5000.times { |i|
    PrettyText.cook("hello world")
  }
  MemoryProfiler::Helpers.full_gc
  puts "rss: #{rss} live objects #{GC.stat[:heap_live_slots]}"

end

Before

rss: 137660 live objects 306775
rss: 259888 live objects 570055
rss: 301944 live objects 798467
rss: 332612 live objects 1052167
rss: 349328 live objects 1268447
rss: 411184 live objects 1494003
rss: 454588 live objects 1734071
rss: 451648 live objects 1976027
rss: 467364 live objects 2197295
rss: 536948 live objects 2448667
rss: 600696 live objects 2677959
rss: 613720 live objects 2891671
rss: 622716 live objects 3140339
rss: 624032 live objects 3368979
rss: 640780 live objects 3596883
rss: 641928 live objects 3820451
rss: 640112 live objects 4035747
rss: 722712 live objects 4278779
/home/sam/Source/discourse/lib/pretty_text.rb:185:in `block in markdown': Script Timed Out (PrettyText::JavaScriptError)
    from /home/sam/Source/discourse/lib/pretty_text.rb:350:in `block in protect'
    from /home/sam/Source/discourse/lib/pretty_text.rb:348:in `synchronize'
    from /home/sam/Source/discourse/lib/pretty_text.rb:348:in `protect'
    from /home/sam/Source/discourse/lib/pretty_text.rb:161:in `markdown'
    from /home/sam/Source/discourse/lib/pretty_text.rb:218:in `cook'
    from tmp/mem_leak.rb:30:in `block (2 levels) in <main>'
    from tmp/mem_leak.rb:29:in `times'
    from tmp/mem_leak.rb:29:in `block in <main>'
    from tmp/mem_leak.rb:27:in `times'
    from tmp/mem_leak.rb:27:in `<main>'

After

rss: 137556 live objects 306646
rss: 259576 live objects 314866
rss: 261052 live objects 336258
rss: 268052 live objects 333226
rss: 269516 live objects 327710
rss: 270436 live objects 338718
rss: 269828 live objects 329114
rss: 269064 live objects 325514
rss: 271112 live objects 337218
rss: 271224 live objects 327934
rss: 273624 live objects 343234
rss: 271752 live objects 333038
rss: 270212 live objects 329618
rss: 272004 live objects 340978
rss: 270160 live objects 333350
rss: 271084 live objects 319266
rss: 272012 live objects 339874
rss: 271564 live objects 331226
rss: 270544 live objects 322366
rss: 268480 live objects 333990
rss: 271676 live objects 330654

Recommend a new stable release with this asap. Deploying a monkey patch to Discourse

cowboyd commented 9 years ago

If I understand correctly, this is because while the referenced objects are being gc'd, the actual entries containing the week references are not?

SamSaffron commented 9 years ago

yes, exactly, blogged a bit about this as well. http://samsaffron.com/archive/2015/03/31/debugging-memory-leaks-in-ruby

dedene commented 9 years ago

:+1:

cowboyd commented 9 years ago

Released with v0.12.2

SamSaffron commented 9 years ago

wow, awesome. time to kill my monkey patch :)

On Tue, Apr 7, 2015 at 1:52 PM, Charles Lowell notifications@github.com wrote:

Released with v0.12.2 https://rubygems.org/gems/therubyracer/versions/0.12.2

— Reply to this email directly or view it on GitHub https://github.com/cowboyd/therubyracer/pull/336#issuecomment-90349234.

rubyjs / therubyracer

FIX: severe memory leak in WeakValueMap #336