Is Ridley affected by memory leak in celluloid > 0.16.0, < 0.17.2?

[renier]

See https://github.com/celluloid/celluloid/issues/670

[dannypurcell]

It seems so, I think it has something to do with Celluloid's use of a internal thread cache.

Here is a snippet that demonstrates the issue

for i in 1..10000 do 
  r = Ridley.new(connection_params) 
  r.terminate 
  `ps ax -o pid,rss | grep -E "^[[:space:]]*#{$$}"`.strip.split.map(&:to_i)
    .tap{ | _, mem | 
      puts "i: #{i} mem(mb): #{mem/1024.0}"
    } 
end

Output from pry

pry(main)> for i in 1..10000 do r = Ridley.new(connection_params); r.terminate; `ps ax -o pid,rss | grep -E "^[[:space:]]*#{$$}"`.strip.split.map(&:to_i).tap{|_, mem| puts "i: #{i} mem(mb):#{mem/1024.0}"; } end

i: 1 mem(mb):63.75
...
i: 265 mem(mb):176.6015625
Celluloid::Task::TerminatedError: task was terminated
--
 0: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/actor.rb:345:in `each'
 1: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/actor.rb:345:in `cleanup'
 2: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/actor.rb:329:in `shutdown'
 3: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/actor.rb:321:in `handle_crash'
 4: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/actor.rb:166:in `rescue in run'
 5: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/actor.rb:148:in `run'
 6: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/actor.rb:130:in `block in start'
 7: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in `block in initialize'
 8: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in `block in get_thread'
 9: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in `block in create'
10: ~/.rvm/gems/ruby-2.3.1/gems/logging-2.1.0/lib/logging/diagnostic_context.rb:450:in `block in create_with_logging_context'
11: (celluloid):0:in `remote procedure call'
12: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in `value'
13: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in `method_missing'
14: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/proxies/cell_proxy.rb:17:in `_send_'
15: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid.rb:169:in `new'
16: ~/.rvm/gems/ruby-2.3.1/gems/ridley-4.5.1/lib/ridley/client.rb:149:in `initialize'
17: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `public_send'
18: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `dispatch'
19: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in `dispatch'
20: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in `block in invoke'
21: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in `block in task'
22: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in `block in task'
23: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in `block in initialize'
24: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in `block in create'
25: (celluloid):0:in `remote procedure call'
26: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in `value'
27: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in `method_missing'
28: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/proxies/cell_proxy.rb:17:in `_send_'
29: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid.rb:169:in `new'
30: ~/.rvm/gems/ruby-2.3.1/gems/ridley-4.5.1/lib/ridley.rb:29:in `new'
31: (pry):5:in `__pry__'

After the test crashes Celluloid is unable to do anything else

pry(main)> Celluloid.shutdown

Exception: ThreadError: can't create Thread: Resource temporarily unavailable
--
 0: ~/.rvm/gems/ruby-2.3.1/gems/logging-2.1.0/lib/logging/diagnostic_context.rb:447:in `start'
 1: ~/.rvm/gems/ruby-2.3.1/gems/logging-2.1.0/lib/logging/diagnostic_context.rb:447:in `create_with_logging_context'
 2: ~/.rvm/gems/ruby-2.3.1/gems/logging-2.1.0/lib/logging/diagnostic_context.rb:412:in `start'
 3: ~/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:81:in `block in timeout'
 4: ~/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `block in catch'
 5: ~/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `catch'
 6: ~/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:33:in `catch'
 7: ~/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/timeout.rb:106:in `timeout'
 8: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:66:in `shutdown'
 9: ~/.rvm/gems/ruby-2.3.1/gems/celluloid-0.16.0/lib/celluloid.rb:156:in `shutdown'
10: (pry):6:in `__pry__'

[lamont-granquist]

looks fixed upstream in celluloid..