brandonhilkert / sucker_punch

Sucker Punch is a Ruby asynchronous processing library using concurrent-ruby, heavily influenced by Sidekiq and girl_friday.
MIT License
2.64k stars 114 forks source link

Memory leaks in 1.6.0 #137

Closed film42 closed 8 years ago

film42 commented 9 years ago

We use sucker punch for a cache warming gem we wrote called Firebolt. We recently saw some memory leaks happening, and were able to point back to sucker punch (or the new celluloid) as the cause. We've been using sucker_punch in prod for quite a while now, and saw improvement after we rolled back to 1.5.1.

This is where this issue sucks. I don't have the VisualVM dumps, so that's not very helpful, and all I can point at is that there definitely more than 10 threads from Celluloid, and there were definitely more than 25 threads waiting on ConditionObjects. Even though I don't have better evidence to provide for the cause, I wanted to get this issue open to see if anyone else has seen any issues.

If I get some time, I'll try to come up with a reproducible example.

brandonhilkert commented 9 years ago

@film42 Thanks for the heads up. I haven't heard of anyone else experiencing this, but I also doubt that most sucker_punch users are doing much more than using it to send emails in the background. For now, I'd say stick to an older version. Going forward, I have plans to pull out Celluloid in favor of https://github.com/ruby-concurrency/concurrent-ruby for v2. There's a branch in there with all that. It's not quite ready, but I only mention it b/c I'd hate for you to dig in hard to Celluloid and then have it disappear and the problem is solved, or at least different.

Hope that helps and let me know if you find out any more.

kt0 commented 8 years ago

We're facing similar issue with 1.6, but testing with 1.5 and also concurrent branch it's operating normally.

brandonhilkert commented 8 years ago

@film42 @kt0 I released 2.0.0.beta1 today. Please give that a shot and let me know how it goes.

film42 commented 8 years ago

Thank you!