search

nesquena / backburner

Simple and reliable beanstalkd job queue for ruby
http://nesquena.github.com/backburner
MIT License
428 stars 68 forks source link

New thread-pool-based worker #123

Closed contentfree closed 8 years ago

contentfree commented 8 years ago

Here's the new worker, @nesquena.

nesquena commented 8 years ago

This is great, thanks Dave! Any ideas as to why travis always fails during the install now? Would be nice to get the tests passing again :(

contentfree commented 8 years ago

Hmmm… I do not, but I agree that it would be. (99% sure all the tests passed before I checked in, but it was a month ago so I don't recall for certain.)

nesquena commented 8 years ago

Can you confirm tests are passing locally again now?

geoffroymontel commented 8 years ago

Hello ! I submitted also a thread based worker recently, is it comparable or should I drop it ? :) https://github.com/nesquena/backburner/pull/120 All the best

nesquena commented 8 years ago

@contentfree how does his threaded worker compare to the one we just merged?

contentfree commented 8 years ago

It's similar, though the thread pool will likely be more resilient. @geoffroymontel's spins up the threads once (as my original fixed thread pool implementation did) and it won't self-heal like the latest thread pool implementation will.

I do like his Thread.current use will see if I can put it to use in a similar way.

Re: the tests – Strangely, running the full test suite fails (several failures in test/workers/simple_worker_test.rb, for example) but they don't fail when run individually (at least not simple_worker_test.rb). Looks like we're not resetting things quite right. Haven't found the cause yet, though…

geoffroymontel commented 8 years ago

Indeed my code was not resilient. I was not aware of the ruby-concurrency gem, thanks for letting me know about it !