Implement Child Batching

resque / php-resque

An implementation of Resque in PHP.

MIT License

217 stars 77 forks source link

Implement Child Batching #11

Open danhunsaker opened 5 years ago

danhunsaker commented 5 years ago

See chrisboulton/php-resque#35 for details and an early implementation.

In a nutshell, reuse job processes for up to X jobs rather than forking for every one.

mfn commented 5 years ago

In a nutshell, reuse job processes for up to X jobs rather than forking for every one.

The X-factor could be multiple conditions:

number of jobs executed
total runtime of the child
memory usage (i.e. only execute jobs as long as certain threshold isn't met)

Such a change is definitely a game changer for performance.

danhunsaker commented 5 years ago

Indeed so! Looking forward to seeing what comes of this one.

rajibahmed commented 5 years ago

Hi @danhunsaker, I am busy these day. I will have install everything and start working on it from this weekend. Can we create slack or discord channel to talk realtime ?

danhunsaker commented 5 years ago

That sounds like an excellent idea. I'll get you the details once that's created.