lumoslabs / scripterator

Lightweight script harness and DSL for iterating over and running operations on ActiveRecord model records
MIT License
10 stars 1 forks source link

Use redis key to pause or slow down scripterator run #4

Open lost-theory opened 10 years ago

lost-theory commented 10 years ago

Sometimes we need to pause / scale back on how fast a parallel scripterator run is running because it's causing too much load, causing performance problems, we want to investigate some errors, etc.

It'd be cool to have a flag in redis that controlled this.

For example if one_timer_script:blabla:paused exists it would pause on the next ID and wait for that key to be deleted.

Or if one_timer_script:blabla:delay was set to 0.5 it would sleep 0.5 seconds before processing the next ID.