search

chaps-io / gush

Fast and distributed workflow runner using ActiveJob and Redis
MIT License
1.03k stars 103 forks source link

Using Redis TTL to expire keys after a certain point #48

Closed dmitrypol closed 6 years ago

dmitrypol commented 6 years ago

Per discussion in https://github.com/chaps-io/gush/issues/26 here is a first attempt at expiring workflow keys. Please consider for including this feature in the library or modify as appropriate.

  1. Added config.ttl to use Redis TTL to expire keys set for various work…
  2. Added expire_workflow and expire_job methods to client
  3. Right now you need to manually call expire! to set expiration for the workflow keys. Alternative design would be to leverage callback that will automatically call this after last job in workflow completes.

I added a test for Client.expire_workflow but I think fakeredis TTL does not work quite the same as Redis (had issues w it in the past).

On separate note I updated README with suggestion on how to check whether workflow is running w particular class name. Helps avoid overlapping workflow same class.

Dmitry

pokonski commented 6 years ago

Hey @dmitrypol!

Apologies for super late reply, this looks really great! I only have some notes about the text in readme, but will adjust after merging :)

And yeah, this is a good time to switch to actual Redis servers to test TTL.

pokonski commented 6 years ago

@dmitrypol this is now released in version 1.1.0 :1st_place_medal:

dmitrypol commented 6 years ago

Thanks for merging it. Completely understand slow response, life is busy.