search

chaps-io / gush

Fast and distributed workflow runner using ActiveJob and Redis
MIT License
1.04k stars 104 forks source link

Detailed job status? #8

Closed xtagon closed 8 years ago

xtagon commented 9 years ago

Hi,

I use Sidekiq workers already, but Gush appeals to me because Sidekiq doesn't have a way to define such workflows/dependencies. Since Gush "just uses Sidekiq" is it compatible with Sidekiq plugins? Specifically, I would like a way to track progress/status of jobs. Let's say I have a job that is going to process 100 files, I would like to be able to add logic into my process method to count how many files have been done at any given moment, and store it in Redis so that I can query it while the job or workflow runs.

For some tasks, this will mean keeping track of the progress of a number of serial/parallel jobs, for others I want progress of an individual job. Is there a way to do this with Gush, if not, could it be planned for the future?

Thanks for listening, Justin

pokonski commented 9 years ago

Hi, Justin.

Do you have any particular plugin on your mind? We could always test it :) On May 22, 2015 1:37 AM, "Justin Workman" notifications@github.com wrote:

Hi,

I use Sidekiq workers already, but Gush appeals to me because Sidekiq doesn't have a way to define such workflows/dependencies. Since Gush "just uses Sidekiq" is it compatible with Sidekiq plugins? Specifically, I would like a way to track progress/status of jobs. Let's say I have a job that is going to process 100 files, I would like to be able to add logic into my process method to count how many files have been done at any given moment, and store it in Redis so that I can query it while the job or workflow runs.

For some tasks, this will mean keeping track of the progress of a number of serial/parallel jobs, for others I want progress of an individual job. Is there a way to do this with Gush, if not, could it be planned for the future?

Thanks for listening, Justin

— Reply to this email directly or view it on GitHub https://github.com/pokonski/gush/issues/8.

xtagon commented 9 years ago

sidekiq-status has almost all of the features I would need. I'll try to test it with Gush next week.

But what would be really great would be a way to manage a higher-level status for workflows and not just individual jobs.

pokonski commented 8 years ago

For anyone interested, there's https://github.com/chaps-io/gush-control which adds gush server command that runs Web UI with details about running and finished workflows :)