search

jhuckaby / Cronicle

A simple, distributed task scheduler and runner with a web based UI.
http://cronicle.net
Other
3.88k stars 388 forks source link

Looking for a good way to track a chained event #380

Open frizwi opened 3 years ago

frizwi commented 3 years ago

Summary

I'm trying to use Cronicle as a quick and dirty workflow tool by using chained events. Using it as a web UI works perfectly and am extremely happy with all the features.

However, when kicking off an event using the REST API, there is no easy way to "follow" the chain reaction. It would be really cool if there was a source JOB ID also attached to the downstream jobs so that I can get_active_jobs and search with this JOB ID to find out where in the chain the workflow was up to.

For now, I'm going to try and set up the chain_data in a way that propagates my initial request which I can then match later.

Any ideas for a better way/

Thanks heaps, -Farhan

Steps to reproduce the problem

Your Setup

Version 0.8.56 installed in the default location

Operating system and version?

Ubuntu 16 LTS

Node.js version?

Welcome to Node.js v12.18.0.

Cronicle software version?

Are you using a multi-server setup, or just a single server?

Are you using the filesystem as back-end storage, or S3/Couchbase?

Can you reproduce the crash consistently?

Log Excerpts

mikeTWC1984 commented 3 years ago

Yes, I think it's a good approach to populate some id in chain data. Then just poll "all active jobs" api and check if any of them contain your key. Other way is probably not to use chaining. If you know beforehand which jobs you are going to run, then just run them one by one via rest. I build a plugin for my custom fork that implements workflow using rest api: https://github.com/cronicle-edge/cronicle-edge/blob/main/bin/workflow.js It can be ran as standalone script, just specify base url and api key. It was mainly meant to run N jobs in parallel, but can be adopted to run them 1 by 1 (just set concurrency parameter to 1). Order is based on event title.

I think next version of Cronicle will also include more advanced workflow options.

frizwi commented 3 years ago

Thanks for the info. I originally had in mind setting up the chain reaction from within the web GUI and just kicking off the initial job via the REST API. But I then realised that if I use the REST API to do the chaining (from within my plugins) then it opens up many more possibilities!