Move event-finder, job-scheduler env vars to runtime settings

Sotera / watchman

Watchman: An open-source social-media event-detection system

GNU General Public License v2.0

20 stars 7 forks source link

Move event-finder, job-scheduler env vars to runtime settings #90

Closed lukewendling closed 7 years ago

lukewendling commented 7 years ago

Problem: for env vars that we regularly update for diff envs, consider moving the vars to runtime settings.

Recommended protocol: env var -> settings value -> some default val (Date.now)

TODO:

[ ] handle conn failures when reading from db. perhaps cache previous values in memory?
[ ] create a consistent naming scheme for settings keys (ex. job_monitors:start_time)

what else?

@justinlueders

lukewendling commented 7 years ago

closing. job-queue is the only service that stores state in memory. once it stores the job ids in redis, this issue should be obsolete.

also, b/c job-scheduler is now its own service and not coupled with job queue, it can be stopped independently in order to allow job queue to finish its current run. at that point, job queue can be restarted, to allow for env var changes, debugging, etc.