search

observatoire-mobilite / odmkraken

The **kraken** is the orchestration layer responsible for gathering and postprocessing ODM's mobility data
MIT License
0 stars 0 forks source link

Feature/dsn from envvar #14

Closed gilgeorges closed 2 years ago

gilgeorges commented 2 years ago

Fixes #13, and renames resources to a more sane standard. Also moves the main repo to the main __init__.py file, so that dagit can be started using dagit -m odmkraken

gilgeorges commented 2 years ago

At this point, I smuggle in three further improvements to install2.ps1, so that Windows deployments:

  1. Use the QueuedRunCoordinator by default, meaning jobs are not executed directly but dispatched to the dagster-daemon and dispatched within the limits imposed by dagster.yaml (currently: max. 4 jobs in parallel)
  2. As a consequence, the dagster-daemon service needs access to the application secrets (i.e. database DSNs) too. So this is added to NSSM
  3. The Remove function now explicitly deletes the dagster and postgres data directories to simplify redeployment.