Move App Systemd Tasks to Active Jobs

[NoRePercussions]

Partial progress towards #544.

Removed:

• Backup cleaning reminder

Moved:

• Email pulling
• Slack notifications

Note: email pulling is no longer long-running.

Same:

• TS and TS Indexing

Note: for moving to Docker, it may make sense to put the TS server into its own container so that Docker can handle its lifecycle.

[DaAwesomeP]

Note: for moving to Docker, it may make sense to put the TS server into its own container so that Docker can handle its lifecycle.

We can run them from the same container (but separately) and just change the command. This is how I run Sidekiq (and multiple scaled instances of it) in the same Compose instance as Rails:

sidekiq:
  image: ghcr.io/example/example:main
  restart: always
  environment:
    DATABASE_URL: "mariadb://example:example@postgresql/example_production"
    REDIS_URL: "redis://redis:6379/1"
    RAILS_MAILER_DEFAULT_URL_HOST: "example"
    RAILS_MAILER_DEFAULT_URL_PORT: "443"
    RAILS_MAILER_DEFAULT_URL_PROTOCOL: "https"
    RAILS_MAILER_SMTP_HOST: "example"
    RAILS_MAILER_SMTP_PORT: "587"
  command: bundle exec sidekiq
  networks:
    - app-mariadb
    - app-redis
  depends_on:
    - redis
    - mariadb  
  deploy:
    replicas: 3
redis:
  image: redis
  restart: always
  networks:
    - app-redis
  volumes:
    - redis:/data
  command: redis-server --save 60 1 --loglevel warning

[NoRePercussions]

I don't like the idea of writing our own scheduling code, whether it is self-requeueing or scheduling other jobs. There's a lot of pitfalls, and it might result in more maintenance time than it is worth. It seems like sidekiq-scheduler got most of their own race condition problems sorted out last month (after they were raised in 2016). I like how it looks in general.

[DaAwesomeP]

I don't like the idea of writing our own scheduling code, whether it is self-requeueing or scheduling other jobs. There's a lot of pitfalls, and it might result in more maintenance time than it is worth. It seems like sidekiq-scheduler got most of their own race condition problems sorted out last month (after they were raised in 2016). I like how it looks in general.

For recurring jobs I agree, self-requeuing has a lot of race conditions to work through. If it ever stops then there is a chance it won't start again.

However, spawning non-requeing jobs from jobs is fine. For example, a recurring job every 5 minutes (running on a scheduler) could spawn individual one-shot jobs to send event notifications at exactly the right time (this example feature should go in a separate PR though).