Jobs that run continuously while waiting for SQS events

[lorenyu]

As part of https://github.com/navapbc/template-infra/issues/522 we implemented jobs that get triggered when a file is uploaded to S3. That is only appropriate for low-latency / low-throughput jobs like hourly/daily/weekly ETL jobs that get triggered when a file from an external data source gets uploaded to a bucket.

For high frequency, low-latency event-based jobs like processing user uploads, or real-time asynchronous tasks like submitting a claim to a legacy system, we would want the ability to have a continuously running service that processes jobs from a job queue. This ticket is for implementing that.

Implementation notes

1. In following with the same design principles for the existing jobs, we want jobs to be configurable via the app-config module. We currently have a file_upload_jobs.tf file in env-config module, so we could have an analogous file continuous_jobs.tf or something (not sure what to name it) and include it as part the service config similar to what we did for file_upload_jobs.

For each job in the configuration, the service module would:

1. Create an additional ECS service and override the container command
2. Create an SQS queue
3. Create a policy that can access the SQS queue and add it to the service task role
4. Override environment variables to add JOB_QUEUE_NAME environment variable for the service tasks

For testing purposes we probably want to add some example use case to the example app that's provided in tempate-infra and used in platform-test

Deployment notes

Note that changes to the example app aren't propagated from template-infra to platform-test automatically. You'd have to make those changes to platform-test manually.

[rocketnova]

@lorenyu Could you refine this ticket some more?

[lorenyu]

Sorry forgot to refine on Wednesday, just added some notes