Research: Bring terminate_backend() task into source control as Celery task

[lbeaufort]

There's a script in an on-premise server that's terminating queries over 10 minutes. We should bring this task into source control as Celery task and management command.

Action items:

• [ ] Document existing process - how often is this running? what kind of logging is happening? We can probably keep the logging as-is and just bring the terminate task, but will need to think about what makes the most sense.
• [ ] Look at existing tasks that touch the database. The task will need write access - refresh_materialized comes to mind
• [ ] Write a management command that can be run manually and is run with celery beat on a certain schedule.

Completion criteria:

• [ ] Complete research into whether we should terminate backend task and convert to a celery task, and not running on an on-premise server

[lbeaufort]

@dzhang-fec would like to keep this task running on the on-prem servers until Celery is more stable, so de-prioritizing for now.

[dzhang-fec]

the Celery is not stable and hard to change per requests by spike ETL data loading and web high loading . Yes, we can keep this task running on the on-prem servers until Celery is more stable. Thanks!

[dzhang-fec]

Update: It is fine now during the high loading of the Nov. election. I suggest to keep this at on-prem servers for now.