Describe what We need to model a process for using a task queue to make cloud ingest processing more scalable. Currently the ingest processes are designed for a cron driven set of tasks. This will not scale in a cloud environment. We need to evaluate and model a different process for queing up ingest templates or possibly even subdividing certain ingest templates into succinct tasks that can be performed in a container.
Describe what We need to model a process for using a task queue to make cloud ingest processing more scalable. Currently the ingest processes are designed for a cron driven set of tasks. This will not scale in a cloud environment. We need to evaluate and model a different process for queing up ingest templates or possibly even subdividing certain ingest templates into succinct tasks that can be performed in a container.