checking remaining funds sent back to Hub vs actual remaining
syncing SSO users
reporting and carrying out over budget switch offs
Many of these tasks iterate through each project sequentially, meaning there could be long wait times for all to complete, and the slow running/failure for one project could impact the task running for subsequent projects. Especially given many involve calls to Azure's APIs, which are often slow and unreliable.
Some tasks do use separate processes for each project using fork, but this is uncontrolled and may lead to problems if we have a large number of projects.
We should investigate and implement using background jobs for these tasks, to make their running more controlled, observable and scalable. A suitable queuing backend must be implemented, such as Sidekiq or Resque.
Currently we have a significant number of rake tasks running on a schedule:
Many of these tasks iterate through each project sequentially, meaning there could be long wait times for all to complete, and the slow running/failure for one project could impact the task running for subsequent projects. Especially given many involve calls to Azure's APIs, which are often slow and unreliable.
Some tasks do use separate processes for each project using
fork, but this is uncontrolled and may lead to problems if we have a large number of projects.We should investigate and implement using background jobs for these tasks, to make their running more controlled, observable and scalable. A suitable queuing backend must be implemented, such as Sidekiq or Resque.