Improve Task Queue Management

[qtgye]

Overview

This additional feature should:

• Create a cleaner interface in handling multiple requests to scheduled tasks operations.
• Should gracefully handle multiple requests for a certain task operation, such as when multiple admins request to add a task at the same time which can result to a task being added twice.
• Should be able to monitor queue status. Gracefully stop/remove crashed or erratic tasks.

Implementation

• [ ] Create a QueueManager service Features:
  • [ ] A singleton service that will be one single instance across multiple client requests
  • [ ] Should be the one directly communicating with the ScheduledTasks in terms of handling task queue
  • [ ] Should have a clean up functionality, in which it checks the queue for any idling/crashed/hanging tasks that needs to be removed.
  • [ ] An endpoint for commands
  • [ ] Possible command types include:
    • [ ] enqueue - Queueing a task
    • [ ] start - Starting a task
    • [ ] stop - Stopping a task
    • [ ] dequeue - Dequeueing a task
  • [ ] Possible structure for a command request:

       {
           "command": "enqueue",
           "task": "task_id",  
           "payload": {}, // optional additional data  
       }

  • [ ] A command requests queue
    • [ ] Should contain a list of command requests from the admin. This can be a simple array of command requests data.
  • [ ] A command resolver list
    • [ ] A map of command resolvers.
  • [ ] A command request resolver
    • [ ] For every command request, a Promise (the command resolver) should be created. Within the promise, the request resolver will invoke appropriate process for the requested command. e.g., starts a task through the ScheduledTasks service for a start command request. Resolved value can then be consumed by the parent module (i.e., the QueueController)
    • [ ] If an active command resolver already exists for a command request, it will be used for that request as well. When that resolver's Promise resolves (or rejects), it will propagate to all similar command requests.
• [ ] QueueController updates
  • [ ] Update to use QueueManager
• [ ] Possibility of integrating ScheduledTasks as a service.
• [ ] Queue UI Updates
  • [ ] Change QueueItem status to: QUEUED, RUNNING, or STOPPED
  • [ ] Add "remove" or "dequeue" functionality to Queue

cc @BigKuLaK