The PR implements a new ignore_failures mode of the queue.
Description
Normal behavior of the manager in case the currently running plan fails is to stop execution of the queue and push the current plan to the beginning of the queue so that it could be restarted after the problem is fixed. In the ignore_failures mode, the manager continues with execution of the next plan even if the current plan fails. The queue is still stopped if the current plan is stopped/aborted/halted. The mode may be useful in the system running independent plans and controlled by an autonomous agent, which is responsible for processing execution results.
The queue is still stopped if the manager fails to submit the request to start a plan to the worker, but this is not expected to occur during normal operation.
Motivation and Context
Requested feature.
Summary of Changes for Release Notes
Added
RE Manager status returns the new plan_queue_mode/ignore_failures boolean parameter, which indicates if the mode is enabled.
queue_mode_set API now accepts a value for ignore_failures mode.
The ignore_failures mode may be enabled/disabled using qserver CLI tool:
qserver queue mode set ignore_failures True
qserver queue mode set ignore_failures False
The PR implements a new
ignore_failuresmode of the queue.Description
Normal behavior of the manager in case the currently running plan fails is to stop execution of the queue and push the current plan to the beginning of the queue so that it could be restarted after the problem is fixed. In the
ignore_failuresmode, the manager continues with execution of the next plan even if the current plan fails. The queue is still stopped if the current plan is stopped/aborted/halted. The mode may be useful in the system running independent plans and controlled by an autonomous agent, which is responsible for processing execution results.The queue is still stopped if the manager fails to submit the request to start a plan to the worker, but this is not expected to occur during normal operation.
Motivation and Context
Requested feature.
Summary of Changes for Release Notes
Added
plan_queue_mode/ignore_failuresboolean parameter, which indicates if the mode is enabled.queue_mode_setAPI now accepts a value forignore_failuresmode.ignore_failuresmode may be enabled/disabled usingqserverCLI tool:How Has This Been Tested?
Unit tests are added.