Scheduled exits - Githubissues

Resolves #256. To be reviewed only after #253 is approved (please let me know if I'm blocking you)!

This changeset:

Adds the test /sleep command, which causes the Root process of the Orchestrator to kip for a bit. The prompt remains active. Useful for forcing the Orchestrator to "do something" for a short while.
Adds the exit /at command to schedule exits to happen after certain events. Use exit /at = end to make the Orchestrator exit when the batch queue is empty (i.e. there are no more commands to run). Use exit /at = stop to make the Orchestrator exit the next time an application stops, either of it's own volition (using the Supervisor API), or because it was told to do so with stop /app. Scheduled exits are checked in Root's OnIdle, so they may be blocked by commands that take a long time to execute (like normal exits).
Does not change existing exit behaviour. You still can't use this one in a batch script.
Adds a new way out of the keyboard thread - Root::OnIdle can now pthread_cancel it (and does so when a scheduled exit triggers).

Note that scheduled exits can't be "unscheduled". Life's too short.

Documentation PR: https://github.com/POETSII/orchestrator-documentation/pull/20

By way of example, here's a batch script that runs an application, and causes the Orchestrator to close down once the application has stopped itself. The application in question uses the stop_application Supervisor API call (test_superapi_stop, in the Orchestrator examples repository):

exit /at = stop
load /app = +"test_superapi_stop.xml"
tlink /app = *
place /app = *
compose /app = *
deploy /app = *
init /app = *
run /app = *

Output:

$ ./orchestrate.sh -b test_superapi_stop.poets
POETS> 12:16:58.01:  20(I) The microlog for the command 'load /engine = "../Config/POETSHardwareOneBox.ocfg"' will be written to '../Output/Microlog/Microlog_2021_06_23T12_16_58p0.plog'.
POETS> 12:16:58.01: 140(I) Topology loaded from file ||../Config/POETSHardwareOneBox.ocfg||.
POETS> 12:16:58.01:  20(I) The microlog for the command 'call /file = "/home/mv1g18/repos/orchestrator/test_superapi_stop.poets"' will be written to '../Output/Microlog/Microlog_2021_06_
23T12_16_58p1.plog'.
POETS> 12:16:58.01:  20(I) The microlog for the command 'exit /at = stop' will be written to '../Output/Microlog/Microlog_2021_06_23T12_16_58p2.plog'.
POETS> 12:16:58.01:  69(I) Exit trigger staged - will exit when the next application is completely stopped.
POETS> 12:16:58.01:  20(I) The microlog for the command 'load /app = +test_superapi_stop.xml' will be written to '../Output/Microlog/Microlog_2021_06_23T12_16_58p3.plog'.
POETS> 12:16:58.01: 235(I) Application file ../test_superapi_stop.xml loading...
POETS> 12:16:58.01:  65(I) Application file ../test_superapi_stop.xml loaded in 19 ms.
POETS> 12:16:58.02:  20(I) The microlog for the command 'tlink /app = *' will be written to '../Output/Microlog/Microlog_2021_06_23T12_16_58p4.plog'.
POETS> 12:16:58.02: 234(I) Typelinking graph instance 'test_superapi_stop_instance'...
POETS> 12:16:58.02: 249(I) Successfully typelinked graph instance 'test_superapi_stop_instance'.
POETS> 12:16:58.02:  20(I) The microlog for the command 'place /app = *' will be written to '../Output/Microlog/Microlog_2021_06_23T12_16_58p5.plog'.
POETS> 12:16:58.02: 309(I) Attempting to place graph instance 'test_superapi_stop_instance' using the 'app' method...
POETS> 12:16:58.02: 302(I) Graph instance 'test_superapi_stop_instance' placed successfully.
POETS> 12:16:59.02:  20(I) The microlog for the command 'compose /app = *' will be written to '../Output/Microlog/Microlog_2021_06_23T12_16_58p6.plog'.
POETS> 12:16:59.02: 803(I) Composing graph instance 'test_superapi_stop_instance'...
POETS> 12:16:59.02: 804(I) Graph instance 'test_superapi_stop_instance' composed successfully.
POETS> 12:16:59.02:  20(I) The microlog for the command 'deploy /app = *' will be written to '../Output/Microlog/Microlog_2021_06_23T12_16_59p0.plog'.
POETS> 12:16:59.02: 184(I) Deployment of graph instance 'test_superapi_stop_instance' staged. Waiting for Mothership(s) to acknowledge receipt in the background.
POETS> 12:16:59.02:  20(I) The microlog for the command 'init /app = *' will be written to '../Output/Microlog/Microlog_2021_06_23T12_16_59p1.plog'.
POETS> 12:16:59.02: 187(I) Initialisation of graph instance 'test_superapi_stop_instance' staged. Waiting for Mothership(s) to acknowledge receipt in the background.
POETS> 12:16:59.02:  20(I) The microlog for the command 'run /app = *' will be written to '../Output/Microlog/Microlog_2021_06_23T12_16_59p2.plog'.
POETS> 12:16:59.02: 188(I) Run of graph instance 'test_superapi_stop_instance' staged. Waiting for Mothership(s) to acknowledge receipt in the background.
POETS> 12:17:13.29: 529(I) Mothership (rank 2): Deployment of application 'test_superapi_stop::test_superapi_stop_instance' (to this Mothership) complete.
POETS> 12:17:13.29: 530(I) Mothership (rank 2): Initialising fully-defined application 'test_superapi_stop::test_superapi_stop_instance'.
POETS> 12:17:13.29: 186(I) Application 'test_superapi_stop::test_superapi_stop_instance' successfully deployed on all Motherships it is mapped to.
POETS> 12:17:13.34: 531(I) Mothership (rank 2): Initialisation of application 'test_superapi_stop::test_superapi_stop_instance' (to this Mothership) complete.
POETS> 12:17:13.34: 186(I) Application 'test_superapi_stop::test_superapi_stop_instance' ready to start on all Motherships it is mapped to.
POETS> 12:17:13.34: 532(I) Mothership (rank 2): Starting (running) fully-initialised application 'test_superapi_stop::test_superapi_stop_instance'.
POETS> 12:17:13.34: 186(I) Application 'test_superapi_stop::test_superapi_stop_instance' running on all Motherships it is mapped to.
POETS> 12:17:13.36: 526(I) Mothership: Supervisor for application 'test_superapi_stop::test_superapi_stop_instance' has requested it to be stopped.
POETS> 12:17:13.36: 189(I) Stop of graph instance 'test_superapi_stop_instance' staged. Waiting for Mothership(s) to acknowledge receipt in the background.
POETS> 12:17:13.36: 533(I) Mothership (rank 2): Stopping application 'test_superapi_stop::test_superapi_stop_instance' (which has been started).
POETS> 12:17:13.37: 186(I) Application 'test_superapi_stop::test_superapi_stop_instance' stopped on all Motherships it is mapped to.
POETS> 12:17:13.37:  68(I) Exit trigger hit - closing down the Orchestrator. The prompt is disabled.
*control returns to calling process*

Another example that schedules an exit for the end of the batch script, then sleeps for five seconds:

exit /at = end
test /sleep = 5000

Output:

POETS> 14:48:49.00:  20(I) The microlog for the command 'load /engine = "../Config/POETSHardwareOneBox.ocfg"' will be written to '../Output/Microlog/Microlog_2021_06_23T14_48_49p0.plog'.
POETS> 14:48:49.00: 140(I) Topology loaded from file ||../Config/POETSHardwareOneBox.ocfg||.
POETS> 14:48:49.00:  20(I) The microlog for the command 'call /file = "/home/mark/repos/orchestrator/test.poets"' will be written to '../Output/Microlog/Microlog_2021_06_23T14_48_49p1.plog'.
POETS> 14:48:49.00:  20(I) The microlog for the command 'exit /at = end' will be written to '../Output/Microlog/Microlog_2021_06_23T14_48_49p2.plog'.
POETS> 14:48:49.00:  67(I) Exit trigger staged - will exit when the command queue is empty.
POETS> 14:48:54.00:  20(I) The microlog for the command 'test /sleep = 5000' will be written to '../Output/Microlog/Microlog_2021_06_23T14_48_49p3.plog'.
POETS> 14:48:54.01:  68(I) Exit trigger hit - closing down the Orchestrator. The prompt is disabled.
*control returns to calling process*

POETSII / Orchestrator

Scheduled exits #261