Node-Management Send fire_event for deployement steps

[carlito-scality]

Component: salt

Why this is needed:

In the UI, we want to watch events (see Salt docs) sent during a Node deployment via Salt.

When starting a job through SaltAPI using the async variant, we retrieve a JID (Job ID). However, the runner itself only sends two events: salt/run/<jid>/new and salt/run/<jid>/ret. The states/other runners triggered by the parent runner will not know the parent JID, hence leaving no way to link them to the parent job.

Consequently, we cannot show progress information during the deployment if relying on native events.

What should be done:

We need to be able to track all jobs spawned by a runner from the UI. Also, we need to track progress with higher-level info than detailed state names, since we want to make it as easy as possible for our users.

Implementation proposal (strongly recommended):

On Salt side, we should:

• [ ] save the orchestrate JID in the target Node annotation metalk8s.scality.com/active-orchestration
• [ ] remove the annotation on success/failure of the orchestrate
• [ ] fire custom events from the orchestrate to provide high-level information

The UI should then:

• [ ] replace saved JIDs in LocalStorage with the metalk8s.scality.com/active-orchestration Node annotation
• [ ] watch custom events respecting metalk8s/orchestrate/deploy_node/<jid>/<suffix> for high-level info
• [ ] display high-level events in the main Nodes list page and the detail Job page

Test plan:

One should be able to:

• start a Node deployment and watch high-level events until success/failure
• start a Node deployment and watch low-level events until success/failure

---

Relates to: #1045

[gdemonet]

Quick update:

• We managed to retrieve and use the orchestration JID from within an Orchestrate SLS (it is available to state modules in the __orchestration_jid__ global)
• We managed to fire custom events using this JID, so that we can link them to the orchestrate job in the UI
• We also managed to mark this JID in an annotation on the target Node (for metalk8s.orchestrate.deploy_node)

Now, all of this works until we start using SaltAPI: for some reason, the JID returned by calling runner_async is not passed down to state.orchestrate as the orchestration_jid, and all aforementioned events and annotation are now decorrelated from the parent job.

It is not just related to the async aspect of the execution, since salt-run --async state.orchestrate metalk8s... has the expected behaviour.