Work flow configuration in one location

MarkTNO commented 4 months ago

Currently the work flows need to be configured in two locations: the orchestrator and the 'frontend' (for instance omotes-rest).

This should be moved to a single location. The solution should allow for restarts of the front-end and the orchestrator. The proposal is as follows:

Add an 'omotes_work_flows' queue.
Pass the configuration to the orchestrator during deployment where it is stored in cache. Upon start it will send the config to the 'omotes_work_flows' queue. And it will resend the config when requested.
The frontend starts the OmotesInterface which will then request the configuration via the 'omotes_work_flows' queue (catch the case where the orchestrator has not started up yet).
- OmotesInterface.start() will contain a callback for work flow configuration updates (in case the orchestrator (re)starts with an updated configuration).
Add 'AvailableWorkFlows' and 'WorkFlow' schema's to omotes-sdk-protocol.

lfse-slafleur commented 4 months ago

* Pass the configuration to the orchestrator during deployment where it is stored in cache. Upon start it will send the config to the 'omotes_work_flows' queue. And it will resend the config when requested.
I would propose to add both the expected param_dicts including name and type (so they can be dynamically checked at the SDK)

I would propose that the orchestrator sends it to routing key 'omotes_work_flows' but let the SDK create the queue 'omotes_work_flows' (which automatically binds to the routing key of the same name by default). If there is no SDK present, it will prevent from stale messages queuing up in the queue.

The frontend starts the OmotesInterface which will then request the configuration via the 'omotes_work_flows' queue (catch the case where the orchestrator has not started up yet).

I would propose to change the name of this queue (for frontend/sdk -> orchestrator) as it conflicts with the previous queue (for orchestrator -> frontend/sdk). Perhaps request_omotes_work_flows?

I would propose that the SDK sends the request to the routing key request_omotes_work_flows and let the orchestrator create the queue request_omotes_work_flows. If the orchestrator is not yet active, it will prevent stale requests from gathering in the queue.

Add 'AvailableWorkFlows' and 'WorkFlow' schema's to omotes-sdk-protocol. I would propose to add a 3rd schema which is called 'RequestAvailableWorkflows' which is send by the frontend to the orchestrator to request the queue's.

MarkTNO commented 4 months ago

Project-OMOTES / orchestrator

Work flow configuration in one location #42