Add Flows to CALM

[rocketstack-matt]

Feature Request

Description of Problem:

CALM as of 2024-09 provides a flexible solution for capturing detailed technical architectures and control requirements to enable us to apply constraints onto individual system architectures and reusable patterns. Furthermore we have proved that we can go from architectural constraints in CALM to infrastructure as code / configuration in deployed environments, specifically k8s.

As we look to address more architecture control domains such as resiliency, it becomes apparent that in any sufficiently complex system we will have multiple flows going through the same nodes and relationships. This poses a problem when we're looking to add controls as a requirement such as latency or recovery time must represent the most critical flow.

This can have the effect of obscuring where 99% of flows may have very different requirements to the 1% (e.g. 24 hour recovery vs. 5 minutes). This actually means we can't make decisions such as moving the critical flows to separate systems / components.

Additionally, it is very hard to communicate to developers, especially on older systems, what business processes it is that the system they're developing pertain to or to ask questions about what business flows are impacted by particular changes to the software.

Potential Solutions:

We should add a new "Flow" construct into the CALM model. This would allow us to capture, in business terms, the purpose of a flow and should enable us to map that flow to the technical components (nodes / relationships) that it flows through and in what order. We should also be able to attach control requirements / configurations to business flows; this would enable us to place controls where they make most logical sense (note, I am not suggesting controls can't be put on the technical architecture, different controls make sense to be places in technical vs. business architecture depending on the control).

There seem to logical options we could pursue:

1. Add a new top level collection to CALM which would allow us to capture business flows related to specific architecture document.
2. Add a standalone construct which would enable us to capture individual flows as discrete artefacts which could be stored in a suitable artefact store, but which could reference one or more technical architectures using a similar mechanism to the detailed-architecture link.

I am leaning to the second option being more versatile, although think discussion is required.

[rocketstack-matt]

@LeighFinegold @develontopia @dc-ms

[LeighFinegold]

Adding a "Flow" construct to the CALM model will significantly enhance our ability to manage and understand complex systems.

With an observable architecture, there are lot of benefits that can be had:-

Dynamic Flow Views: This will allow us to dynamically build flow views of our architecture, providing clearer insights into how various components interact within the system.

Change Management: Communication around change management becomes much more straightforward. If we can clearly identify how changes impact business flows, it streamlines discussions and decisions.

Risk Assessment: We can calculate risk scores more effectively of a change. For example, if I change component X, we can immediately see its impact on 20 business flows, allowing for more informed decision-making.

Component Strain: This approach can enable rationalization whether we are placing too much strain on certain components, leading to better resource allocation and system performance.

I’ve taken a stab at a first version https://github.com/LeighFinegold/architecture-as-code/commit/f10a781fb64a7a1ced142664413d07702f9978ae to kick off the discussion. I think flows only require relationships from main architecture definition, as nodes can then be derived. It’s possible I've conflated another concept that CALM needs/will have in relation to data lineage when trying to come up with flow business requirements. Also I assume maybe this spec needs to also link evidence that we have implemented the flow? Other things to consider is when certain flows simultaneously communicate over nodes. I suspect in that scenario we need to come up with a sequence number system like 2.1 and use validators to ensure sequences makes sense

Looking forward to see how this definition evolves.

[jpgough-ms]

@rocketstack-matt @LeighFinegold duplicate issue of: #303 ?