florianschanda commented 1 year ago

It shall be possible to define a tracing policy where several upstream targets can be specified via trace to:, but the coverage shall be satisfied as soon as one link to at least one target is found.

Example:

activity "test" {
  trace to: "system requirement" or "software requirement";
}

Trace to itself shall be allowed.

phiwuu commented 2 months ago

Note: The parsing of the "trace to" keyword is happening here: https://github.com/bmw-software-engineering/lobster/blob/c87654f6fe5dca0e6155f2a3912495f99f840076/lobster/config/parser.py#L192

Currently the config parser uses the needs_tracing_down attribute to remember if a level needs traces downwards. This is parsed here: https://github.com/bmw-software-engineering/lobster/blob/c87654f6fe5dca0e6155f2a3912495f99f840076/lobster/config/parser.py#L188 A possible refactoring is to use a list of levels instead of a Boolean value. The list will then include all levels.

But it is unclear how to compute the "downstream" coverage if one of the incoming connections is optional. Consider the following tracing policy:

graph TD;
level1[Level 1];
level2[Level 2.a];
level3[Level 3];
level4[Level 2.b];
level5[Level 5];

level1 --> level2 --> |or| level3;
level1 --> level4 --> |or| level3;
level2 --> level5;

Here the arrows shall indicate the resulting tracing policy. That is, the Level 2.a has a trace to to Level 1, and so on.

Now assume that Level 3 uses the new "or" feature to enforce links to either Level 2.a or Level 2.b. What is the coverage of level 2.a? Are all downstreams mandatory?

phiwuu commented 2 months ago

Use Cases

Here is a summary of use cases that we need to support.

Upstream Alternatives

Imagine a project allows several database alternatives for their software requirements, like TRLC, Codebeamer, IBM Doors. The tracing policy will look like this:

flowchart BT
    PR[Product Requirements]
    Solution1[Codebeamer]
    Solution2[TRLC]
    Solution3[IBM Doors]
    SWR[Implementation]
    SWR-->|Path A|Solution1-->|Path A|PR
    SWR-->|Path B|Solution2-->|Path B|PR
    SWR-->|Path C|Solution3-->|Path C|PR

All three paths A, B, and C must be possible individually. That means, one of the paths could be used, but is not mandatory.

Level Self References

It must be possible to allow self-references within one level.

flowchart BT
    subgraph Software Requirements
    SWR1[SW Req 1]-->SWR2[SW Req 2]-->SWR3[SW Req 3]
    SWR4[SW Req 4]
    end
    subgraph System Requirements
    SYSR1[Sys Req 1]
    SYSR2[Sys Req 2]
    end
    subgraph Implementation
    Impl1[Implementation 1]-->SWR1
    Impl2[Implementation 2]-->SWR4
    end
    SWR3-->SYSR1
    SWR4-->SYSR2

Possible Feature Design

The new feature to support these use cases could behave as follows:

the trace to keyword allows or to address several levels
the trace to keyword allows to address the level itself, even if it is the only level given in trace to (remember you can add justifications why a reference is not needed, and this way you can require traceability within the top-level, except for the topmost element in that level).
any such trace is taken into full account when computing the coverage downwards (means: the downstream is mandatory)
requires has precedence over trace to when figuring out if a reference is optional or mandatory (so it is possible to make a relation optional, even if it is modelled as mandatory through trace to)

Example

Let's look at an example policy. Each arrow pointing to an or diamond represents the trace to keyword, and all arrows leaving the or diamond give the list of target levels combined with or.

flowchart BT
    ProdReq[Product Requirements]
    CB[Codebeamer]
    TRLC[TRLC]
    DOORS[IBM Doors]
    Impl[Implementation]
    or_after_CB{or}
    or_after_impl{or}
    or_after_TRLC{or}
    or_after_DOORS{or}
    Impl --> or_after_impl
    TRLC --> or_after_TRLC
    DOORS --> or_after_DOORS
    CB --> or_after_CB
    or_after_impl --> CB
    or_after_impl --> TRLC
    or_after_impl --> DOORS
    or_after_CB --> CB
    or_after_CB --> ProdReq
    or_after_TRLC --> TRLC
    or_after_TRLC --> ProdReq
    or_after_DOORS --> DOORS
    or_after_DOORS --> ProdReq

In this example the config part where TRLC requires itself or Product Requirements looks like this:

requirements "TRLC" {
   source: "trlc.lobster";
   trace to: "TRLC" or "Product Requirements";
}

Everything mandatory

So far all trace to relations in this diagram are mandatory. This means a product requirement must be broken down into three elements:

an element in codebeamer
an element in TRLC
an element in IBM Doors.

If any of these downstream references is missing, then the product requirement shall be counted as "not okay" in the coverage.

Introduce conditional downstreams

If the user wants to give a choice how to break down the product requirement (instead of enforcing all three lower levels), then the user must add requires to the "Product Requirements" level:

requirements "Product Requirements" {
   source: "product_requirements.lobster";
   requires: "Codebeamer" or "TRLC" or "IBM Doors";
}

This now overwrites the enforced trace from Product Requirements to Codebeamer by a conditional trace that only at least one downstream must be found, but not all three at the same time. The image below depicts the full configuration that a user has to create in the tracing policy. A solid arrow in the diagram means that the reference is mandatory. A dashed arrow means the reference is conditional (at least one must be given).

flowchart BT
    subgraph "trace to (upstream)"
    direction BT
    ProdReq[Product Requirements]
    CB[Codebeamer]
    TRLC[TRLC]
    DOORS[IBM Doors]
    Impl[Implementation]
    or_after_CB{or}
    or_after_impl{or}
    or_after_TRLC{or}
    or_after_DOORS{or}
    Impl --> or_after_impl
    TRLC --> or_after_TRLC
    DOORS --> or_after_DOORS
    CB --> or_after_CB
    or_after_impl -.-> CB
    or_after_impl -.-> TRLC
    or_after_impl -.-> DOORS
    or_after_CB -.-> CB
    or_after_CB -.-> ProdReq
    or_after_TRLC -.-> TRLC
    or_after_TRLC -.-> ProdReq
    or_after_DOORS -.-> DOORS
    or_after_DOORS -.-> ProdReq
    ProdReq --> ProdReq
    end
    subgraph "requires (downstream)"
    direction TB
    Impl2[Implementation]
    ProdReq2[Product Requirements]
    CB2[Codebeamer]
    TRLC2[TRLC]
    DOORS2[IBM Doors]
    or_below_ProdReq{or}
    or_below_codebeamer{or}
    or_below_TRLC{or}
    or_below_DOORS{or}
    ProdReq2 --> or_below_ProdReq
    or_below_ProdReq -.-> |conditional| CB2
    or_below_ProdReq -.-> |conditional| TRLC2
    or_below_ProdReq -.-> |conditional| DOORS2
    or_below_ProdReq -.-> |conditional| ProdReq2
    CB2 --> or_below_codebeamer
    or_below_codebeamer -.-> Impl2
    or_below_codebeamer -.-> CB2
    TRLC2 --> or_below_TRLC
    or_below_TRLC -.-> TRLC2
    or_below_TRLC -.-> Impl2
    DOORS2 --> or_below_DOORS
    or_below_DOORS -.-> DOORS2
    or_below_DOORS -.-> Impl2
    end

The complete tracing policy file would look like this:

requirements "Product Requirements" {
   source: "prod.lobster";
   trace to: "Product Requirements"
   requires: "Product Requirements" or "Codebeamer" or "TRLC" or "IBM Doors";
}

requirements "Codebeamer" {
   source: "codebeamer.lobster";
   trace to: "Codebeamer" or "Product Requirements";
   requires: "Codebeamer" or "Implementation";
}

requirements "TRLC" {
   source: "trlc.lobster";
   trace to: "TRLC" or "Product Requirements";
   requires: "TRLC" or "Implementation";
}

requirements "IBM Doors" {
   source: "doors.lobster";
   trace to: "IBM Doors" or "Product Requirements";
   requires: "IBM Doors" or "Implementation";
}

requirements "Implementation" {
   source: "implementation.lobster";
   trace to: "Codebeamer" or "TRLC" or "IBM Doors";
}

Summary

The user will end up specifying the upstream direction and the downstream direction explicitly.
If no downstream links are specified, then the upstream links are considered as mandatory downstream links.

phiwuu commented 1 month ago

New proposal:

The configuration file shall only support the following rules:

trace to specifies all the outgoing targets.
trace from specifies all the incoming targets.
Multiple lines of the same rule are treated as if they had an and between them.
or is supported in each line, to specify that only one target is needed, not all of them.
Self-references are also allowed.
Neither trace from nor trace to causes an implicit rule.

Advantages:

This means, the logic to implement for trace to and trace from is the same, except that the direction is inversed.
The trace to keyword is backwards compatible.
trace to no longer has an implicit rule for the target of level. Everything is specified in an explicit manner.

Disadvantage:

It is a breaking change, because requires will no longer be supported, and trace from will be added.
The requirements manager has to specify both the "upwards" direction and the "downwards" direction explicitly.
The config parser needs to support forward lookup (e.g. read all levels, resolve references later)

Examples:

requirements "Laws" {}

requirements "System Requirements" {
  trace to: "Software Requirements" or "System Requirements";
  trace to: "Component Test";
}

requirements "Software Requirements" {
   trace to: "Software Requirements" or "System Requirements";
   trace to: "Laws";
   trace from: "Software Requirements" or "Implementation"
   trace from: "Component Test"
}

requirements "Implementation" {
   trace to: "Unit Test";
}

requirements "Unit Test" {
   trace to: "Component Test";
   trace to: "Software Requirements";
}

requirements "Component Test" {
   trace to: "Software Requirements";
}

An item is not OK, if:

at least one condition given by all "trace to" is violated
or, at least one condition given by all "trace from" is violated Otherwise, the item is OK. This means, if no trace to and trace from is specified at all (like with "Laws" in the example above), then the item is OK.

Justifications

just_up (to be renamed to justification to): if an Item has got a value in just_up, then the item is OK with respect to trace to.
just_down (to be renamed to justification from): if an Item has got a value in just_down, then the item is OK with respect to trace from.
just_global is just a shortcut to give a value for just_up and just_down at the same time.

bmw-software-engineering / lobster

allow optional up-linking #10