luftfartsverket / reqstool-client

Reqstool is a tool for managing requirements with related software verification cases (aka tests) and verification results (test results)
https://luftfartsverket.github.io/reqstool-client/
MIT License
2 stars 0 forks source link

Requirement references throws error if requirement_ids are not defined #42

Closed lfvbarcus closed 7 months ago

lfvbarcus commented 7 months ago

If you are using the references field when you are defining a requirement, the schema allows you to either specifiy references like this

    references:
      - requirement_ids:
          - "REQ 0825"
        sources:
          - "ext-101"

or like this:

    references:
      - sources:
          [
            "E08-EXAMPLE: 3.1.2.1.1 SUB_RESP",
            "E08-EXAMPLE: 3.1.2.1",
          ]

if you are writing your references like in example 2, the the report command will throw an error since it expects a value in the requirement_ids field.

The questions is what is correct? Should you be able to define sources without requirement_ids? In that case, the logic in report.py (line 178) needs to change, otherwise, the schema needs to be updated to not allow sources to be defined without requirements_ids

lfvjimisola commented 7 months ago

This is the correct behaviour.

https://github.com/Luftfartsverket/reqstool-client/blob/main/src/reqstool/resources/schemas/v1/requirements.schema.json#L274

lfvbarcus commented 7 months ago

This is the correct behaviour.

  • requirement_ids are urn_id where the urn can be left out (default then to current urn).
  • sources: has nothing to do with urn_ids. it's an array of strings describing where this reference comes from. see the schema for more details.

https://github.com/Luftfartsverket/reqstool-client/blob/main/src/reqstool/resources/schemas/v1/requirements.schema.json#L274

Alright. Then we will have to consider this a bug in reqstool-client since it expects the requirments_id field to be present.

lfvjimisola commented 7 months ago

Why is it a bug?

That requirements_id needs to be present if references is is the expected behavior to my knowledge and what we decided on before.

How is it suppose to work instead?

lfvbarcus commented 7 months ago

I would consider this a bug since if you define:

    references:
      - sources:
          [
            "E08-EXAMPLE: 3.1.2.1.1 SUB_RESP",
            "E08-EXAMPLE: 3.1.2.1",
          ]

and would like a report of your requirements the application will crash.

I'm not saying that it should work in any other way, just that it currently doesn't work as it is intended to.

lfvjimisola commented 7 months ago

I would consider this a bug since if you define:

    references:
      - sources:
          [
            "E08-EXAMPLE: 3.1.2.1.1 SUB_RESP",
            "E08-EXAMPLE: 3.1.2.1",
          ]

and would like a report of your requirements the application will crash.

I'm not saying that it should work in any other way, just that it currently doesn't work as it is intended to.

But this is not how one is suppose to specify references. I say again, sources is not an urn_id list - check schema. This is the current valid format:

    references:
      - requirement_ids: ["REQ010"]
        sources: ["SWIM", "ED254"]
      - requirement_ids: ["REQ011"]

where sources is an array of strings that just is human readable to hint about the referenced req id(s).

lfvjimisola commented 7 months ago

Yes, the require for requirements_id is missing. I thought I wrote that above but it seems as if I never sent it. Sorry about the confusion.

lfvbarcus commented 7 months ago

That's fine. Then a change to the schema is all that needs to be done.

rlinhem commented 7 months ago

What we seen is a use of 'sources' where the reference is pointing to a specific section of a external document holding the original text which the requirement is from. In this case we don't reference another requirement but something outside reqstool. In this case the source of the requirement encoded in reqstool. A valid use of the 'references.sources' field, and according to the schema.

lfvbarcus commented 7 months ago

Could reference sources other than other requirements be written in the rationale field instead? I think that the reference part of the requirements.yml originally where only used to reference other requirements.

rlinhem commented 7 months ago

'references' part of requirements has been rather unclear until latest schema. In rationale we can put whatever info and that's where references to sources of the requirement was before the 'references.sources' field was in the schema.

lfvJonas commented 7 months ago

I think the current 'references' part works fine when you are referring to already created requirements in Reqstool. But currently there is no structured way to reference a external documents outside Reqstool.

I see three use cases for 'references':

  1. Reference a requirement in Reqstool. This use case we have covered and is implemented

  2. Reference a external document with requirements In this case we just mapping the requirement into Reqstool. But we still need a way to reference the original requirement and the document .

  3. Reference a external document with not requirements. When creating a requirement in Reqstool based on a document that has no requirements. We need to be able to reference a document and where in the document.

The description/rational part might be sufficient for the last two use cases. But it would be nice the have something more structured.

lfvbarcus commented 7 months ago

Sources will be removed from schema and requirements_ids will be a required field if references are defined. References value will aslo change slightly. See below.

references:
  requirement_ids:
      - "REQ 0825"
      - "sys001:req101"
lfvbarcus commented 7 months ago

So, this issue should be addressed in the following manner:

We should update the schema for the references field of the requirements.yml