openETCS / requirements

WP2: Top Level Project for openETCS requirements
5 stars 13 forks source link

Justification for requirements are missing in almost all cases. #17

Open KlausRuedigerHase opened 11 years ago

KlausRuedigerHase commented 11 years ago

Traceability is a key issue in all state of the art requirements management systems and especially required by CENELEC EN5012X. Except a couple general remarks and introductory statements there is almost no justification given for the individual requirement items. This shortcoming has now created conflicts in the subsequent work package 7, when first time those WP2 requirements with respect to the tools chain have been applied. And will most likely create more issues in further WPs.

A good specification document needs to structure the requirements down to “elementary requirement items” which has mostly been done. They need to be unique and not conflicting each other (that should be verified again).

However:

Justification for each individual requirement item by citing a source (like FPP, Standards, Norms, State-of-the-Art documents, etc.), which justifies that item in particular is almost completely missing in almost all Documents 2.1 … 2.9. .

Unjustified items must be avoided, since they might create cost that could and should be avoided. In addition all requirement items should at least be classified according to their priority, like:

Mandatory (M), Highly Recommended (HR), Recommended (R), Not recommended (NR), Forbidden (F); See: CENELEC is using such classification system;

EN 50128:2011, Annex A, (normative), "Criteria for the Selection of Techniques and Measures" Page 65.

CONCLUSION: The entire set of WP2 requirements need to be revised in order to accommodate this general requirement of "Traceability" since that is HR for SIL1+2 and M for SIL3+4. See Table A5 Page 71, EN50128:2011

Otherwise documents D2.x cannot be used for safety systems!

KRH

sbaro commented 11 years ago

Dear Klaus-Rüdiger,

Regarding traceability and CENELEC, I would like to point out that traceability is only a tool to ensure something. Here, what we wanted to ensure is that the requirements are necessary and sufficient, and this was done with a "review by peer", which is a well-accepted way of proving that a document makes sense.

I certainly agree that traceability should have be enforced. But in order to do this, it would have been necessary to have an upstream User Requirement Specification expressing clearly there needs. Clearly, the FPP did not clarify sufficiently the precise scope of the project, and therefore is not fit for the purpose of traceability. I also want to make clear that the writing of the requirements was particularly difficult, due to the lack of clear upstream documentation.

Nevertheless, we tried to write the documents using our own experience of SIL4 systems, the state of the art, and the other partners' experience. We tried to formalize the goals of the project, and we wrote them down, as we understood them.

Now, the document has been formally reviewed, including by you. This document is already being used by the downstream work packages. As the project manager, you can of course decide to question the whole document, but on your own responsibility. In our point of view, the document is approved and only maintenance should occur (if inconsistency can be demonstrated on a particular point, for example).

I consider that this type of remarks were to be done during the review process. We produced intentionally intermediate documents to allow the stream of the project to go without unnecessary interruption and it seems that it worked. These documents have been produced now several months ago. Your considerations are in order to make the project restart from zero; it is quite inconsistent in term of project management even "agile" management!

I will add that we are also surprised that you waited after the ITEA2 review to claim that this document is not fit for used. Considering the impact in terms of delays on the project, I think that if you question the document, it is such an important matter that it should have been put on the table during the review.

Regards,

Gilles DALMAS

KlausRuedigerHase commented 11 years ago

The issue cannot be closed without improvements to the documents. The documents lacks justification of requirement items traced back to its original source as well as ranking of importance of each requirement. It does not matter why it was not done, it needs to be done in order to resolve problems in WP7, which otherwise are difficult or impossible to resolve. The FPP is only one of several sources. The FPP is the result of a common effort of all parties including those who complain about its insufficiency. The FPP however explains clearly that the main goal of openETCS is the implementation of "open proofs" for the ETCS onboard. That results into the "Mandatory" requirement of "open source software" for tools. The FPP also explains, why tools need to be open source with the need for "long term maintenance" due to the expected long lifetime of the ETCS and rail equipment in general. Until now there is no proprietary software concept available that can guarantee long term availability, see TOPCASED / OPEES (referenced in the FPP). Just to give one example for justification of requiremet items. This needs to be done for all requirement items. Again: More than enough reasons to improve the D2.x documents. The project does not need to start from zero, only because one or more documents need further improvements. Bug issues can only be closed once the bug is removed, therefore it was reopened.

sbaro commented 11 years ago

My position is still the same. Fitness of the requirements was demonstrated here by a "review by peer". I do not believe that traceability will help in this context.

To quote you:

One of the key advantages of an open project is the possibility of a "continues and fully independent peer review" since documents are open to the general public including all experts of that particular domain. Therefore nothing is really "final" (as in real life), anyway that is an opportunity for quality improvements not possible with closed projects.

We are in an open project context. So we cannot prevent people to add traceability if they want to do so.