search

KhronosGroup / KSCAF_DocGuidelines

Khronos Safety Critical Advisory Forum’s guidelines for developing a safety critical technology specification.
1 stars 2 forks source link

Implementation details: impact on testability #34

Open bnaodovic opened 6 years ago

bnaodovic commented 6 years ago

Implementations in the safety environment have a requirement for a significantly more detailed and exhaustive testing than in the non-safety environment. If an API function has parameters with very broad ranges, then their impact on testability should be discussed. Similarly, if a function has parameters for which allowed and disallowed sets of values are hard to determine, there should be discussion in the API specification how tests such as fault injection should be handled with respect to the safety standards that require such tests. In addition to issues for implementers, the mentioned examples of parameters can cause issues to users as well, since coding guidelines, such as MISRA guidelines, usually have a rule requiring that validity of parameters passed to library functions be checked. This could lead to different implementers performing this validity check in different ways, resulting in different validation accuracy, yet all of them be given the same approval of conformance to the API specification.

irudkin commented 6 years ago

Need to turn this into a set of individual guidelines

bnaodovic commented 6 years ago

Format suitable for the guidelines document:

SC profile shall discuss impact of functions with broad-range parameters on testability of the implementation of the API.

Rationale: Implementations in the safety environment have a requirement for a significantly more detailed and exhaustive testing than in the non-safety environment. If an API function has a parameter with a broad range of possible values, then it may be infeasible to implement all the testing specified by various safety standards to which the API implementation may need to adhere to. If no alternatives for the function exist with narrow-range parameters, then there shall be reasons listed in a discussion section why no alternative exists. This section can then be referenced in safety reports of the API implementations.

Example: [TODO]

SC profile shall discuss impact of functions with parameters for which allowed and disallowed sets of values are hard to determine on testability of the implementation of the API.

Rationale: Implementations in the safety environment have a requirement for a significantly more detailed and exhaustive testing than in the non-safety environment. If an API function has a parameter for which allowed and disallowed sets of values are hard to determine, then it may be infeasible to implement all the testing specified by various safety standards to which the API implementation may need to adhere to. If no alternatives for the function exist with parameter for which the mentioned sets can be feasibly determined, then there shall be reasons listed in a discussion section why no alternative exists. This section can then be referenced in safety reports of the API implementations.

Example: [TODO]

SC profile should discuss impact of functions with broad-range parameter or parameters on testability of user code calling those functions.

Rationale: A typical requirement for development in the safety environment is for parameter values passed to library functions be tested for correctness. If there is user code calling API function with a broad-range parameter, then it may be infeasible to implement all the testing specified by various safety standards to which the user code may need to adhere to. If no alternatives for the function exist with narrow-range parameters, then there should be reasons listed in a discussion section why no alternative exists. This section could then be referenced in safety report which encompasses the mentioned code.

Example: [TODO]

SC profile should discuss impact of functions with parameter or parameters for which allowed and disallowed sets of values are hard to determine on testability of user code calling those functions.

Rationale: A typical requirement for development in the safety environment is for parameter values passed to library functions be tested for correctness. If there is user code calling API function with parameter for which allowed and disallowed sets of values are hard to determine, then it may be infeasible to implement all the testing specified by various safety standards to which the code may need to adhere to. If no alternatives for the function exist with parameter for which the mentioned sets can be feasibly determined, then there should be reasons listed in a discussion section why no alternative exists. This section can then be referenced in safety report which encompasses the mentioned code.

Example: [TODO]