shadazar
commented
7 years ago This text has been incorporated in to the document. Closing issue.
Closed DeOrellana closed 7 years ago
shadazar
commented
7 years ago This text has been incorporated in to the document. Closing issue.
Erik Noreke 2016-08-30 00:54:04 PDT
All behavior of the API must be testable. Every item mentioned in the specification must be able to be tested in a repeatable manner. An undeterministic API cannot be fully tested as behavior may change from run to run.
Comment 1 Erik Noreke 2016-09-12 07:37:45 PDT
Assigned per WG call 2016-09-12
Comment 2 Erik Noreke 2016-09-20 13:07:29 PDT
Setting QA contact to non-member SCAP mailing list.
Comment 3 SteveRamm 2016-10-03 01:24:07 PDT
Just a comment: A requirement may be fully testable but not testable within the scope of Khronos conformance tests. Therefore an SC API should contain a rider, or Khronos should advertise a general rider that if an API is accepted as conformant this does not guarantee that all requirements are met. Examples of such a requirements: "An implementation must provide documentation of the values of error codes and the circumstances under which they are returned" This is testable by inspection of the documentation, but not testable by Khronos conformance tests. "The implementation must not use hardware interrupts" Again, this is testable (by running with interrupts disabled), but presents difficulties with conformance tests which are necessarily hardware agnostic.
Comment 4 Erik Noreke 2016-10-03 07:24:32 PDT
Assigning to Illya to wordsmith 20161003
Comment 5 illya@codeplay.com 2016-10-10 03:50:38 PDT
Related to but not the same as Khronos Conformance Test(s) for SC API #15993
Comment 6 illya@codeplay.com 2016-11-15 07:31:41 PST
Scope All aspects of the API; the functions, the combinations of the functions’ parameters, all enumeration values, all possible return codes specified, must be testable in a deterministic and repeatable manner irrespective of how all and each function or task is implemented. Being fully testable, deterministic and repeatable also extends to functions which are expected to be re-entrant or thread-safe.
For the case of negative testing (data values which are known to cause failure) this requirement still applies.
For the cases of undefined behaviour, the entering in to an undefined state or safe operational mode this requirement still applies. See x.x.x. Underfined behaviour.
Note 1: An SC API implementation while it is conformant to all Khronos conformant tests cannot be accepted as all the item’s requirements have been met where additional requirements have been specified.
(or) Note 2. The Khronos conformant tests cannot seen as a complete and satisfactory set of requirements for an item used in a safety context. Additional requirements and tests are necessary.
Comment 7 illya@codeplay.com 2016-11-16 00:34:04 PST
Comment 6: This is penciled in, to be discussed. Notes 1 and 2 were penciled in because of Steve Ramm's valid comment 3. The Notes could indeed be replaced with a general rider comment else where (but where?). Idea was have Note 1 or 2 but not both.