search

hollobit / WG3_TCM

a matrix to provide the clarified definition and relationship information of trustworthiness characteristics between in the AI/ML standards
3 stars 0 forks source link

No definitions for characteristics #12

Open hollobit opened 2 years ago

hollobit commented 2 years ago

Below characteristics didn't have any definition:

paulbrucecotton commented 2 years ago

I have several responses to this list of missing definitions.

  1. At least the "robustness" and "transparency" characteristics are defined in DTS 5723 (see WG3 N2729):

3.1.1 trustworthiness ability to meet stakeholders’ (3.3.9) expectations in a verifiable (3.3.12) way Note 1 to entry: Depending on the context or sector, and also on the specific product or service, data, technology and process used, different characteristics apply and need verification to ensure stakeholders’ expectations are met. Note 2 to entry: Characteristics of trustworthiness include, for instance, accountability (3.2.1), accuracy (3.2.2), authenticity (3.2.3), availability (3.2.4), controllability (3.2.5), integrity (3.2.7, 3.2.8), privacy (3.2.9), quality (3.2.10, 3.2.11), reliability (3.2.12, 3.2.13), resilience (3.2.14, 3.2.15), robustness (3.2.16), safety (3.2.17), security (3.2.18), transparency (3.2.19, 3.2.20) and usability (3.2.21). Note 3 to entry: Trustworthiness is an attribute that can be applied to services, products, technology, data and information as well as to organizations. Note 4 to entry: Verifiability (3.3.12) includes measurability (Error: Reference source not found) and demonstrability by means of objective evidence

  1. "oversight" is defined in ISO/IEC 38507:2022:

3.2 Terms related to governance 3.2.1 oversight monitoring of the implementation of organizational and governance policies and management of associated tasks, services and products set by the organization, in order to adapt to changes in internal or external circumstances

Note 1 to entry: Effective oversight needs general understanding of a situation. Oversight is one of the ‘principles of governance’ covered in depth in ISO 37000:2021, 6.4

  1. "certification"

Instead of "certification" we should be using "conformity" of "conformity assessment". See the ISO House style:

CONFORMITY AND CONFORMITY-RELATED TERMS The word “compliance” is used in connection with legislation and regulations, which are not generally mentioned in ISO documents. See Legal statements.

Use the word “conformity” when referring to requirements. The phrase “in accordance with” can also be used for expressing conformity-related requirements. For example:

“Interlocking and guard locking devices shall conform to ISO 14119.”

“Ports and stud ends conforming to this document are not intended to connect with ports and stud ends that conform to ISO 1179 or threads that conform to ISO 7-1.”

“The relative density of the hose inner tube shall not exceed 2,155 when tested in accordance with ISO 7258.”

ISO/CASCO, the Committee on conformity assessment, has produced definitions for conformity-related terminology, e.g. “competence” and “assessment”. If there is a need to define this kind of terminology in a standard, search the Online Browsing Platform to find the CASCO definition to use as a source. Do not develop a new definition for the document. Contact CASCO at an early stage when developing a standard containing conformity assessment elements: CASCO@iso.org

For example see related definitions in ISO/IEC 17000:2020, Conformity assessment — Vocabulary and general principles: https://www.iso.org/obp/ui/#iso:std:iso-iec:17000:ed-2:v2:en

4.1 conformity assessment demonstration that specified requirements (5.1) are fulfilled

Note 1 to entry: The process of conformity assessment as described in the functional approach in Annex A can have a negative outcome, i.e. demonstrating that the specified requirements are not fulfilled. Note 2 to entry: Conformity assessment includes activities defined elsewhere in this document, such as but not limited to testing (6.2), inspection (6.3), validation (6.5), verification (6.6), certification (7.6), and accreditation (7.7). Note 3 to entry: Conformity assessment is explained in Annex A as a series of functions. Activities contributing to any of these functions can be described as conformity assessment activities. Note 4 to entry: This document does not include a definition of “conformity”. “Conformity” does not feature in the definition of “conformity assessment”. Nor does this document address the concept of compliance.

/paulc

harm-ellens commented 2 years ago

Controlability and Human Oversight are part of the new proposed PWI in WG3, but it doesn't have its own project number yet

Human dignity has been defined by UNESCO and OECD - they may be useful starting points

Testing has just been started under SC42 JWG2

Robustness is covered in 24029-2 and offers this introduction, although not a definition:

In the context of neural networks, robustness specifications typically represent different conditions 245 that can naturally or adversarially change in the domain (see Clause 5.2) in which the neural 246 network is deployed.

paulbrucecotton commented 2 years ago

Robustness is defined in ISO/IEC 22989 which is imported by the parts of the ISO/IEC 24029 series:

3.5.12
robustness ability of a system to maintain its level of performance under any circumstances

/paulc

hollobit commented 2 years ago

I've added definitions brought from 22989

Also, added definitions