ucoProject / UCO

This repository is for development of the Unified Cyber Ontology.
Apache License 2.0
80 stars 34 forks source link

UCO semi-open vocabularies need to become enumerations of `xsd:string`s #629

Open ajnelson-nist opened 3 months ago

ajnelson-nist commented 3 months ago

Background

UCO gives sets of strings that suggest values to use for certain properties. UCO has called this the "Semi-open vocabulary" design pattern: While certain strings should be used, strings not included in UCO's ontology files can still be used instead when the user knows the value they are using is not yet in UCO, and/or might never be in UCO for whatever reason (e.g., confidentiality of non-public vocabularies).

UCO uses a syntax for its semi-open vocabularies that uses what has appeared to date to be an OWL design pattern for defining a datatype as a set of typed string values. However, it turns out the pattern used by UCO is invalid OWL syntax. Issue 593 covers half of the issues raised by an externally-developed OWL syntax reviewing tool, ROBOT[^1]. Issue 593's correction has no known impact on user data.

The other half of the syntax correction takes this form, which unfortunately differs from UCO's user data guidance since UCO's earliest examples. Given the known non-0 impact on user data, this is being treated as a non-fast-track proposal.

Here is (again, selected for its small size) the Bitness vocabulary, as implemented in UCO 1.3.0 and long before:

vocabulary:BitnessVocab
    a rdfs:Datatype ;
    rdfs:label "Bitness Vocabulary"@en-US ;
    rdfs:comment "Defines an open-vocabulary of word sizes that define classes of operating systems."@en ;
    owl:equivalentClass [
        a rdfs:Datatype ;
        owl:onDatatype xsd:string ;
        owl:oneOf (
            "32"^^vocabulary:BitnessVocab
            "64"^^vocabulary:BitnessVocab
        ) ;
    ] ;
    .

Issue 593 cut the triple [] owl:onDatatype xsd:string .. This Issue finishes the OWL syntax correction by changing the members to values typed only as xsd:string (which is the OWL-wide (IIRC, RDF-wide) default when no other type is stated):

vocabulary:BitnessVocab
        a rdfs:Datatype ;
        rdfs:label "Bitness Vocabulary"@en-US ;
        rdfs:comment "Defines an open-vocabulary of word sizes that define classes of operating systems."@en ;
        owl:equivalentClass [
                a rdfs:Datatype ;
                owl:onDatatype xsd:string ;
                owl:oneOf (
                        "32"
                        "64"
                ) ;
        ] ;
        .

The syntax error this addresses is that the enumeration attempted to assign a datatype to the enumeration's member-literals; but, that datatype is not yet defined at the time the enumeration is being consumed by the parsing process, because they are inside the referenced datatype's definition. ROBOT describes this as a cyclic reference in a definition, and calls it a syntax error. The "Bug description" section below goes into the details describing why the cycle is also a symptom of another syntax error.

Requirements

Requirement 1

UCO vocabularies must adhere to OWL syntactic requirements for custom datatypes.

Requirement 2

Any UCO vocabulary X must not use itself as a datatype in its enumerated literals, due to OWL syntax requirements (cited in "Bug description" section).

Risk / Benefit analysis

Benefits

Risks

Competencies demonstrated

Competency 1

A hash is being migrated from UCO 1.3.0 to UCO 1.4.0. The form in UCO 1.3.0 is:

{
    "@context": {
        "kb": "http://example.org/kb/",
        "types": "https://ontology.unifiedcyberontology.org/uco/types/",
        "vocabulary": "https://ontology.unifiedcyberontology.org/uco/types/"
    },
    "@graph": {
        "@id": "kb:Hash-9ed55c42-3204-4e45-ace7-7ae6bd7d8f38",
        "@type": "types:Hash",
        "types:hashMethod": {
            "@type": "vocabulary:HashNameVocab",
            "@value": "SHA3-256"
        },
        "types:hashValue": {
            "@type": "xsd:hexBinary",
            "@value": "36f028580bb02cc8272a9a020f4200e346e276ae664e45ee80745574e2f5ab80"
        }
    }
}

Competency Question 1.1

How is the hash supposed to be spelled with this proposal?

Result 1.1

Note that two lines are dropped: The "vocabulary" prefix from the context dictionary will never be needed by users, and hashMethod no longer needs the typed-literal JSON object in its spelling.

{
    "@context": {
        "kb": "http://example.org/kb/",
        "types": "https://ontology.unifiedcyberontology.org/uco/types/"
    },
    "@graph": {
        "@id": "kb:Hash-9ed55c42-3204-4e45-ace7-7ae6bd7d8f38",
        "@type": "types:Hash",
        "types:hashMethod": "SHA3-256",
        "types:hashValue": {
            "@type": "xsd:hexBinary",
            "@value": "36f028580bb02cc8272a9a020f4200e346e276ae664e45ee80745574e2f5ab80"
        }
    }
}

Competency Question 1.2

How is a hash using an algorithm name not in UCO supposed to be spelled?

Result 1.2

# UCO 1.4.0 form
kb:Hash-9ed55c42-3204-4e45-ace7-7ae6bd7d8f38
    a types:Hash ;
    types:hashMethod "FooHash" ;
    types:hashValue "098f6bcd4621d373cade4e832627b4f6"^^xsd:hexBinary ;
    .

Competency Question 1.3

How should semi-open vocabulary properties' ranges be spelled?

Result 1.3

These properties' ranges should continue to be spelled as the union of xsd:string and the UCO Vocabulary IRI. The OWL 2 Syntax document, Section 9.4 gives an example for a restricted string-property. UCO's semi-open vocabulary still permits general xsd:string, so an OWL Union'd range should be used in owl:DatatypePropertys, and vocabulary membership should be reviewed with SHACL.

Solution suggestion

No changes are necessary to the owl:DatatypeProperty OWL definitions currently using the semi-open vocabulary pattern, taking for example types:hashMethod:

types:hashMethod
    a owl:DatatypeProperty ;
    rdfs:label "hashMethod"@en ;
    rdfs:comment "A particular cryptographic hashing method (e.g., MD5)."@en ;
    rdfs:range [
        a rdfs:Datatype ;
        owl:unionOf (
            vocabulary:HashNameVocab
            xsd:string
        ) ;
    ] ;
    .

The SHACL shapes for the semi-open vocabulary pattern have been coming as trios of property shapes. They need to be adjusted as follows for the implementing next UCO SEMVER-minor release, so current data raises SHACL warnings instead of errors. Note that the sh:Info-severity "gentle suggestion" shape is now the shape that bears the enumeration.

types:Hash
    sh:property
        [
            sh:datatype xsd:string ;
            sh:message "As of UCO 1.4.0, the datatype to use for types:hashMethod should be xsd:string.  Not using xsd:string will be an error in UCO 2.0.0." ;
            sh:path types:hashMethod ;
            sh:severity sh:Warning ;
        ] ,
        [
            sh:maxCount "1"^^xsd:integer ;
            sh:minCount "1"^^xsd:integer ;
            sh:nodeKind sh:Literal ;
            sh:path types:hashMethod ;
        ] ,
        [
            sh:message "Value is not member of the vocabulary HashNameVocab." ;
            sh:in (
                "MD5"
                "MD6"
                "SHA1"
                "SHA224"
                "SHA256"
                "SHA3-224"
                "SHA3-256"
                "SHA3-384"
                "SHA3-512"
                "SHA384"
                "SHA512"
                "SSDEEP"
            ) ;
            sh:path types:hashMethod ;
            sh:severity sh:Info ;
        ]
        ;
    .

(An aside: the new sh:Warning shape makes some new cases that previously failed now pass as warnings, like using an integer for hashMethod's value. Given the warnings, this feels like a low-risk relaxation.)

For UCO 2.0.0, the first two shapes would combine, leaving the implementation as follows.

types:Hash
    sh:property
        [
            sh:datatype xsd:string ;
            sh:maxCount "1"^^xsd:integer ;
            sh:minCount "1"^^xsd:integer ;
            sh:nodeKind sh:Literal ;
            sh:path types:hashMethod ;
        ] ,
        [
            sh:message "Value is not member of the vocabulary HashNameVocab." ;
            sh:in (
                "MD5"
                "MD6"
                "SHA1"
                "SHA224"
                "SHA256"
                "SHA3-224"
                "SHA3-256"
                "SHA3-384"
                "SHA3-512"
                "SHA384"
                "SHA512"
                "SSDEEP"
            ) ;
            sh:path types:hashMethod ;
            sh:severity sh:Info ;
        ]
        ;
    .

Due to a processing efficiency issue, one final modification from current practice is suggested: The "Gentle suggestion" property shape should change from an inlined, blank-node-identified sh:PropertyShape to a class-independent, IRI-identified sh:NodeShape that uses the targeting statement ... sh:targetObjectsOf types:hashMethod . The reason for this is that certain SHACL reporting practices, like those used in UCO's unit testing, inline all blank node property shapes in the SHACL result-sets. For some vocabularies, especially longer ones that suggest intensive extension (in particular, the 200+-member vocabulary:ObservableObjectRelationshipVocab), this could be hopelessly obscuring in the result sets whenever users engage in local vocabulary extension.

This pattern is suggested for the implementation instead, to offer users (1) a reference to an IRI they can look up on the documentation site instead of getting an inlined and long anonymous shape, and (2) the opportunity to opt out of any specific "Gentle suggestion" shapes by using sh:deactivate:

types:hashMethod-objects-in-shape
    a sh:NodeShape ;
    sh:in (
        "MD5"
        "MD6"
        "SHA1"
        "SHA224"
        "SHA256"
        "SHA3-224"
        "SHA3-256"
        "SHA3-384"
        "SHA3-512"
        "SHA384"
        "SHA512"
        "SSDEEP"
    ) ;
    sh:message "Value is not member of the vocabulary HashNameVocab." ;
    sh:severity sh:Info ;
    sh:targetObjectsOf types:hashMethod ;
    .

The replacement for the class's sh:property links would be as follows, using an rdfs:seeAlso to preserve some reference on the documentation website without any other mechanical impact:

 types:Hash
+   rdfs:seeAlso types:hashMethod-objects-in-shape ;
    sh:property
        [
            sh:datatype xsd:string ;
            sh:maxCount "1"^^xsd:integer ;
            sh:minCount "1"^^xsd:integer ;
            sh:nodeKind sh:Literal ;
            sh:path types:hashMethod ;
-       ] ,
-       [
-           sh:message "Value is not member of the vocabulary HashNameVocab." ;
-           sh:in (
-               "MD5"
-               "MD6"
-               "SHA1"
-               "SHA224"
-               "SHA256"
-               "SHA3-224"
-               "SHA3-256"
-               "SHA3-384"
-               "SHA3-512"
-               "SHA384"
-               "SHA512"
-               "SSDEEP"
-           ) ;
-           sh:path types:hashMethod ;
-           sh:severity sh:Info ;
        ]
        ;
    .

If a user wishes to opt out of the gentle suggestions, they can add this triple in their extension SHACL shapes:

types:hashMethod-objects-in-shape
    sh:deactivated true ;
    .

Bug description

While at least one tool reports that the current UCO vocabulary pattern is a cyclic definition error, the OWL syntax specification and RDF mapping documents together describe the problem as an inappropriate use of an OWL structure, DatatypeDefinition.

Looking through the OWL 2 Mapping to RDF document, Table 16 includes this mapping:

IF this pattern is in the RDF graph G:

*:x owl:equivalentClass y .
{ DR(*:x) ≠ ε amd DR(y) ≠ ε }

THEN the following axiom is added to the ontology OG:

DatatypeDefinition( DR(*:x) DR(y) )

For quick reference, the smbols above mean:

Taken together, owl:equivalentClass as a predicate between two certain nodes pertaining to DataRanges imply a DatatypeDefinition.

*:x being in a DatatypeDefinition imposes a certain constraint on *:x, with which UCO has been incongruous to date. Section 9.4 of the OWL 2 Syntax document defines that in the structure DatatypeDefinition ( DT DR ), the Datatype that is the first parameter (DT, or *:x, or uco-vocabulary:BitnessVocab) is a "synonym for DR..." And: "The datatypes defined by datatype definition axioms ... have empty lexical spaces and therefore they must not occur in literals."

The example in the OWL 2 syntax document that immediately follows that quote illustrates a datatype a:SSN, which is part of a datatype definition that would be spelled like this in Turtle:

# First axiom: "Declaration( Datatype( a:SSN ) )"
a:SSN
    a rdfs:Datatype ;
    .

# Second axiom: "DatatypeDefinition( ..."
a:SSN
    owl:equivalentClass [
        a rdfs:Datatype ;
        owl:onDatatype xsd:string ;
        owl:withRestrictions (
            [
                xsd:pattern "[0-9]{3}-[0-9]{2}-[0-9]{4}" ;
            ]
        ) ;
    ] ;
    .

# Third axiom: "DataPropertyRange( a:hasSSN a:SSN )"
a:hasSSN
    rdfs:range a:SSN ;
    .

Note the end of the example block states "there can be no literals of datatype a:SSN".

In UCO's case, there can be no literals of datatype vocabulary:BitnessVocab, vocabulary:HashNameVocab, etc.

Coordination