Should extensions be restrictable to an enclosing type?

Per some discussion I had with the cti-python-stix2 maintainers at oasis-open/cti-python-stix2#519, I'd like to get some clarification about how lenient (or restrictive) the spec is intended to be regarding the use of extensions (both those defined in the spec and custom as described in section 7.3) in specific object types.

Since the implementation of support for section 7.3 custom content in the Python library, it is possible to parse and serialise SCOs as defined in the spec with extensions which aren't "defined" for that SCO type. The specific example I gave in that issue is a file SCO which I added a unix-account-ext extensions to, which is obviously not meaningful. On the master branch, the following document will be validated without error, and can be constructed programmatically and serialized.

{
  "type": "file",
  "id": "file--48f88b7c-af82-410a-b1b4-a1a80a7f43ef",
  "name": "foo",
  "extensions": {
    "unix-account-ext": {
      "gid": 1001
    }
  }
}

From our readings of the spec, it seems like we (myself and the Python lib maintainers) are on the same page about it being reasonably lenient since there is nothing which specifically states that extensions are limited by their enclosing type, but that such leniency results in it being possible to create weird objects. A few choice quotes from the spec:

Specifies any extensions of the object, as a dictionary.

Dictionary keys SHOULD be the id of a STIX Extension object or the name of a predefined object extension found in this specification, depending on the type of extension being used.

-- section 3.2, table 1 defining the extensions common property

The wording here is lenient enough that it could be considered valid to include any extension which is not otherwise restricted in some way in an extensions dictionary. That seems sensible since we're defining the common property in section 3.2 and the definition needs to be valid for all instances of that dictionary in any document. I'm including it for context for the quotes that follow.

The File object defines the following extensions. In addition to these, producers MAY create their own.

ntfs-ext, raster-image-ext, pdf-ext, archive-ext, windows-pebinary-ext

Dictionary keys MUST use the specification defined name (examples above) or be the id of a STIX Extension object, depending on the type of extension being used.

The corresponding dictionary values MUST contain the contents of the extension instance.

*-- section 6.7.1, table 1 defining the extensions property for file type SCOs

To me this vaguely implies that the named extensions are only valid for file SCOs since "the File object defined [those] extensions", but @chisholm suggested [0] that a definition in the spec "defining" something itself doesn't seem meaningful and that since those extensions noted as valid for a file SCO are defined in the spec, they could be considered valid content anywhere that extensions can be included (ie. in any object where extensions are optional in section 3.2, table 2). This would then obviously extend to any extensions "defined by" other SCOs or SDOs. He might like to clarify something here if I've mis-read his interpretation.

As a note, the cti-stix2-json-schemas repo [1] appears to validate SCO-specific extensions by interpreting that they should only be valid for the SCO "defining them". e.g. for file objects those listed in the quote above are defined in the schema for that SCO type [2].

There are three ways to extend STIX using STIX Extensions.

Define one or more new STIX Object types.

Define additional properties for an existing STIX Object type as a nested property extension. This is typically done to represent a sub-component or module of one or more STIX Object types.

Define additional properties for an existing STIX Object type at the object's top-level. This can be done to represent properties that form an inherent part of the definition of an object type.

-- from section 7.3 defining custom extensions

These bullet points suggest to me that it may be the intention for custom extensions to also be potentially restricted to specific STIX object types, but I can find no discussion of a mechanism to describe that in the extension-definition document structure. If the expectation is that this should be validated in some way by the extension-definition's schema, it's not particularly well described and no examples appear to be given.

@rpiazza suggested that the use of the extension-definition schema might be a possible mechanism for enforcement of encapsulating STIX object type(s), but the only way I can think to sensibly define that schema would be for the extension content itself rather than the structure of a top-level object containing such an extension. However, this means that there would be no way for me write a schema which validated top-level properties added by a top-level-property extension-definition, since I would be providing a schema only for the data below the extensions/extension-definition--<uuid> path of the object. I'm unclear on exactly what the schema for an extension-definition is intended to validated and how it should hook into other schemas. Without examples of JSON schemas for the extension-definition's describe in section 7.3 and appendix C2, it's hard to be sure of the intent and available mechanisms.

If possible, it would be helpful for someone(s) from the TC to clarify the intent regarding extensions and encapsulating types. It would also be great to get some advice (or even better, some schema examples) about how custom extension validation is expected to function and whether encapsulating types can be enforced for them.

===== 8< ===== [0] https://github.com/oasis-open/cti-python-stix2/issues/519#issuecomment-876850317 [1] https://github.com/oasis-open/cti-stix2-json-schemas [2] https://github.com/oasis-open/cti-stix2-json-schemas/blob/12a05130e1575a69e5aaf5940c5afd42aade93d5/schemas/observables/file.json

oasis-tcs / cti-stix2

Should extensions be restrictable to an enclosing type? #276