Preservation of STIX content

[clenk]

The spec is ambiguous with regards to what STIX content needs to be preserved across implementations.

The now deprecated section 11 says

A consumer that receives STIX content containing Custom Properties, Objects, or Extensions it does not understand MAY refuse to process the content or MAY ignore those properties or objects and continue processing the content.

Section 7.3 says

Specific extensions, as with specific Custom Properties, MAY NOT be supported across implementations. A consumer that receives STIX content containing a STIX extension that it does not understand MAY refuse to process the content or MAY ignore that extension and continue processing the content.

But they don't address other STIX content, such as predefined objects or optional properties.

A table was sent to the mailing list with a table outlining different possible scenarios on which consensus should be arrived: https://urldefense.us/v3/__https:/lists.oasis-open.org/archives/cti/202104/msg00006.html__;!!BClRuOV5cvtbuNI!U1W06SWBdEqItvQPJ46AuOqy-XG-9oJOK5Ki-jSw-zPncaP1fi9wzz1NbVop6pLKDFh0yf0$

If someone parses an object and doesn't blow up but doesn't preserve an optional property when converting it to their internal data model and propogating the data, have they violated the spec?

This also touches on interop as it requires respondents be able to "parse and display" STIX content in several of its use cases.

---

Duplicate of #263.

[JasonKeirstead]

Sent this to the mailing list copying here.

The spec is very clear on this... if you are not the object producer, and yet you change the data in the object in any way when you retransmit, then you must generate a new ID for it. This includes dropping fields, adding fields, or any other change whatsoever.

"STIX Objects have a single object creator, the entity that generates the id for the object and creates the first version. The object creator MAY (but not necessarily will) be identified in the created_by_ref property of the object. Only the object creator is permitted to create new versions of a STIX Object. Producers other than the object creator MUST NOT create new versions of that object. If a producer other than the object creator wishes to create a new version, they MUST instead create a new object with a new id. They SHOULD additionally create a derived-from Relationship object to relate their new object to the original object that it was derived from."

TIPs and other systems that consume an SDO and then retransmit it later, but with different fields, MUST do so with new object IDs (which start over at version 1) - and ideally, a new created_by_ref (in the previous 2.0, interop tests, created_by_ref was required as part of interop)

If a system is simply dropping fields and re-transmitting with the same ID, then they are not compliant with the spec.

What a system that is not re-transmitting does internally with STIX after it is consumed, is totally out of scope of the spec, it is an implementation matter.

[jordan2175]

Looking at the conformance section it is silent on what to do with optional properties. It only defines required properties. To address this in Section 12.1 the consumer conformance clause, we would need to add a statement that says that consumers MUST support the optional properties that are defined in this specification. However, that would be a material change. If we end up making material change, then we should do this as part of that.

The proposed text would be something like:

It MUST support parsing all properties marked optional in the property table for the STIX Object that it consumes

[allant0]

If a source includes optional properties then they must be included if the entity that is redistributing them is doing so without recreating a new object with a new ID. For the original object, its ID and all properties (whether optional or not) the object in its entirety must be published as-is. This also includes any included extensions that might be added to the object also.

This is the intent of the specification. Any deviation from this would be a problem for multiple reasons.

[jordan2175]

Here is the response I sent to the email list.

This issue has been discussed at length over and over. The consensus of the TC is with the text as it is written. From a level of specificity the versioning rules are general and apply to all of STIX object. However, there is a caveat and additional clarification that was given to Custom Properties. That text was well understood, the ramification of it well understood, and what it meant was well understood and debated. This is how standards are written. Broad rules that cover everything and when needed specific sometimes other rules are given for very specific purposes. That was the case here.

It is also important to note that there was a very small and very vocal minority of people that disagreed with that decision that was made by the TC ever so long ago. But the majority made a decision and it has been in the document for a long time, early early on in STIX 2.0.

There are all kinds of issues in STIX that various people, even myself, do not like. If we could go back in time, I am sure we would change some of them. But that is the nature of consensus and community built things.

[jordan2175]

We talked about this on the call today, and decided to push any changes to address this to a future release.

[ejratl]

The STIX WG discussed this issue and the duplicate today. The outcome of the discussion is that while the specification may not contain text that explicitly addresses the question, the intent of the specification is nonetheless clear, especially in light of this discussion. Please reopen this issue if you disagree.