search

mitre-attack / attack-stix-data

STIX data representing MITRE ATT&CK
https://attack.mitre.org/
Other
348 stars 87 forks source link

STIX versioning appears to be broken across all current versions of the ATT&CK JSON #20

Open agfoster opened 2 years ago

agfoster commented 2 years ago

I believe STIX versioning isn't properly implemented/applied to whatever is generating the current ATT&CK JSON.

From section 3.6 Versioning of the STIX 2.1 spec:

STIX Objects MAY be versioned in order to update, add, or remove information. A version of a STIX Object is identified uniquely by the combination of its id and modified properties. The first version of the object MUST have the same timestamp for the created and modified properties. More recent values of the modified property indicate later versions of the object. Implementations MUST consider the version of the STIX Object with the most recent modified value to be the most recent state of the object. For every new version of an object, the modified property MUST be updated to represent the time that the new version was created. If a consumer receives two objects that are different, but have the same id and modified timestamp, it is not defined how the consumer handles the objects. This specification does not address how implementations should handle versions of the object that are not current.

There are 422 attack-patterns in the current JSON that have non-matching creation/modification times and are missing the required revoked property. There are 297 attack-patterns that do have the revoked property. The first attack-pattern (['objects'][1]) in the is an example of an object that is clearly versioned, but doesn't have the requisite properties.

attack-pattern--0042a9f5-f053-4769-b3ef-9ad018dfa298 is versioned but missing 'revoked': created: 2020-01-14T17:18:32.126Z modified 2022-04-25T14:00:00.188Z

['objects'][1].keys() ['x_mitre_platforms', 'x_mitre_domains', 'object_marking_refs', 'id', 'type', 'created', 'created_by_ref', 'external_references', 'modified', 'name', 'description', 'kill_chain_phases', 'x_mitre_detection', 'x_mitre_is_subtechnique', 'x_mitre_version', 'x_mitre_modified_by_ref', 'x_mitre_data_sources', 'x_mitre_defense_bypassed', 'spec_version', 'x_mitre_attack_spec_version']

agfoster commented 2 years ago

I don't think this has always been this way. The last ATT&CK JSON I worked with extensively was either 8 or 9, and I'm fairly certain that at the time STIX versioning was working. It looks like the various versions of the JSON have been regenerated/modified, and I no longer have access to reference copies to compare.

While this isn't necessarily workflow breaking, it does seem that there is a mix of properly versioned & non-versioned objects intermingled, and some of these objects might or might not have a x_mitre meta versioning. It's a bit confusing and frustrating.

agfoster commented 2 years ago

Taking a look at version 1.0 of the ATT&CK JSON, I get the impression that the "created" time is when the technique was created, not when the STIX object was created. For example:

id, created, modified ('attack-pattern--01df3350-ce05-4bdf-bdf8-0a919a66d4a8', '2017-12-14T16:46:06.044Z', '2018-01-17T12:56:55.080Z') ('attack-pattern--dcaa092b-7de9-4a21-977f-7fcb77e89c48', '2017-12-14T16:46:06.044Z', '2018-01-17T12:56:55.080Z') ('attack-pattern--9b99b83a-1aac-4e29-b975-b374950551a3', '2017-05-31T21:30:26.946Z', '2018-01-17T12:56:55.080Z') .........182 more objects...... ('attack-pattern--e906ae4d-1d3a-4675-be23-22f7311c0da4', '2017-05-31T21:31:05.140Z', '2018-01-17T12:56:55.080Z') ('attack-pattern--c3bce4f4-9795-46c6-976e-8676300bbc39', '2017-05-31T21:30:33.723Z', '2018-01-17T12:56:55.080Z') ('attack-pattern--514ede4c-78b3-4d78-a38b-daddf6217a79', '2017-05-31T21:30:20.148Z', '2018-01-17T12:56:55.080Z')

According to the STIX spec, The created property represents the time at which the object was originally created. and looking at the timestamps from version 1.0, I get the feeling that the created values were scraped from another database & the modify time is whatever time it happened to be when the script started.

Either way - there are 188 objects in ATT&CK json version 1.0 that imply versioning but don't have requisite properties.

agfoster commented 2 years ago

There are 188 attack patterns in ATT&CK 1.0 that aren't using STIX 2.1 versioning properly. ('attack-pattern--01df3350-ce05-4bdf-bdf8-0a919a66d4a8', '2017-12-14T16:46:06.044Z', '2018-01-17T12:56:55.080Z') ('attack-pattern--dcaa092b-7de9-4a21-977f-7fcb77e89c48', '2017-12-14T16:46:06.044Z', '2018-01-17T12:56:55.080Z') ('attack-pattern--9b99b83a-1aac-4e29-b975-b374950551a3', '2017-05-31T21:30:26.946Z', '2018-01-17T12:56:55.080Z')

There are 188 attack patterns in ATT&CK 2.0 that aren't using STIX 2.1 versioning properly. ('attack-pattern--01df3350-ce05-4bdf-bdf8-0a919a66d4a8', '2017-12-14T16:46:06.044Z', '2018-04-18T17:59:24.739Z') ('attack-pattern--dcaa092b-7de9-4a21-977f-7fcb77e89c48', '2017-12-14T16:46:06.044Z', '2018-04-18T17:59:24.739Z') ('attack-pattern--9b99b83a-1aac-4e29-b975-b374950551a3', '2017-05-31T21:30:26.946Z', '2018-04-18T17:59:24.739Z')

There are 219 attack patterns in ATT&CK 3.0 that aren't using STIX 2.1 versioning properly. ('attack-pattern--01df3350-ce05-4bdf-bdf8-0a919a66d4a8', '2017-12-14T16:46:06.044Z', '2018-10-17T00:14:20.652Z') ('attack-pattern--dcaa092b-7de9-4a21-977f-7fcb77e89c48', '2017-12-14T16:46:06.044Z', '2018-10-17T00:14:20.652Z') ('attack-pattern--9b99b83a-1aac-4e29-b975-b374950551a3', '2017-05-31T21:30:26.946Z', '2018-10-17T00:14:20.652Z')

There are 244 attack patterns in ATT&CK 4.0 that aren't using STIX 2.1 versioning properly. ('attack-pattern--01df3350-ce05-4bdf-bdf8-0a919a66d4a8', '2017-12-14T16:46:06.044Z', '2018-10-31T13:45:13.024Z') ('attack-pattern--dcaa092b-7de9-4a21-977f-7fcb77e89c48', '2017-12-14T16:46:06.044Z', '2018-10-31T13:45:13.024Z') ('attack-pattern--9b99b83a-1aac-4e29-b975-b374950551a3', '2017-05-31T21:30:26.946Z', '2018-10-17T00:14:20.652Z')

There are 244 attack patterns in ATT&CK 5.0 that aren't using STIX 2.1 versioning properly. ('attack-pattern--01df3350-ce05-4bdf-bdf8-0a919a66d4a8', '2017-12-14T16:46:06.044Z', '2018-10-31T13:45:13.024Z') ('attack-pattern--dcaa092b-7de9-4a21-977f-7fcb77e89c48', '2017-12-14T16:46:06.044Z', '2018-10-31T13:45:13.024Z') ('attack-pattern--9b99b83a-1aac-4e29-b975-b374950551a3', '2017-05-31T21:30:26.946Z', '2018-10-17T00:14:20.652Z')

There are 266 attack patterns in ATT&CK 6.0 that aren't using STIX 2.1 versioning properly. ('attack-pattern--01df3350-ce05-4bdf-bdf8-0a919a66d4a8', '2017-12-14T16:46:06.044Z', '2019-06-13T14:49:56.024Z') ('attack-pattern--dcaa092b-7de9-4a21-977f-7fcb77e89c48', '2017-12-14T16:46:06.044Z', '2019-10-14T20:45:04.451Z') ('attack-pattern--9b99b83a-1aac-4e29-b975-b374950551a3', '2017-05-31T21:30:26.946Z', '2019-07-16T19:07:04.652Z')

There are 574 attack patterns in ATT&CK 10.0 that aren't using STIX 2.1 versioning properly. ('attack-pattern--d0b4fcdb-d67d-4ed2-99ce-788b12f8c0f4', '2020-02-11T18:46:56.263Z', '2021-04-29T14:49:39.188Z') ('attack-pattern--cabe189c-a0e3-4965-a473-dcff00f17213', '2020-10-15T12:05:58.755Z', '2021-07-28T01:04:39.141Z') ('attack-pattern--3986e7fd-a8e9-4ecb-bfc6-55920855912b', '2020-08-24T13:43:00.028Z', '2021-06-07T19:23:33.039Z')

There are 422 attack patterns in ATT&CK 11.2 that aren't using STIX 2.1 versioning properly. ('attack-pattern--0042a9f5-f053-4769-b3ef-9ad018dfa298', '2020-01-14T17:18:32.126Z', '2022-04-25T14:00:00.188Z') ('attack-pattern--01327cde-66c4-4123-bf34-5f258d59457b', '2020-02-11T18:28:44.950Z', '2022-04-25T14:00:00.188Z') ('attack-pattern--0259baeb-9f63-4c69-bf10-eb038c390688', '2017-05-31T21:31:25.060Z', '2021-04-29T14:49:39.188Z')