himynamesdave
commented
3 months ago @fqrious on Slack you said
"Similarly, when a record is removed from a source object (e.g ATT&CK reference removed from a CAPEC object), the object removed between updates is marked at _is_latest=false, but no new object recreated for it (because it no longer exist in latest version of source object)"
This affects multiple projects, for example ATS is supposed to only return objects where _is_latest==true on manifest/objects, and if this is implemented, old SROs will just be missing from the manifest (unless version is set to all in query)
This is not quite right...
stix2arango logic is fairly simplistic. If md5 of object changes, add the new one (as _is_latest=true), and make is _is_latest=false for all old versions.
arango_taxii_server is slightly different...
Here it is only concerned with creating relationships (SROs) between objects being changed (by stix2arango imports)
Lets me use an example (test 1.3 https://github.com/muchdogesec/arango_cti_processor/blob/adding-tests/tests/README.md#test-13-perform-another-update-to-change-capec-attack-pattern---attck-attack-pattern-relationship-capec-attack)
1.2
"external_references": [
{
"external_id": "CAPEC-158",
"source_name": "capec",
"url": "https://capec.mitre.org/data/definitions/158.html"
},
{
"external_id": "CWE-311",
"source_name": "cwe",
"url": "http://cwe.mitre.org/data/definitions/311.html"
},
{
"description": "Network Sniffing",
"external_id": "T1040",
"source_name": "ATTACK",
"url": "https://attack.mitre.org/wiki/Technique/T1040"
},
{
"description": "Multi-Factor Authentication Interception",
"external_id": "T1111",
"source_name": "ATTACK",
"url": "https://attack.mitre.org/wiki/Technique/T1111"
},
{
"description": "Acquire Access",
"external_id": "T1650",
"source_name": "ATTACK",
"url": "https://attack.mitre.org/wiki/Technique/T1650"
},
{
"description": "Hijack Execution Flow: ServicesFile Permissions Weakness",
"external_id": "T1574.010",
"source_name": "ATTACK",
"url": "https://attack.mitre.org/wiki/Technique/T1574/010"
}
],
"id": "attack-pattern--897a5506-45bb-4f6f-96e7-55f4c0b9021a",
"modified": "2024-01-01T00:00:00.000Z",1.3
"external_references": [
{
"external_id": "CAPEC-158",
"source_name": "capec",
"url": "https://capec.mitre.org/data/definitions/158.html"
},
{
"external_id": "CWE-311",
"source_name": "cwe",
"url": "http://cwe.mitre.org/data/definitions/311.html"
},
{
"description": "Network Sniffing",
"external_id": "T1040",
"source_name": "ATTACK",
"url": "https://attack.mitre.org/wiki/Technique/T1040"
},
{
"description": "Multi-Factor Authentication Interception",
"external_id": "T1111",
"source_name": "ATTACK",
"url": "https://attack.mitre.org/wiki/Technique/T1111"
}
],
"id": "attack-pattern--897a5506-45bb-4f6f-96e7-55f4c0b9021a",
"modified": "2024-01-15T00:00:00.000Z",
"name": "UPDATE OBJECT 3RD TIME",2 of the ATT&CK references inside the CAPEC object (attack-pattern--897a5506-45bb-4f6f-96e7-55f4c0b9021a) are removed between 1.2 and 1.3 with 2 remaining T1040 and T1111 (total 4 ATT&CK links). T1650 and T1574.010 are removed (total 2 ATT&CK links). This is now the same as the original stix-capec-v3.9.json object
stix2arango handles the logic of updating and ageing out these objects based on the md5 hashes changing for the id. This is the logic ATS uses and it works fine.
Now, in 1.2 arango_cti_processor created 6 SROs to link the capec objects to attack. In 1.2, you will also see arango_cti_processor created doc._is_latest == false for all relationship objects from previous tests
This logic works fine when objects are added.
e.g. in 1.0 4 objects created, in 1.1 5 objects created (4 marked as old from 1.0), in 1.2 6 objects created (5 marked as old from 1.1)
However, when objects are removed from the
To solve this, the behaviour of ACTIP could be to check for changes to an id of object (which would cause a s2a change) and then check if the relevant data for the mode (e.g. capec-attack a change in ATT&CK refs) has change, if change like this detected, mark all links ACTIP created relationships from this source objects (and correct mode) as is_latest=false and then recreate the new relationship objects
See test 1.3.
https://github.com/muchdogesec/arango_cti_processor/tree/adding-tests/tests#test-13-perform-another-update-to-change-capec-attack-pattern---attck-attack-pattern-relationship-capec-attack
In this test we have
In the previous test it was
Thus test 2 Should return 15 results. oldest version (1.0) of CAPEC158 had 4 ATT&CK references, old version 1.1 had 5 ATT&CK references, old version 1.2 had 6 (2 of these still remain, each with 4 lines)
Then the new object should have 4 refs
Test 3 Should return 4 results because the new object has both T1040 (1 coa, 1 attack pattern) and T1111 (1 coa, 1 attack pattern)
Update behaviour for SROs created by this script described here: https://github.com/muchdogesec/arango_cti_processor/blob/adding-tests/docs/README.md#updating-sros-on-subsequent-runs
Can be tested using: https://github.com/muchdogesec/arango_cti_processor/blob/main/tests/tests.md#test-14-perform-another-update-to-change-capec-attack-pattern---attck-attack-pattern-relationship-capec-attack