Support new Sigma Indicator structure

[himynamesdave]

As identified in #16 and #17 sigma objects no longer use labels for att&ck (sigma-attack) and cve (sigma-cve) (covered in tests 07 and 08)

instead external_references are now used like so

                {
                    "source_name": "mitre-attack",
                    "description": "tactic",
                    "external_id": "initial_access"
                },
                {
                    "source_name": "mitre-attack",
                    "url": "https://attack.mitre.org/techniques/T1190",
                    "external_id": "T1190"
                },
                {
                    "source_name": "mitre-attack",
                    "url": "https://attack.mitre.org/techniques/T1190.1",
                    "external_id": "T1190.1"
                },
                {
                    "source_name": "mitre-attack",
                    "url": "https://attack.mitre.org/groups/G0125",
                    "external_id": "G0125"
                },
                {
                    "source_name": "mitre-attack",
                    "url": "https://attack.mitre.org/groups/S0125",
                    "external_id": "S0125"
                },
                {
                    "source_name": "cve",
                    "url": "https://nvd.nist.gov/vuln/detail/CVE-2020-1472",
                    "external_id": "CVE-2020-1472"
                },

CVEs

                {
                    "source_name": "cve",
                    "url": "https://nvd.nist.gov/vuln/detail/CVE-2020-1472",
                    "external_id": "CVE-2020-1472"
                },

Identified where "source_name": "cve",, the external_id property is what can be used to find the CVE id in the database

ATT&CK

most of the searches are done in the same way

https://github.com/muchdogesec/arango_cti_processor/blob/main/tests/test-data-research.md#5-sigma-rule-indicator---attck-attack-pattern

Tactic

                {
                    "source_name": "mitre-attack",
                    "description": "tactic",
                    "external_id": "initial_access"
                },

Techniques/Subtechniques

                {
                    "source_name": "mitre-attack",
                    "url": "https://attack.mitre.org/techniques/T1190",
                    "external_id": "T1190"
                },
                {
                    "source_name": "mitre-attack",
                    "url": "https://attack.mitre.org/techniques/T1190.1",
                    "external_id": "T1190.1"
                },

external_id's start with T

Groups

                {
                    "source_name": "mitre-attack",
                    "url": "https://attack.mitre.org/groups/G0125",
                    "external_id": "G0125"
                },

external_id's start with G

Software

                {
                    "source_name": "mitre-attack",
                    "url": "https://attack.mitre.org/groups/S0125",
                    "external_id": "S0125"
                },

external_id's start with S

[himynamesdave]

@fqrious test 7 and 8 still failing.

Seems like they're not being created...

python3 -m unittest tests/test_7_0_sigma_to_attack.py
python3 -m unittest tests/test_8_0_sigma_to_cve.py

[fqrious]

for test 8, the file you're uploading is not to our spec (still using labels instead of external_references), please correct it and it should work.

looking into test 7

[fqrious]

so, I found an issue causing all groups to be skipped, which I have fixed but the test is still failing... I don't think the problem is from the cti_pro... here though, maybe the test code is wrong?

RETURN MERGE(
  FOR doc in sigma_rules_edge_collection
  FILTER doc._arango_cti_processor_note == "sigma-attack" AND doc._is_latest
  COLLECT type = SPLIT(doc.target_ref, "--")[0] into docs
  RETURN {[type]: COUNT(docs[*].doc)}
)

gave me this

[
  {
    "intrusion-set": 42,
    "attack-pattern": 3543,
    "tool": 60,
    "x-mitre-tactic": 10476
  }
]