[Request] Expand custom STIX support in attackToExcel.py

Is your feature request related to a problem?

No, but an extension to existing functionality.

Happy to make and PR-propose the changes myself, but would like feedback from the team. Thanks for reading!

Background

I'm looking to create Excel versions of my custom ATT&CK-spinoff STIX, which is custom-domained items on top of ATT&CK Enterprise, i.e. https://mitre-atlas.github.io/atlas-navigator/ with STIX at https://github.com/mitre-atlas/atlas-navigator-data/blob/main/dist/stix-atlas.json, but have noticed some missing info and have some questions.

Steps to replicate

Upon running the following with mitreattack-python==2.0.14 with a path to that STIX linked above:

import mitreattack.attackToExcel.attackToExcel as attackToExcel

attackToExcel.export(stix_file='dist/stix-atlas.json')

Results and Qs

The output is a directory named enterprise-attack with Excel files enterprise-attack-*.xslx within. As expected, since the default domain is enterprise-attack, though a custom domain isn't supported because of pre-defined domain lookups.
Questions: Can/should the following be done? What are the considerations?
1. Specify the directory name/filename prepend value as an argument
  - Assuming that these names are directly from the domain
2. Can a custom domain be supported?
  - Currently no, since there are pre-defined lookups like MATRIX_PLATFORMS_LOOKUP[domain] and custom code depending on domains

Looking at enterprise-attack-tactics.xslx, I see my custom tactic (top row, with the Enterprise version below), but it doesn't have an ID nor a URL.

The ID and URL are set in the custom STIX as the first element in external_references.

{
  "type": "x-mitre-tactic",
  "spec_version": "2.1",
  "id": "x-mitre-tactic--0872d6ab-8e2c-4b54-bc0a-2fac5fed9f5c",
  "created": "2023-04-25T19:27:08.171328Z",
  "modified": "2023-04-25T19:27:08.171328Z",
  "name": "Collection",
  "description": "The adversary is trying to gather machine learning artifacts and other related information relevant to their goal.\n\nCollection consists of techniques adversaries may use to gather information and the sources information is collected from that are relevant to following through on the adversary's objectives.\nFrequently, the next goal after collecting data is to steal (exfiltrate) the ML artifacts, or use the collected information to stage future operations.\nCommon target sources include software repositories, container registries, model repositories, and object stores.\n",
  "external_references": [
    {
      "source_name": "mitre-atlas",
      "url": "https://atlas.mitre.org/tactics/AML.TA0009",
      "external_id": "AML.TA0009"
    }
  ],
  "x_mitre_shortname": "collection"
},

Questions:

I haven't dug too deep into why/how, but should the ID and URL be pulled from the custom STIX?

My custom STIX objects don't come out in enterprise-attack.xlsx, but they appear in the individual Excel files. For example - the upper window is enterprise-attack-tactics, showing my custom tactic objects (the ones with blank IDs). The bottom window is the tactics sheet of enterprise-attack.xlsx, which only has ATT&CK objects and not any custom ones.
Questions:
1. Is this expected? From a brief look at the code, both workbooks appear to be written in the same loop. Was expecting to see the custom objects in the combined workbook too.
2. Would also propose adding any new-found matrices as sheets in this combined workbook.

Per discussion with @jondricek:

While the best course of action may be to create Relationships to track the domain-plaform relationship and eventually be able to dynamically generate something like MATRIX_PLATFORMS_LOOKUP, that's a future thing.

For now, we'll modify the attackToExcel scripts to be able to support ATLAS' use case of a custom domain and resolve issues stated above. A few starting points to be done in our forked repo

[ ] @vivjamba To have custom IDs show up in the ID column, and also to have custom tactics show up in the tactics sheet: assuming that all MITRE ATT&CK and ATLAS objects have a reference to themselves as the first external reference, remove this check for MITRE_ATTACK_ID_SOURCE_NAMES and replace with just a check for field existences
[ ] @lilyjw Consider adding any new-found matrices as sheets in the combined workbook

A good test for ATT&CK's working state is to run this and there are no differences between current Excel data. python update-attack.py --no-test-exitstatus --extras resources @vivjamba take a look at how to auto-test this

mitre-attack / mitreattack-python