[Security Solution] Add managed index for MITRE ATT&CK data

[dplumlee]

Summary

Our app currently uses a static, script-generated database for all MITRE ATT&CK references we have in the coverage overview page and rule creation. This ties us to one version of MITRE data per kibana release with no way to update the data individually from kibana itself. By adding a managed index where we could load the MITRE version in dynamically, we could allow users to both update kibana and MITRE data autonomously of one another, and load multiple versions of MITRE data if necessary. We could still keep the script-generated per release version of the data too to account for air gapped machines or to use as a fallback if the managed index doesn't have data.

Use cases:

• Could allow users to update their MITRE ATT&CK mappings without having to update kibana as a whole. This would let users keep their rules up to date longer or stay on a specific MITRE version for as long as they require.
• Allow us to utilize multiple MITRE versions in features. This has been brought up in regards to the coverage overview page, where when either MITRE or kibana is updated and current rules potentially no longer map to the display grid as they're either a later or earlier version than the one we reference in the new kibana release. Allowing multiple versions would give the user more context for these mappings and perhaps aid in the updating of custom rules to the most up to date MITRE mappings.

Related customer requests

• Community slack

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[nicpenning]

As a user of the Elastic stack platform, I would like to make an enrich policy based off of the MITRE ATTCK index to enrich my current datasets that lack most of the ATTCK details. For example, some log sources might simply state T1548, but not include the tactic, name, reference information, etc.

So today we enrich these events with the MITRE ATTCK framework as an index using a custom script.

[banderror]

@approksiu:

would be great if the additional update-data can be shipped out of band to that index, so no old kibana we which still support for rule updates is out of sync

@spong:

I'm making some headway on Knowledge Base Integrations. Once in place you should be able to ship the MITRE data as KB content right alongside the detection rules package :grinning: Latest progress update here: https://github.com/elastic/package-spec/issues/693#issuecomment-1944976478