Receiving an error when converting data sources to json

[RedneckHutch]

I am able to convert the ICS and Mobile data source to a json but am unable to convert the sample data for endpoints. Any recommendations?

root@1100301d5:/opt/DeTTECT# python3 ./dettect.py ds -fd sample-data/data-sources-endpoints.yaml -l --health
Traceback (most recent call last):
  File "/opt/DeTTECT/./dettect.py", line 424, in <module>
    _menu(_init_menu())
  File "/opt/DeTTECT/./dettect.py", line 304, in _menu
    generate_data_sources_layer(file_ds, args.output_filename, args.layer_name, layer_settings)
  File "/opt/DeTTECT/data_source_mapping.py", line 199, in generate_data_sources_layer
    my_techniques = _map_and_colorize_techniques(my_data_sources, systems, exceptions, domain, layer_settings)
  File "/opt/DeTTECT/data_source_mapping.py", line 73, in _map_and_colorize_techniques
    applicable_data_sources = get_applicable_data_sources_platform(system['platform'], domain)
  File "/opt/DeTTECT/generic.py", line 642, in get_applicable_data_sources_platform
    applicable_data_sources.update(data_sources[p])
KeyError: 'Windows'

[rubinatorz]

hi @RedneckHutch,

Can you try to clear the cache directory and try it again?

And if it still fails, can you please check the content of data/data_source_platforms.json and check if you see the "ATT&CK Enterprise" entry and within that the "Windows" entry?

[RedneckHutch]

root@1100301d1075:/opt/DeTTECT# rm -r ~/.cache/*
root@1100301d1075:/opt/DeTTECT# python3 ./dettect.py ds -fd sample-data/data-sources-endpoints.yaml -l --health
Traceback (most recent call last):
  File "/opt/DeTTECT/./dettect.py", line 424, in <module>
    _menu(_init_menu())
  File "/opt/DeTTECT/./dettect.py", line 304, in _menu
    generate_data_sources_layer(file_ds, args.output_filename, args.layer_name, layer_settings)
  File "/opt/DeTTECT/data_source_mapping.py", line 199, in generate_data_sources_layer
    my_techniques = _map_and_colorize_techniques(my_data_sources, systems, exceptions, domain, layer_settings)
  File "/opt/DeTTECT/data_source_mapping.py", line 73, in _map_and_colorize_techniques
    applicable_data_sources = get_applicable_data_sources_platform(system['platform'], domain)
  File "/opt/DeTTECT/generic.py", line 642, in get_applicable_data_sources_platform
    applicable_data_sources.update(data_sources[p])
KeyError: 'Windows'

It doesn't look like windows in included.

{
  "ATT&CK-Enterprise": {},
  "ATT&CK-ICS": {
    "Field Controller/RTU/PLC/IED": [
      "Application Log Content",
      "Asset Inventory",
      "Command Execution",
      "Device Alarm",
      "File Access",
      "File Deletion",
      "File Metadata",
      "File Modification",
      "Firmware Modification",
      "Logon Session Creation",
      "Logon Session Metadata",
      "Network Traffic Content",
      "Network Traffic Flow",
      "OS API Execution",
      "Process Creation",
      "Process History/Live Data",
      "Process Termination",
      "Process/Event Alarm",
      "Script Execution",
      "Service Creation",
      "Software",
      "User Account Authentication",
      "Windows Registry Key Modification"
    ],
    "Safety Instrumented System/Protection Relay": [
      "Application Log Content",
      "Asset Inventory",
      "Command Execution",
      "Device Alarm",
      "File Access",
      "File Deletion",
      "File Metadata",

[RedneckHutch]

Added Windows.

{
  "ATT&CK-Enterprise": {
    "Windows": []
},
  "ATT&CK-ICS": {
    "Field Controller/RTU/PLC/IED": [
      "Application Log Content",
      "Asset Inventory",

Now get the following

root@1100301d1075:/opt/DeTTECT# python3 ./dettect.py ds -fd sample-data/data-sources-endpoints.yaml -l --health
Traceback (most recent call last):
  File "/opt/DeTTECT/./dettect.py", line 424, in <module>
    _menu(_init_menu())
  File "/opt/DeTTECT/./dettect.py", line 304, in _menu
    generate_data_sources_layer(file_ds, args.output_filename, args.layer_name, layer_settings)
  File "/opt/DeTTECT/data_source_mapping.py", line 199, in generate_data_sources_layer
    my_techniques = _map_and_colorize_techniques(my_data_sources, systems, exceptions, domain, layer_settings)
  File "/opt/DeTTECT/data_source_mapping.py", line 73, in _map_and_colorize_techniques
    applicable_data_sources = get_applicable_data_sources_platform(system['platform'], domain)
  File "/opt/DeTTECT/generic.py", line 642, in get_applicable_data_sources_platform
    applicable_data_sources.update(data_sources[p])
KeyError: 'Linux'

I am going to pull down a new data/data_source_platforms.json file.

[RedneckHutch]

Update:

Replacing data/data_source_platforms.json with an updated copy appears to have resolved the issue.

root@1100301d1075:/opt/DeTTECT# python3 ./dettect.py ds -fd sample-data/data-sources-endpoints.yaml -l --health
File written:   output/data_sources_data-sources-sample.json

[rubinatorz]

Great! The data_source_platforms.json is auto generated by pipelines and I see that for a short time this file was not filled for ATT&CK-Enterprise, maybe beause it couldn't fetch the data from MITRE at that moment of time. Good action to resolve the action by getting the latest version from the repo.