[Question] How to transform/override the data model name?

[0xFustang]

Hi! I am not sure if this is the right place to ask questions about the pipelines, please apologies if this is not the right place.

I am trying to convert rules related to process_creation into Splunk CIM data model (using Endpoint.Processes). In my context, the default data model Endpoint.Processes was copied into Endpoint.Foo to avoid using the default one when it gets updated.

The question is, how can I change/override the data model name in the conversion? Should I use transformations configuration file?

My test:

Rule: https://raw.githubusercontent.com/SigmaHQ/sigma/master/rules/windows/process_creation/proc_creation_win_bitsadmin_download_susp_extensions.yml

Commands:

$ sigma convert \
    --target splunk \
    --pipeline splunk_cim \
    ./proc_creation_win_bitsadmin_download_susp_extensions.yml
Parsing Sigma rules  [####################################]  100%
Processes.process_path="*\\bitsadmin.exe" OR Processes.original_file_name="bitsadmin.exe" Processes.process IN ("* /transfer *", "* /create *", "* /addfile *") Processes.process IN ("*.7z*", "*.asax*", "*.ashx*", "*.asmx*", "*.asp*", "*.aspx*", "*.bat*", "*.cfm*", "*.cgi*", "*.chm*", "*.cmd*", "*.dll*", "*.gif*", "*.jpeg*", "*.jpg*", "*.jsp*", "*.jspx*", "*.log*", "*.png*", "*.ps1*", "*.psm1*", "*.rar*", "*.scf*", "*.sct*", "*.txt*", "*.vbe*", "*.vbs*", "*.war*", "*.wsf*", "*.wsh*", "*.xll*", "*.zip*")

I would like to have:

Foo.process_path="*\\bitsadmin.exe" OR Foo.original_file_name="bitsadmin.exe" Foo.process IN ("* /transfer *", "* /create *", "* /addfile *") Foo.process ...

[thomaspatzke]

That's the right place for your question 😉

The data model name is defined by the data_model_set processing state variable that can be set with the set_state transformation in a YAML processing pipeline definition that replaces the existing data model processing pipeline used on your command similar to the pipeline built-in in the Splunk backend. The pipeline YAML must be somewhat like:

name: Custom data models
priority: 100
transformations:
- id: custom_process_dm
  type: set_state
  key: data_model_set
  value: Endpoint.Foo
  rule_conditions:
  - type: logsource
    category: process_creation
- id: custom_process_mapping
  type: field_name_mapping
  mapping:
    ImageFileName: Foo.process_path
    ...
  rule_conditions:
  - type: logsource
    category: process_creation

[0xFustang]

Thank you very much @thomaspatzke.

I modified my pipeline YAML into:

name: Custom data models
priority: 100
transformations:
  - id: custom_process_dm
    type: set_state
    key: data_model_set
    val: 'Endpoint.Foo'
    rule_conditions:
      - type: logsource
        category: process_creation
  - id: custom_process_mapping
    type: field_name_mapping
    mapping:
      OriginalFileName: Foo.original_file_name
      Image: Foo.process_path
      CommandLine: Foo.process
    rule_conditions:
      - type: logsource
        category: process_creation

And I get the following result:

$ sigma convert --target splunk -p custom_data_model.yml ./proc_creation_win_bitsadmin_download_susp_extensions.yml
Parsing Sigma rules  [####################################]  100%
Foo.process_path="*\\bitsadmin.exe" OR Foo.original_file_name="bitsadmin.exe" Foo.process IN ("* /transfer *", "* /create *", "* /addfile *") Foo.process IN ("*.7z*", "*.asax*", "*.ashx*", "*.asmx*", "*.asp*", "*.aspx*", "*.bat*", "*.cfm*", "*.cgi*", "*.chm*", "*.cmd*", "*.dll*", "*.gif*", "*.jpeg*", "*.jpg*", "*.jsp*", "*.jspx*", "*.log*", "*.png*", "*.ps1*", "*.psm1*", "*.rar*", "*.scf*", "*.sct*", "*.txt*", "*.vbe*", "*.vbs*", "*.war*", "*.wsf*", "*.wsh*", "*.xll*", "*.zip*") | table Foo.original_file_name,Foo.process_path,Foo.process

However, I am not able to use the data_model splunk format, I get the following output:

sigma convert --target splunk -f data_model -p custom_data_model.yml ./proc_creation_win_bitsadmin_download_susp_extensions.yml
Parsing Sigma rules  [####################################]  100%
Error while conversion: No fields specified by processing pipeline

Should I add another condition in my pipeline YAML?

[0xFustang]

name: Custom data models
priority: 100
transformations:
  - id: custom_process_dm
    type: set_state
    key: data_model_set
    val: 'Endpoint.Foo'
    rule_conditions:
      - type: logsource
        category: process_creation

... as a bit of context I changed value: to val since I had the error:

sigma.exceptions.SigmaConfigurationError: Error in processing rule 1: Error in transformation: SetStateTransformation.__init__() got an unexpected keyword argument 'value'

EDIT:

Actually, I managed to get what I want. But I am not sure if it's how I should do it 😆 :

name: Custom data models
priority: 100
transformations:
  - id: custom_process_dm
    type: set_state
    key: data_model_set
    val: Endpoint.Foo
    rule_conditions:
      - type: logsource
        category: process_creation
  - id: custom_process_dm
    type: set_state
    key: fields
    val:
      CommandLine: Foo.process
      Image: Foo.process_path
      OriginalFileName: Foo.original_file_name
    rule_conditions:
      - type: logsource
        category: process_creation
  - id: custom_process_mapping
    type: field_name_mapping
    mapping:
      OriginalFileName: Foo.original_file_name
      Image: Foo.process_path
      CommandLine: Foo.process
    rule_conditions:
      - type: logsource
        category: process_creation
        product: windows
  - id: custom_process_dm
    type: set_state
    key: fields
    val:
      Foo.process: CommandLine
      Foo.process_path: Image
      Foo.original_file_name: OriginalFileName

Result:

$ sigma convert -t splunk -p custom_data_model.yml ./proc_creation_win_bitsadmin_download_susp_extensions.yml -f data_model
Parsing Sigma rules  [####################################]  100%
| tstats summariesonly=false allow_old_summaries=true fillnull_value="null" count min(_time) as firstTime max(_time) as lastTime from datamodel=Endpoint.Foo where Foo.process_path="*\\bitsadmin.exe" OR Foo.original_file_name="bitsadmin.exe" Foo.process IN ("* /transfer *", "* /create *", "* /addfile *") Foo.process IN ("*.7z*", "*.asax*", "*.ashx*", "*.asmx*", "*.asp*", "*.aspx*", "*.bat*", "*.cfm*", "*.cgi*", "*.chm*", "*.cmd*", "*.dll*", "*.gif*", "*.jpeg*", "*.jpg*", "*.jsp*", "*.jspx*", "*.log*", "*.png*", "*.ps1*", "*.psm1*", "*.rar*", "*.scf*", "*.sct*", "*.txt*", "*.vbe*", "*.vbs*", "*.war*", "*.wsf*", "*.wsh*", "*.xll*", "*.zip*") by Foo.process Foo.process_path Foo.original_file_name | `drop_dm_object_name(Foo)` | convert timeformat="%Y-%m-%dT%H:%M:%S" ctime(firstTime) | convert timeformat="%Y-%m-%dT%H:%M:%S" ctime(lastTime)

[thomaspatzke]

Looks good 👍