sigma list pipelines command fails

[fukusuket]

Hello, thank you for maintaining tools :) I found the following issue, so I would appreciate it if you could check it.

Describe the bug sigma list pipelines command failed with latest simga-cli(0.5.3).

Step to Reproduce

1. python3 -m pip install sigma-cli
2. sigma list pipelines

Expected behavior sigma list pipelines command succeeds.

Actual behavior The command failed with the following stack trace.

...% sigma list pipelines
Traceback (most recent call last):
  File "/Users/fukusuke/.pyenv/versions/3.11.0/bin/sigma", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/fukusuke/.pyenv/versions/3.11.0/lib/python3.11/site-packages/sigma/cli/main.py", line 15, in main
    cli()
  File "/Users/fukusuke/.pyenv/versions/3.11.0/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/fukusuke/.pyenv/versions/3.11.0/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/fukusuke/.pyenv/versions/3.11.0/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/fukusuke/.pyenv/versions/3.11.0/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/fukusuke/.pyenv/versions/3.11.0/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/fukusuke/.pyenv/versions/3.11.0/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/fukusuke/.pyenv/versions/3.11.0/lib/python3.11/site-packages/sigma/cli/list.py", line 49, in list_pipelines
    pipeline = pipeline_resolver.resolve_pipeline(name)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/fukusuke/.pyenv/versions/3.11.0/lib/python3.11/site-packages/sigma/processing/resolver.py", line 32, in resolve_pipeline
    return pipeline()
           ^^^^^^^^^^
  File "/Users/fukusuke/.pyenv/versions/3.11.0/lib/python3.11/site-packages/sigma/pipelines/splunk/splunk.py", line 58, in splunk_windows_pipeline
    items=[
          ^
  File "/Users/fukusuke/.pyenv/versions/3.11.0/lib/python3.11/site-packages/sigma/pipelines/splunk/splunk.py", line 61, in <listcomp>
    transformation=AddConditionTransformation({ "source": "WinEventLog:" + source}),
                                                          ~~~~~~~~~~~~~~~^~~~~~~~
TypeError: can only concatenate str (not "list") to str

Environment I confirmed in the following environments.

• Python 3.11.0
• macOS 13.1
• sigma-cli 0.5.3

Additional context Above error occurs in the following code. It seems that originally source was expected to be of type str(not list)

• https://github.com/SigmaHQ/pySigma-backend-splunk/blob/v0.3.7/sigma/pipelines/splunk/splunk.py#L61

However, the windows_logsource_mapping value now contains the type list from the following commits.(pySigma 0.8.10)

• https://github.com/SigmaHQ/pySigma/pull/67/commits/a4338e01bed78edbf47e4e9450d666b505432c63

Because the pySigma version is 0.8.1 in poetry.lock, this issue did not occur when I did with poetry install.

It seems to be an issue related to pySigma-backend-splunk, so it may be better to create an issue in the pySigma-backend-splunk repository?

[thomaspatzke]

Confirmed!

[thomaspatzke]

Introduced bis this commit which fixed the log sources but broke compatibility by introducing list elements. I believe this bug is also contained in other backends. Will revert the change in the main branch and move it to 0.9.x that contains further breaking changes.

[thomaspatzke]

Fixed in 0.8.11.

[fukusuket]

Thank you for your quick response :)

[fukusuket]

I confirmed that the command succeeds with the steps below 😀

1. pip uninstall pysigma <- to remove pySigma 0.8.10
2. pip install sigma-cli
3. sigma list pipelines

   fukusuke@fukusukenoAir ~ % sigma list pipelines
   +----------------------------+----------+----------------------------------------------------------------------------------+---------------------------+
   | Identifier                 | Priority | Processing Pipeline                                                              | Backends                  |
   +----------------------------+----------+----------------------------------------------------------------------------------+---------------------------+
   | sysmon                     | 10       | Generic Log Sources to Sysmon Transformation                                     | all                       |
   | crowdstrike_fdr            | 10       | Generic Log Sources to CrowdStrike Falcon Data Replicator (FDR) Transformation   | all                       |
   | splunk_windows             | 20       | Splunk Windows log source conditions                                             | splunk                    |
   | splunk_sysmon_acceleration | 25       | Splunk Windows Sysmon search acceleration keywords                               | splunk                    |
   | splunk_cim                 | 20       | Splunk CIM Data Model Mapping                                                    | splunk                    |
   | ecs_windows                | 20       | Elastic Common Schema (ECS) Windows log mappings from Winlogbeat from version 7  | elasticsearch, opensearch |
   | ecs_windows_old            | 20       | Elastic Common Schema (ECS) Windows log mappings from Winlogbeat up to version 6 | elasticsearch, opensearch |
   | ecs_zeek_beats             | 20       | Elastic Common Schema (ECS) for Zeek using filebeat >= 7.6.1                     | elasticsearch, opensearch |
   | ecs_zeek_corelight         | 20       | Elastic Common Schema (ECS) mapping from Corelight                               | elasticsearch, opensearch |
   | zeek                       | 20       | Zeek raw JSON field naming                                                       | all                       |
   | windows                    | 10       | Generic Log Sources to Windows Transformation                                    | all                       |
   +----------------------------+----------+----------------------------------------------------------------------------------+---------------------------+