Can't import results from Splunk

[Gandalf098]

Hi,

First of all thanks a lot for the awesome tool,

I just have a problem with the format of fields sent from splunk to beagle, I tried wineventlog and xmleventlog, both of them are not working

here is the log snippet

2020-10-17T17:14:03 | beagle.web.api.views.new:240 | INFO | Recieved upload request for datasource=, transformer=, backend= 2020-10-17T17:14:03 | beagle.web.api.views.new:243 | INFO | Transforming data to a graph. 2020-10-17T17:14:03 | beagle.web.api.views._setup_params:441 | DEBUG | Setting up parameters 2020-10-17T17:14:03 | beagle.web.api.views._setup_params:452 | INFO | ExternalDataSource params received {'spl': 'index=botsv1 sourcetype="win*" sourcetype=wineventlog src_user="LOCAL SERVICE"', 'earliest': '0'} 2020-10-17T17:14:03 | beagle.web.api.views._setup_params:464 | DEBUG | Set up parameters 2020-10-17T17:14:03 | beagle.datasources.splunk_spl.setup_session:101 | INFO | Creating Splunk client for host=172.17.0.1 2020-10-17T17:14:03 | beagle.transformers.generic_transformer.init:20 | INFO | Created Generic Transformer. 2020-10-17T17:14:03 | beagle.transformers.base_transformer.run:77 | DEBUG | Launching transformer 2020-10-17T17:14:03 | beagle.transformers.base_transformer.run:86 | DEBUG | Started producer thread 2020-10-17T17:14:03 | beagle.transformers.base_transformer.run:98 | DEBUG | Started 3 consumer threads 2020-10-17T17:14:03 | beagle.datasources.splunk_spl.events:119 | INFO | Creating splunk search with sid=1602954843.214, waiting for job=Done 2020-10-17T17:14:03 | beagle.datasources.splunk_spl.events:122 | DEBUG | Job not done, sleeping 2020-10-17T17:14:08 | beagle.datasources.splunk_spl.events:125 | INFO | Job is done, getting results 2020-10-17T17:14:08 | beagle.datasources.splunk_spl.events:130 | INFO | Processed 3 splunk results 2020-10-17T17:14:08 | beagle.transformers.base_transformer._producer_thread:125 | DEBUG | Producer Thread Thread-1 finished after 3 events 2020-10-17T17:14:08 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-2 finished after processing 4 events 2020-10-17T17:14:08 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-4 finished after processing 1 events 2020-10-17T17:14:08 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-3 finished after processing 1 events 2020-10-17T17:14:08 | beagle.transformers.base_transformer.run:111 | INFO | Finished processing of events, created 0 nodes. 2020-10-17T17:14:08 | beagle.backends.networkx.init:56 | INFO | Initialized NetworkX Backend 2020-10-17T17:14:08 | beagle.backends.networkx.graph:73 | INFO | Beginning graph generation. 2020-10-17T17:14:08 | beagle.common._merge_batch:87 | DEBUG | Merging batch of size 0 2020-10-17T17:14:08 | beagle.common._merge_batch:105 | DEBUG | Merged down to size 0 2020-10-17T17:14:08 | beagle.backends.networkx.graph:83 | INFO | Completed graph generation. 2020-10-17T17:14:08 | beagle.backends.networkx.graph:84 | INFO | Graph contains 0 nodes and 0 edges. 2020-10-17T17:14:08 | beagle.web.api.views._create_graph:516 | INFO | Cleaning up tempfiles 2020-10-17T17:14:08 | beagle.web.api.views._create_graph:523 | INFO | Finished generating graph

[Gandalf098]

and this is an example of one of the three events in splunk

08/10/2016 02:57:28 PM LogName=Security SourceName=Microsoft Windows security auditing. EventCode=4688 EventType=0 Type=Information ComputerName=we8105desk.waynecorpinc.local TaskCategory=Process Creation OpCode=Info RecordNumber=493922 Keywords=Audit Success Message=A new process has been created.

Creator Subject: Security ID: NT AUTHORITY\LOCAL SERVICE Account Name: LOCAL SERVICE Account Domain: NT AUTHORITY Logon ID: 0x3E5

Target Subject: Security ID: NULL SID Account Name: - Account Domain: - Logon ID: 0x0

Process Information: New Process ID: 0x1374 New Process Name: C:\Windows\System32\conhost.exe Token Elevation Type: %%1936 Mandatory Label: Mandatory Label\System Mandatory Level Creator Process ID: 0xb68 Creator Process Name: C:\Program Files\Windows Defender\MpCmdRun.exe Process Command Line: \??\C:\Windows\system32\conhost.exe 0xffffffff -ForceV1

Token Elevation Type indicates the type of token that was assigned to the new process in accordance with User Account Control policy.

Type 1 is a full token with no privileges removed or groups disabled. A full token is only used if User Account Control is disabled or if the user is the built-in Administrator account or a service account.

Type 2 is an elevated token with no privileges removed or groups disabled. An elevated token is used when User Account Control is enabled and the user chooses to start the program using Run as administrator. An elevated token is also used when an application is configured to always require administrative privilege or to always require maximum privilege, and the user is a member of the Administrators group.

Type 3 is a limited token with administrative privileges removed and administrative groups disabled. The limited token is used when User Account Control is enabled, the application does not require administrative privilege, and the user does not choose to start the program using Run as administrator.

[Gandalf098]

I also tried to rename the fields to match the generic transformer constants (renamed some of the field including ( ex. eventtype to event_type)

still nothing worked