Cyb3rWard0g / HELK

The Hunting ELK
GNU General Public License v3.0
3.73k stars 675 forks source link

Fresh New Install on blank System, no ElastAlerts #495

Open PaulTheHelk opened 3 years ago

PaulTheHelk commented 3 years ago

Describe the problem

I've installed the HELK with the provided Documentation from here: https://thehelk.com/installation.html using option 2 in the install dialog with no know or seen error during that. I've created a test-machine (Windows box) with sysmon and winlogbeat 7.8.1 which seems to work fine and delivers some data. BUT, there are no elastevents here. Even when i set the filter to elastalert_status.

Provide the output of the following commands

Get operating system and version
cat /etc/os-release 
NAME="Ubuntu"
VERSION="18.04.5 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.5 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

Get disk space, memory, processor cores, and docker storage
Docker Space:
Dateisystem    Größe Benutzt Verf. Verw% Eingehängt auf
/dev/sdb1       147G     14G  127G   10% /

Memory:
              gesamt      belegt       frei     gemeinsam    Zwischen   verfügbar
Speicher:          31          22           1           0           8           8
Auslager:           1           0           1

Cores:
4

Get output of the HELK docker containers:

CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS              PORTS                                                                                                                                                                                                  NAMES
a53d17308475        confluentinc/cp-ksql-cli:5.1.3                        "/bin/sh"                5 hours ago         Up 5 hours                                                                                                                                                                                                                 helk-ksql-cli
aabc70be4c16        confluentinc/cp-ksql-server:5.1.3                     "/etc/confluent/dock…"   5 hours ago         Up 5 hours          0.0.0.0:8088->8088/tcp                                                                                                                                                                                 helk-ksql-server
babc858405fb        otrf/helk-kafka-broker:2.4.0                          "./kafka-entrypoint.…"   5 hours ago         Up 5 hours          0.0.0.0:9092->9092/tcp                                                                                                                                                                                 helk-kafka-broker
1412b916a067        otrf/helk-zookeeper:2.4.0                             "./zookeeper-entrypo…"   5 hours ago         Up 5 hours          2181/tcp, 2888/tcp, 3888/tcp                                                                                                                                                                           helk-zookeeper
ba1ee3f649b6        otrf/helk-logstash:7.6.2.1                            "/usr/share/logstash…"   5 hours ago         Up 5 hours          0.0.0.0:3515->3515/tcp, 0.0.0.0:5044->5044/tcp, 0.0.0.0:5514->5514/tcp, 0.0.0.0:5514->5514/udp, 0.0.0.0:8515-8516->8515-8516/tcp, 0.0.0.0:8531->8531/tcp, 0.0.0.0:8515-8516->8515-8516/udp, 9600/tcp   helk-logstash
6e174c517bee        otrf/helk-nginx:0.3.0                                 "/opt/helk/scripts/n…"   5 hours ago         Up 5 hours          0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp                                                                                                                                                               helk-nginx
3966bb3e6ac0        docker.elastic.co/kibana/kibana:7.6.2                 "/usr/share/kibana/s…"   5 hours ago         Up 5 hours          5601/tcp                                                                                                                                                                                               helk-kibana
5e7cb45ad95c        docker.elastic.co/elasticsearch/elasticsearch:7.6.2   "/usr/share/elastics…"   5 hours ago         Up 5 hours          9200/tcp, 9300/tcp                                                                                                                                                                                     helk-elasticsearch`

Provide the HELK installation logs located at /var/log/helk-install.log if you are having install errors

Dont have any errors. 

What version of HELK are you using

run the command from within the HELK root directory cat .git/refs/heads/master and include what date you cloned the HELK repo

cat .git/refs/heads/master
f70c4b8c3fce9c14fe7457b7350797b69ca4deb0
cloned today, 19.08.2020

What version of Winlogbeat are you using if you are using Windows/WEF logs

winlogbeat 7.8.1 (Installed yesterday)
What steps did you take trying to fix the issue
How could we replicate the issue

Simply create some events with mimikatz

Any additionally code or log context you would like to provide
nope

Any additional context or input you have

All other fields are filled. i can see the mimikatz-events in discovery-window:

{
  "_index": "logs-endpoint-winevent-sysmon-2020.08.19",
  "_type": "_doc",
  "_id": "9fe2e2d4729e9c78c68326205b9c0397cb34cd30",
  "_version": 1,
  "_score": null,
  "_source": {
    "etl_host_agent_ephemeral_uid": "21510937-a747-4ab6-b04f-d012ed69f16b",
    "@version": "1",
    "process_id": "5460",
    "z_elastic_ecs": {
      "user": {
        "type": "User",
        "identifier": "S-1-5-18",
        "domain": "NT-AUTORITÄT",
        "name": "SYSTEM"
      },
      "event": {
        "created": "2020-08-19T13:23:44.264Z",
        "kind": "event",
        "provider": "Microsoft-Windows-Sysmon",
        "action": "Process accessed (rule: ProcessAccess)",
        "code": 10
      },
      "agent": {},
      "log": {},
      "host": {},
      "ecs": {
        "version": "1.5.0"
      },
      "winlog": {
        "process": {
          "thread": {}
        }
      }
    },
    "record_number": 12753,
    "target_process_guid": "1c560278-0e9a-5f3d-0c00-000000000800",
    "source_name": "Microsoft-Windows-Sysmon",
    "process_call_trace": "C:\\WINDOWS\\SYSTEM32\\ntdll.dll+9c534|C:\\WINDOWS\\System32\\KERNELBASE.dll+271ee|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+bac96|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+bb06d|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+babd5|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+84a30|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+84868|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+84633|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+c1a49|C:\\WINDOWS\\System32\\KERNEL32.DLL+17bd4|C:\\WINDOWS\\SYSTEM32\\ntdll.dll+6ce51",
    "event_original_time": "2020-08-19T13:23:42.883Z",
    "level": "informationen",
    "task": "Process accessed (rule: ProcessAccess)",
    "process_granted_access": 4112,
    "etl_host_agent_type": "winlogbeat",
    "host_name": "desktop-7g39ag8",
    "event_id": 10,
    "beat_version": "7.8.1",
    "etl_kafka_topic": "winlogbeat",
    "action": "processaccess",
    "etl_kafka_time": 1597843422884,
    "beat_hostname": "DESKTOP-7G39AG8",
    "process_name": "mimikatz.exe",
    "etl_host_agent_uid": "b563bc04-d11c-4cfd-940f-b6c1e5c1a8cf",
    "process_path": "c:\\users\\testclient\\mimikatz\\releases\\x64\\mimikatz.exe",
    "event_timezone": "UTC",
    "etl_pipeline": [
      "all-filter-0098",
      "all-add_processed_timestamp",
      "fingerprint-winlogbeats7",
      "winlogbeat_7_and_above-field_nest_cleanup",
      "winlogbeat_7_and_above-field_cleanups",
      "1500",
      "1522",
      "winevent-sysmon-all-1531",
      "general_rename-various_global_options",
      "general_rename-SourceProcessGUID",
      "general_rename-TargetProcessGUID",
      "general_rename-SourceProcessId",
      "general_rename-TargetProcessId",
      "process_granted_access-hex2dec",
      "split-process_path-grok-process_name",
      "split-target_process_path-grok-target_process_name",
      "provider_guid-cleanup",
      "process_guid-cleanup",
      "target_process_guid-cleanup",
      "winevent-hostname-cleanup",
      "final-cleanup-message_field"
    ],
    "event_recorded_time": "2020-08-19T13:23:42.884Z",
    "beat_name": "DESKTOP-7G39AG8",
    "@timestamp": "2020-08-19T13:23:42.883Z",
    "type": "wineventlog",
    "version": 3,
    "process_granted_access_orig": "0x1010",
    "target_process_path": "c:\\windows\\system32\\lsass.exe",
    "target_process_id": "784",
    "etl_version": "2020.04.19.01",
    "RuleName": "-",
    "process_guid": "1c560278-26d0-5f3d-7101-000000000800",
    "opcode": "Info",
    "target_process_name": "lsass.exe",
    "provider_guid": "5770385f-c22a-43e0-bf4c-06f5698ffbd9",
    "event_original_message": "Process accessed:\nRuleName: -\nUtcTime: 2020-08-19 13:23:42.883\nSourceProcessGUID: {1c560278-26d0-5f3d-7101-000000000800}\nSourceProcessId: 5460\nSourceThreadId: 5156\nSourceImage: C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe\nTargetProcessGUID: {1c560278-0e9a-5f3d-0c00-000000000800}\nTargetProcessId: 784\nTargetImage: C:\\WINDOWS\\system32\\lsass.exe\nGrantedAccess: 0x1010\nCallTrace: C:\\WINDOWS\\SYSTEM32\\ntdll.dll+9c534|C:\\WINDOWS\\System32\\KERNELBASE.dll+271ee|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+bac96|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+bb06d|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+babd5|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+84a30|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+84868|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+84633|C:\\Users\\testclient\\mimikatz\\Releases\\x64\\mimikatz.exe+c1a49|C:\\WINDOWS\\System32\\KERNEL32.DLL+17bd4|C:\\WINDOWS\\SYSTEM32\\ntdll.dll+6ce51",
    "thread_id": "5156",
    "etl_kafka_partition": 0,
    "etl_kafka_offset": 10301,
    "log_name": "Microsoft-Windows-Sysmon/Operational",
    "etl_processed_time": "2020-08-19T13:23:47.328Z"
  },
  "fields": {
    "@timestamp": [
      "2020-08-19T13:23:42.883Z"
    ],
    "etl_processed_time": [
      "2020-08-19T13:23:47.328Z"
    ]
  },
  "highlight": {
    "target_process_name": [
      "@kibana-highlighted-field@lsass.exe@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1597843422883
  ]
}
neu5ron commented 3 years ago

are you certain events should be occuring? like replying logs with malicious data or things you know should be alerting?

PaulTheHelk commented 3 years ago

yes it should allert for mimikatz usage. i used mimikatz on a test-client, i can see all the events about thatin discovery-mode, but no alerts coming up.

priamai commented 3 years ago

Yes this should work, I will try to replicate your setup since I have just started testing HELK. Now off the bat if you look at the elastalert rules: https://github.com/Cyb3rWard0g/HELK/tree/master/docker/helk-elastalert/rules I don't see one that specifically target Powershell Mimikaz detection. But there is here a script that pulls sigma rules to be converted: https://github.com/Cyb3rWard0g/HELK/blob/master/docker/helk-elastalert/scripts/pull-sigma.sh First check will be to inspect the docker container and check the converted rules to see if they contain the mimikatz detection. I will do that today as an exercise. BTW this is why I was suggesting to integrate Praeco so that we have visibility into what rules are enabled. Cheers!

neu5ron commented 3 years ago

@priamai I am open to praeco at this point, but I see this in the docs: https://github.com/johnsusek/praeco#can-i-import-my-current-elastalert-rules-into-praeco have you used it before with sigma converted elastalert rules?

Either way, the elastalert/sigma management is definitely on the table for a rewrite/re-engineering. So I am open for ideas that work or people have used.

priamai commented 3 years ago

@neu5ron I am going to use your converter script into their docker setup and see what happens. I will let you know here.

priamai commented 3 years ago

@PaulTheHelk looking at the Sigma rules there are two possible candidate matching your execution. Your index is: logs-endpoint-winevent-sysmon-

The first one based on logs-endpoint-winevent-sysmon-* should trigger:

alert:
- debug
buffer_time:
  minutes: 5
description: Detects multiple suspicious process in a limited timeframe
doc_type: doc
filter:
- query:
    query_string:
      query: (event_id:"1" AND process_command_line:("arp.exe" OR "at.exe" OR "attrib.exe" OR "cscript.exe" OR "dsquery.exe" OR "hostname.exe" OR "ipconfig.exe" OR "mimikatz.exe" OR "nbtstat.exe" OR "net.exe" OR "netsh.exe" OR "nslookup.exe" OR "ping.exe" OR "quser.exe" OR "qwinsta.exe" OR "reg.exe" OR "runas.exe" OR "sc.exe" OR "schtasks.exe" OR "ssh.exe" OR "systeminfo.exe" OR "taskkill.exe" OR "telnet.exe" OR "tracert.exe" OR "wscript.exe" OR "xcopy.exe" OR "pscp.exe" OR "copy.exe" OR "robocopy.exe" OR "certutil.exe" OR "vssadmin.exe" OR "powershell.exe" OR "wevtutil.exe" OR "psexec.exe" OR "bcedit.exe" OR "wbadmin.exe" OR "icacls.exe" OR "diskpart.exe"))
index: logs-endpoint-winevent-sysmon-*
max_threshold: 5
metric_agg_key: _id
metric_agg_type: cardinality
name: Sysmon_61ab5496-748e-4818-a92f-de78e20fe7f1_0 Quick Execution of a Series of Suspicious Commands
priority: 4
query_key: host_name.keyword
realert:
  minutes: 0
timestamp_field: etl_processed_time
type: metric_aggregation

And the second one based on winevt logs-endpoint-winevent-* (should not trigger):

alert:
- debug
buffer_time:
  minutes: 5
description: Detects multiple suspicious process in a limited timeframe
doc_type: doc
filter:
- query:
    query_string:
      query: (event_id:"4688" AND process_command_line:("arp.exe" OR "at.exe" OR "attrib.exe" OR "cscript.exe" OR "dsquery.exe" OR "hostname.exe" OR "ipconfig.exe" OR "mimikatz.exe" OR "nbtstat.exe" OR "net.exe" OR "netsh.exe" OR "nslookup.exe" OR "ping.exe" OR "quser.exe" OR "qwinsta.exe" OR "reg.exe" OR "runas.exe" OR "sc.exe" OR "schtasks.exe" OR "ssh.exe" OR "systeminfo.exe" OR "taskkill.exe" OR "telnet.exe" OR "tracert.exe" OR "wscript.exe" OR "xcopy.exe" OR "pscp.exe" OR "copy.exe" OR "robocopy.exe" OR "certutil.exe" OR "vssadmin.exe" OR "powershell.exe" OR "wevtutil.exe" OR "psexec.exe" OR "bcedit.exe" OR "wbadmin.exe" OR "icacls.exe" OR "diskpart.exe"))
index: logs-endpoint-winevent-*
max_threshold: 5
metric_agg_key: _id
metric_agg_type: cardinality
name: 61ab5496-748e-4818-a92f-de78e20fe7f1_0 Quick Execution of a Series of Suspicious Commands
priority: 4
query_key: host_name.keyword
realert:
  minutes: 0
timestamp_field: etl_processed_time
type: metric_aggregation

Now correct if I am wrong but I do not see in your event the process_command_line field indicating that in your windows setup you did enable the command line logging (needs to be done explicitly).

My suggestion is to create another simple rule to at least catch the binary name or file hash which is frankly quite embarrassing to miss....

priamai commented 3 years ago

So just as an example a new rule should be added into the docker config such as this:

alert:
- debug
buffer_time:
  minutes: 5
description: Detects multiple suspicious process in a limited timeframe
doc_type: doc
filter:
- query:
    query_string:
      query: (event_id:"10" AND process_name:("arp.exe" OR "at.exe" OR "attrib.exe" OR "cscript.exe" OR "dsquery.exe" OR "hostname.exe" OR "ipconfig.exe" OR "mimikatz.exe" OR "nbtstat.exe" OR "net.exe" OR "netsh.exe" OR "nslookup.exe" OR "ping.exe" OR "quser.exe" OR "qwinsta.exe" OR "reg.exe" OR "runas.exe" OR "sc.exe" OR "schtasks.exe" OR "ssh.exe" OR "systeminfo.exe" OR "taskkill.exe" OR "telnet.exe" OR "tracert.exe" OR "wscript.exe" OR "xcopy.exe" OR "pscp.exe" OR "copy.exe" OR "robocopy.exe" OR "certutil.exe" OR "vssadmin.exe" OR "powershell.exe" OR "wevtutil.exe" OR "psexec.exe" OR "bcedit.exe" OR "wbadmin.exe" OR "icacls.exe" OR "diskpart.exe"))
index: logs-endpoint-winevent-sysmon-*
max_threshold: 5
metric_agg_key: _id
metric_agg_type: cardinality
name: Sysmon_61ab5496-748e-4818-a92f-de78e20fe7f1_0 Quick Execution of a Series of Suspicious Commands
priority: 4
query_key: host_name.keyword
realert:
  minutes: 0
timestamp_field: etl_processed_time
type: metric_aggregation

Please notice also that event_id should be set to 10 which refers to process access.