search

cisagov / CHIRP

A DFIR tool written in Python.
Creative Commons Zero v1.0 Universal
1.04k stars 90 forks source link

Application Hangs after Traceback errors #35

Closed RITOps closed 3 years ago

RITOps commented 3 years ago

🐛 Summary

Traceback error comes up and app seems to freeze while trying to scan files during YARA section.

To reproduce

Log into Win2012R2 server as domain admin, go to chirp directory and kick off app via Powershell (admin mode) /.chirp.exe Left the process running overnight. Following day found app window with errors: Traceback errors (see attached).

CHIRP process still in Task Manager, but stuck at 0% CPU utilization.

This occurs on version 1.03 and 1.04 on Win2012R2

Ran version 1.05 on Win2012R2 and getting Traceback error with Unicode errors as shown below. This is preceded by Traceback lines that are identical with each occurrence. UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 4544: invalid start byte UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 4447: invalid start byte UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 3871: invalid start byte

Expected behavior

Tool is expected to run to completion.

Any helpful log output or screenshots

Win2012R2 CHIRP Error_Hangs

Version 1.05 PS C:\kworking\chirp> cd.. PS C:\kworking> cd chirp1.05 PS C:\kworking\chirp1.05> ./chirp.exe 16:20:23 EVENTS Reading Windows Powershell event logs. scan.py:69 16:20:24 EVENTS Reading KernelMode event logs. scan.py:69 EVENTS Reading Application event logs. scan.py:69 16:20:25 REGISTRY Reading HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot scan.py:65 REGISTRY Key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot does not exist. registry.py:93 REGISTRY Reading HKEY_LOCAL_MACHINE\Software\Microsoft\CTF scan.py:65 REGISTRY Reading HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Image File Execution scan.py:65 Options\ REGISTRY Found 0 hit(s) for Sibot - Registry indicator. scan.py:47 REGISTRY Found 0 hit(s) for Teardrop - Registry Activity indicator. scan.py:47 REGISTRY Found 0 hit(s) for IFEO Persistence indicator. scan.py:47 YARA Enumerating the entire filesystem due to ['simpleseesharp : Webshell Unclassified', run.py:141 'reGeorgTunnel : Webshell Commodity', 'sportsball : Webshell', 'Detection for the use of procdump to dump LSASS process memory.', 'CISA Solar Fire', 'CISA Teardrop', 'CrowdStrike Rempack', 'CrowdStrike Sunspot', 'FireEye Cosmic Gale', 'FireEye Sunburst']... this is going to take a while. 16:20:49 EVENTS Reading Security event logs. scan.py:69 16:29:26 YARA Beginning processing. run.py:100 Traceback (most recent call last): File "C:\Users\\AppData\Local\Temp\ONEFIL~3\chirp.py", line 17, in run.run() File "C:\Users\\AppData\Local\Temp\ONEFIL~3\chirp\run.py", line 20, in run File "C:\Users\\AppData\Local\Temp\ONEFIL~3\chirp\run.py", line 30, in run_plugins File "C:\Users\\AppData\Local\Temp\ONEFIL~3\asyncio\base_events.py", line 616, in run_until_complete File "C:\Users\\AppData\Local\Temp\ONEFIL~3\chirp\run.py", line 44, in _run_coroutines File "C:\Users\\AppData\Local\Temp\ONEFIL~3\chirp\plugins\network\scan.py", line 44, in run File "C:\Users\\AppData\Local\Temp\ONEFIL~3\chirp\plugins\network\network.py", line 37, in parse_dns UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 4544: invalid start byte 16:35:55 YARA We're still working on scanning files. 50000 processed. run.py:96 16:40:34 YARA We're still working on scanning files. 100000 processed. run.py:96 16:43:17 YARA We're still working on scanning files. 150000 processed. run.py:96 16:45:09 YARA We're still working on scanning files. 200000 processed. run.py:96

This is another Win2012R2 server, with CHIRP v1.05 - UnicodeError 0xff in position 4447 error. 11:05:40 EVENTS Reading KernelMode event logs. scan.py:69 EVENTS Reading Application event logs. scan.py:69 11:05:41 REGISTRY Reading HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Wind scan.py:65 ows\CurrentVersion\sibot REGISTRY Key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Wind registry.py:93 ows\CurrentVersion\sibot does not exist. REGISTRY Reading HKEY_LOCAL_MACHINE\Software\Microsoft\CTF scan.py:65 REGISTRY Reading scan.py:65 HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\ REGISTRY Found 0 hit(s) for Sibot - Registry indicator. scan.py:47 REGISTRY Found 0 hit(s) for Teardrop - Registry Activity scan.py:47 indicator. REGISTRY Found 0 hit(s) for IFEO Persistence indicator. scan.py:47 YARA Enumerating the entire filesystem due to run.py:141 ['simpleseesharp : Webshell Unclassified', 'reGeorgTunnel : Webshell Commodity', 'sportsball : Webshell', 'Detection for the use of procdump to dump LSASS process memory.', 'CISA Solar Fire', 'CISA Teardrop', 'CrowdStrike Rempack', 'CrowdStrike Sunspot', 'FireEye Cosmic Gale', 'FireEye Sunburst']... this is going to take a while. 11:09:35 YARA Beginning processing. run.py:100 Traceback (most recent call last): File "C:\Users\\AppData\Local\Temp\ONEFIL~4\chirp.py", line 17, in <modu le> run.run() File "C:\Users\\AppData\Local\Temp\ONEFIL~4\chirp\run.py", line 20, in r un File "C:\Users\\AppData\Local\Temp\ONEFIL~4\chirp\run.py", line 30, in r un_plugins File "C:\Users\\AppData\Local\Temp\ONEFIL~4\asyncio\base_events.py", lin e 616, in run_untilcomplete File "C:\Users\\AppData\Local\Temp\ONEFIL~4\chirp\run.py", line 44, in run_coroutines File "C:\Users\\AppData\Local\Temp\ONEFIL~4\chirp\plugins\network\scan.p y", line 44, in run File "C:\Users\\AppData\Local\Temp\ONEFIL~4\chirp\plugins\network\networ k.py", line 37, in parse_dns UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 4447: inval id start byte 11:11:52 EVENTS Reading Windows Powershell event logs. scan.py:69 11:12:14 EVENTS Reading Security event logs. scan.py:69

Add any screenshots of the problem here.

RITOps commented 3 years ago

One of our Win2016 servers is also hanging on version 1.05. See output below:

17:24:02 NETWORK Read 507 records, found 0 IoC hits. scan.py:56 REGISTRY Reading HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot scan.py:65 REGISTRY Key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot does not exist. registry.py:93 REGISTRY Reading HKEY_LOCAL_MACHINE\Software\Microsoft\CTF scan.py:65 REGISTRY Reading HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Image File Execution scan.py:65 Options\ REGISTRY Found 0 hit(s) for Sibot - Registry indicator. scan.py:47 REGISTRY Found 0 hit(s) for Teardrop - Registry Activity indicator. scan.py:47 REGISTRY Found 0 hit(s) for IFEO Persistence indicator. scan.py:47 YARA Enumerating the entire filesystem due to ['simpleseesharp : Webshell Unclassified', run.py:141 'reGeorgTunnel : Webshell Commodity', 'sportsball : Webshell', 'Detection for the use of procdump to dump LSASS process memory.', 'CISA Solar Fire', 'CISA Teardrop', 'CrowdStrike Rempack', 'CrowdStrike Sunspot', 'FireEye Cosmic Gale', 'FireEye Sunburst']... this is going to take a while. 17:24:03 EVENTS Reading KernelMode event logs. scan.py:69 EVENTS Reading Security event logs. scan.py:69 18:09:53 EVENTS Reading Application event logs. scan.py:69 18:11:59 YARA Beginning processing. run.py:100 18:14:55 EVENTS Reading Windows Powershell event logs. scan.py:69 18:16:20 EVENTS Read 399688 logs, found 0 matches. scan.py:138 18:16:32 YARA We're still working on scanning files. 50000 processed. run.py:96 18:19:16 YARA We're still working on scanning files. 100000 processed. run.py:96 Traceback (most recent call last): File "C:\Users\\AppData\Local\Temp\ONEFIL~1\chirp.py", line 17, in run.run() File "C:\Users\\AppData\Local\Temp\ONEFIL~1\chirp\run.py", line 20, in run File "C:\Users\\AppData\Local\Temp\ONEFIL~1\chirp\run.py", line 30, in run_plugins File "C:\Users\\AppData\Local\Temp\ONEFIL~1\asyncio\base_events.py", line 616, in run_until_complete File "C:\Users\\AppData\Local\Temp\ONEFIL~1\chirp\run.py", line 44, in _run_coroutines File "C:\Users\\AppData\Local\Temp\ONEFIL~1\chirp\plugins\yara\run.py", line 162, in run File "C:\Users\\AppData\Local\Temp\ONEFIL~1\aiomultiprocess\pool.py", line 145, in results_generator File "C:\Users\\AppData\Local\Temp\ONEFIL~1\aiomultiprocess\pool.py", line 308, in results aiomultiprocess.types.ProxyException: Traceback (most recent call last): File "C:\Users\\AppData\Local\Temp\ONEFIL~1\aiomultiprocess\pool.py", line 110, in run File "C:\Users\\AppData\Local\Temp\ONEFIL~1\chirp\plugins\yara\run.py", line 111, in _run UnicodeEncodeError: 'utf-8' codec can't encode character '\ud8d0' in position 33: surrogates not allowed

DeemOnSecurity commented 3 years ago

Interesting, two separate Unicode errors. I will try to push a patch this evening! I will ping when I merge and build.

RITOps commented 3 years ago

We have another Win2016 Server that is having a similar issue, with a different Unicode error. The program, like the others, hangs after a while and never completes. Please see below:

16:42:58 EVENTS Reading Security event logs. scan.py:69 16:42:58 EVENTS Reading KernelMode event logs. scan.py:69 EVENTS Reading Windows Powershell event logs. scan.py:69 16:42:58 EVENTS Reading Application event logs. scan.py:69 16:42:58 REGISTRY Reading HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot scan.py:65 REGISTRY Key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot does not exist. registry.py:93 REGISTRY Reading HKEY_LOCAL_MACHINE\Software\Microsoft\CTF scan.py:65 16:42:59 REGISTRY Reading HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Image File Execution scan.py:65 Options\ REGISTRY Found 0 hit(s) for Sibot - Registry indicator. scan.py:47 REGISTRY Found 0 hit(s) for Teardrop - Registry Activity indicator. scan.py:47 REGISTRY Found 0 hit(s) for IFEO Persistence indicator. scan.py:47 YARA Enumerating the entire filesystem due to ['simpleseesharp : Webshell Unclassified', run.py:141 'reGeorgTunnel : Webshell Commodity', 'sportsball : Webshell', 'Detection for the use of procdump to dump LSASS process memory.', 'CISA Solar Fire', 'CISA Teardrop', 'CrowdStrike Rempack', 'CrowdStrike Sunspot', 'FireEye Cosmic Gale', 'FireEye Sunburst']... this is going to take a while. 16:46:54 YARA Beginning processing. run.py:100 Traceback (most recent call last): File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp.py", line 17, in run.run() File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\run.py", line 20, in run File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\run.py", line 30, in run_plugins File "C:\Users\\AppData\Local\Temp\ONEFIL~2\asyncio\base_events.py", line 616, in run_until_complete File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\run.py", line 44, in _run_coroutines File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\plugins\network\scan.py", line 44, in run File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\plugins\network\network.py", line 37, in parse_dns UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 5617: invalid start byte 16:51:21 YARA We're still working on scanning files. 50000 processed. run.py:96 16:57:46 YARA We're still working on scanning files. 100000 processed. run.py:96 17:01:50 YARA We're still working on scanning files. 150000 processed. run.py:96 17:05:25 YARA We're still working on scanning files. 200000 processed. run.py:96 17:08:16 YARA We're still working on scanning files. 250000 processed. run.py:96 17:11:27 YARA We're still working on scanning files. 300000 processed. run.py:96 17:14:19 YARA We're still working on scanning files. 350000 processed. run.py:96 17:18:14 YARA We're still working on scanning files. 400000 processed. run.py:96

RITOps commented 3 years ago

Interesting, two separate Unicode errors. I will try to push a patch this evening! I will ping when I merge and build.

Thank you for looking into this.

RITOps commented 3 years ago

Another server having similar issues with Unicode errors, but different position. 17:00:04 REGISTRY Reading HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot scan.py:65 REGISTRY Key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot does not exist. registry.py:93 REGISTRY Reading HKEY_LOCAL_MACHINE\Software\Microsoft\CTF scan.py:65 REGISTRY Reading HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Image File Execution scan.py:65 Options\ REGISTRY Found 0 hit(s) for Sibot - Registry indicator. scan.py:47 REGISTRY Found 0 hit(s) for Teardrop - Registry Activity indicator. scan.py:47 REGISTRY Found 0 hit(s) for IFEO Persistence indicator. scan.py:47 YARA Enumerating the entire filesystem due to ['simpleseesharp : Webshell Unclassified', run.py:141 'reGeorgTunnel : Webshell Commodity', 'sportsball : Webshell', 'Detection for the use of procdump to dump LSASS process memory.', 'CISA Solar Fire', 'CISA Teardrop', 'CrowdStrike Rempack', 'CrowdStrike Sunspot', 'FireEye Cosmic Gale', 'FireEye Sunburst']... this is going to take a while. 17:00:05 EVENTS Reading Application event logs. scan.py:69 17:00:08 EVENTS Reading KernelMode event logs. scan.py:69 17:00:10 EVENTS Reading Windows Powershell event logs. scan.py:69 17:00:20 EVENTS Reading Security event logs. scan.py:69 17:02:22 YARA Beginning processing. run.py:100 Traceback (most recent call last): File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp.py", line 17, in run.run() File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\run.py", line 20, in run File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\run.py", line 30, in run_plugins File "C:\Users\\AppData\Local\Temp\ONEFIL~2\asyncio\base_events.py", line 616, in run_until_complete File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\run.py", line 44, in _run_coroutines File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\plugins\network\scan.py", line 44, in run File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\plugins\network\network.py", line 37, in parse_dns UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 5910: invalid start byte 17:07:03 YARA We're still working on scanning files. 50000 processed. run.py:96 17:12:38 YARA We're still working on scanning files. 100000 processed. run.py:96

RITOps commented 3 years ago

Win2016 Server with similar UnicodeError with different location. Adding output below.

16:45:57 EVENTS Reading Security event logs. scan.py:69 16:46:05 REGISTRY Reading HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot scan.py:65 16:46:06 REGISTRY Key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot does not exist. registry.py:93 REGISTRY Reading HKEY_LOCAL_MACHINE\Software\Microsoft\CTF scan.py:65 REGISTRY Reading HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Image File Execution scan.py:65 Options\ REGISTRY Found 0 hit(s) for Sibot - Registry indicator. scan.py:47 REGISTRY Found 0 hit(s) for Teardrop - Registry Activity indicator. scan.py:47 REGISTRY Found 0 hit(s) for IFEO Persistence indicator. scan.py:47 YARA Enumerating the entire filesystem due to ['simpleseesharp : Webshell Unclassified', run.py:141 'reGeorgTunnel : Webshell Commodity', 'sportsball : Webshell', 'Detection for the use of procdump to dump LSASS process memory.', 'CISA Solar Fire', 'CISA Teardrop', 'CrowdStrike Rempack', 'CrowdStrike Sunspot', 'FireEye Cosmic Gale', 'FireEye Sunburst']... this is going to take a while. 16:57:18 EVENTS Reading Windows Powershell event logs. scan.py:69 16:57:26 EVENTS Reading Application event logs. scan.py:69 16:58:10 EVENTS Reading KernelMode event logs. scan.py:69 17:00:57 YARA Beginning processing. run.py:100 Traceback (most recent call last): File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp.py", line 17, in run.run() File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\run.py", line 20, in run File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\run.py", line 30, in run_plugins File "C:\Users\\AppData\Local\Temp\ONEFIL~2\asyncio\base_events.py", line 616, in run_until_complete File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\run.py", line 44, in _run_coroutines File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\plugins\network\scan.py", line 44, in run File "C:\Users\\AppData\Local\Temp\ONEFIL~2\chirp\plugins\network\network.py", line 37, in parse_dns UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 5105: invalid start byte 17:20:09 YARA We're still working on scanning files. 50000 processed. run.py:96 17:36:38 YARA We're still working on scanning files. 100000 processed. run.py:96 17:48:44 YARA We're still working on scanning files. 150000 processed. run.py:96 17:54:40 YARA We're still working on scanning files. 200000 processed. run.py:96 17:56:18 YARA We're still working on scanning files. 250000 processed. run.py:96 18:03:33 YARA We're still working on scanning files. 300000 processed. run.py:96 18:15:59 YARA We're still working on scanning files. 350000 processed. run.py:96 18:25:48 YARA We're still working on scanning files. 400000 processed. run.py:96 18:38:49 YARA We're still working on scanning files. 450000 processed. run.py:96 18:55:09 YARA We're still working on scanning files. 500000 processed. run.py:96 19:20:22 YARA We're still working on scanning files. 550000 processed. run.py:96 19:44:00 YARA We're still working on scanning files. 600000 processed. run.py:96 19:58:25 YARA We're still working on scanning files. 650000 processed. run.py:96 20:21:57 YARA We're still working on scanning files. 700000 processed. run.py:96 20:37:50 YARA We're still working on scanning files. 750000 processed. run.py:96

DeemOnSecurity commented 3 years ago

When #33 is merged, I will build the release for this fix.