Add a `-V, --scan-vss-backups` option to `General Options`

YamatoSecurity commented 4 months ago

Old .evtx logs may be found in the Volume Shadow Copy Service backups so it would be nice to have a --scan-vss-backups option that is used when Hayabusa is doing a live analysis with -l in order to obtain older logs. Since there will usually be duplicate logs it would usually be used with the -X, --remove-duplicate-detections option or after creating results with --low-memory-mode, have the duplicate events deleted with sort-csv.

Ideally it would be best to list out the shadow copy backups with native Windows API calls but I don't think there is any easy way to do that so it will probably be easier at first to use built in Windows commands and parse the results.

To list out VSS backups we can use the command vssadmin list shadows /for=c: which will give a list of Shadow Copy Volume which will look like \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1

It is then possible to mount by creating a link to it with mklink /d D:\shadowtest \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1\

However, when I run Hayabusa against D:\shadowtest\Windows\System32\winevt\Logs\ it gives me the following error:

An error occurred while trying to read input.

Trying to run hayabusa.exe csv-timeline -d \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1\Windows\System32\winevt\Logs\ also gives the same error.

However, vssadmin list shadows also gives Volume information in the Original Volume field. It looks like (C:)\\?\Volume{2223423-aae8-823d-8dafdadf9j3e3}\

When I run hayabusa.exe csv-timeline -d \\?\Volume{2223423-aae8-823d-8dafdadf9j3e3}\Windows\System32\winevt\Logs\ using the Volume ID instead, Hayabusa succeeds in scanning the VSS backups.

So we just need a way to find the original volume names with vssadmin list shadows, parse the output and then have hayabusa add those directories when scanning.

@fukusuket Are you interested in this issue? I think this month you will be busy preparing with BSides, etc.. so no need to rush. It can be added for v2.15.0, etc...

hitenkoku commented 4 months ago

@YamatoSecurity

Is it correct to say that this function is available only on Windows because it uses Windows API commands?

It is unclear from where to what point hayabusa should respond.

I think it is too much to ask Hayabusa to do everything from vssadmin list shadow, which requires administrator privileges, but how about supporting it with another script?

YamatoSecurity commented 4 months ago

Yes, this functionality would only be possible if the user specifies -l, --live-analysis on a Windows machine with local Administrator privileges. Here is a link to some C++ source code that might help us out: https://github.com/albertony/vss?tab=readme-ov-file#vshadow-source-code https://github.com/microsoft/Windows-classic-samples/blob/main/Samples/Win7Samples/winbase/vss/vshadow/query.cpp

It does require some COM programming and may be different for Win 7 versus more recent versions of Windows. So would probably be easier just to call vssadmin and parse the output..

fukusuket commented 4 months ago

Thank you so much for mention :) Sounds interesting! As mentioned in the comments above, there seem to be several options, so I'd like to start by researching each one💪

fukusuket commented 3 months ago

20240411 MTG memo:

COMで行くか直接コマンド実行させるか……
別スクリプトでやるか？
- 速度が遅くなる可能性が高い
参考実装
vssadminやCOMをHayabusaから直接呼ぶ場合
- Client/Server/Locale/Powershell/Terminal/Cmdあたりで差異がないか確認する点が多くなりそう

YamatoSecurity commented 2 months ago

@fukusuket I think I found a better way to do this than COM. We can query the information through WMI! All we need to do is get the VolumeName information. In my test Win VM, I first created a snapshot with wmic shadowcopy call create Volume=C:\ then you can query the volume shadow information with the following PowerShell:

$shadowCopies = Get-WmiObject -Namespace "Root\cimv2" -Class "Win32_ShadowCopy"

foreach ($shadow in $shadowCopies) {
    Write-Output "Volume Name: $($shadow.VolumeName)"
}

This crate can query WMI and thankfully seems to be maintained: https://github.com/ohadravid/wmi-rs Since it is WMI, I do not think the query will change depending on the Windows version.

fukusuket commented 2 months ago

Sounds good! I'll try it💪

YamatoSecurity commented 2 months ago

@fukusuket Great! Thanks! Here is a reference that may help you: https://github.com/trickster0/OffensiveRust/blob/master/wmi_execute/src/main.rs

fukusuket commented 2 months ago

@YamatoSecurity

I tried following code(with wmi = "0.13.3")!

use std::collections::HashMap;
use wmi::{COMLibrary, Variant, WMIConnection};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let com_con = COMLibrary::new()?;
    let wmi_con = WMIConnection::new(com_con.into())?;
    let query = "SELECT * FROM Win32_ShadowCopy";
    let results: Vec<HashMap<String, Variant>> = wmi_con.raw_query(query)?;
    let volumes: Vec<_> = results
        .iter()
        .filter_map(|map| map.get("VolumeName"))
        .collect();
    println!("{:?}", volumes);
    Ok(())
}

then got following expected result👍

C:\tmp\vss\target\release>sample.exe
[String("\\\\?\\Volume{1355c4a7-6b31-43a1-80f4-e35b98d9695f}\\"), String("\\\\?\\Volume{1355c4a7-6b31-43a1-80f4-e35b98d9695f}\\")]

The above results match the vssadmin results.

C:\tmp\vss\target\release>vssadmin list shadows
vssadmin 1.1 - ボリューム シャドウ コピー サービス管理コマンド ライン ツール
(C) Copyright 2001-2013 Microsoft Corp.

シャドウ コピー セット ID: {7f812a7d-4ad0-4a24-8c7b-c5b20649e1cd} の内容
   1 個のシャドウ コピー、作成時刻: 2024/05/04 17:33:36
      シャドウ コピー ID: {e3ff3feb-3ccc-4118-8f1a-ca09b55dc686}
         元のボリューム: (C:)\\?\Volume{1355c4a7-6b31-43a1-80f4-e35b98d9695f}\
         シャドウ コピー ボリューム: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy8
         元のコンピューター: mouse
         サービス コンピューター: mouse
         プロバイダー: 'Microsoft Software Shadow Copy provider 1.0'
         種類: ClientAccessibleWriters
         属性: 恒久, クライアント アクセス可能, 自動リリースなし, 差分, 自動回復

シャドウ コピー セット ID: {bc538b53-b2bf-4dd0-8182-dcb42f1fed6a} の内容
   1 個のシャドウ コピー、作成時刻: 2024/05/10 15:52:58
      シャドウ コピー ID: {8cfe7010-b6da-4c51-ac35-6f7b209806b7}
         元のボリューム: (C:)\\?\Volume{1355c4a7-6b31-43a1-80f4-e35b98d9695f}\
         シャドウ コピー ボリューム: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy9
         元のコンピューター: mouse
         サービス コンピューター: mouse
         プロバイダー: 'Microsoft Software Shadow Copy provider 1.0'
         種類: ClientAccessibleWriters
         属性: 恒久, クライアント アクセス可能, 自動リリースなし, 差分, 自動回復

fukusuket commented 2 months ago

The expected detection results were obtained as shown below.

-l option

hayabusa.exe csv-timeline -l -o main.csv -w -C -q
Start time: 2024/05/10 16:32

Total event log files: 356
Total file size: 209.3 MB

Loading detection rules. Please wait.
...
Deprecated rules: 208 (5.06%) (Disabled)
Experimental rules: 854 (20.79%)
Stable rules: 240 (5.84%)
Test rules: 3,014 (73.37%)
Unsupported rules: 45 (1.10%) (Disabled)

Hayabusa rules: 162
Sigma rules: 3,946
Total detection rules: 4,108

Creating the channel filter. Please wait.

Evtx files loaded after channel filter: 26
Detection rules enabled after channel filter: 2,054

Results Summary:

Events with hits / Total events: 17,941 / 106,505 (Data reduction: 88,564 events (83.15%))

Total | Unique detections: 18,071 | 58
Total | Unique critical detections: 186 (1.03%) | 3 (0.00%)
Total | Unique high detections: 144 (0.80%) | 8 (43.10%)
Total | Unique medium detections: 319 (1.77%) | 11 (18.97%)
Total | Unique low detections: 15,802 (87.44%) | 11 (18.97%)
Total | Unique informational detections: 1,620 (8.96%) | 25 (13.79%)

volume shadow

hayabusa.exe csv-timeline -d \\?\Volume{1355c4a7-6b31-43a1-80f4-e35b98d9695f}\Windows\System32\winevt\Logs\ -q -o timeline-1.csv -C -w
Start time: 2024/05/10 16:34

Total event log files: 356
Total file size: 209.3 MB

Loading detection rules. Please wait.
...
Evtx files loaded after channel filter: 26
Detection rules enabled after channel filter: 2,054

Results Summary:

Events with hits / Total events: 17,941 / 106,507 (Data reduction: 88,566 events (83.16%))

Total | Unique detections: 18,071 | 58
Total | Unique critical detections: 186 (1.03%) | 3 (0.00%)
Total | Unique high detections: 144 (0.80%) | 8 (43.10%)
Total | Unique medium detections: 319 (1.77%) | 11 (18.97%)
Total | Unique low detections: 15,802 (87.44%) | 11 (18.97%)
Total | Unique informational detections: 1,620 (8.96%) | 25 (13.79%)

fukusuket commented 2 months ago

I think the following specifications. What do you think?

-l, --live-analysis option is required
If --scan-vss-backups specified, then scan volume shadow in addition to C:\Windows\System32\winevt\Logs\
- If there are multiple volume shadows（元のボリューム）, scan all of them（or Is it a specification that only one exists?)
Add --scan-vss-backups in General Options (Therefore, it can be used with the following command)
- computer-metrics/eid-metrics/logon-summary/pivot-keywords-list/search/csv-timeline/json-timeline

If I have misunderstood the specifications, please let me know!🙏

fukusuket commented 2 months ago

I have one more question! If original volume specified, it means scanning from backup file?(not from the current evtx)

YamatoSecurity commented 2 months ago

-l, --live-analysis option is required

Yes, this is only possible with live analysis so we should require this.

If --scan-vss-backups specified, then scan volume shadow in addition to C:\Windows\System32\winevt\Logs\

Yes!

If there are multiple volume shadows（元のボリューム）, scan all of them（or Is it a specification that only one exists?)

There can be multiple volume shadows so I would like to scan them all. Note: There will probably be many duplicate events so users should probably also specify -X, --remove-duplicate-detections. The problem will be if users want to use low memory mode because it is live analysis, then they won't be able to specify -X. We can handle this after the scan with the sort command that also deletes duplicate events.

Add --scan-vss-backups in General Options (Therefore, it can be used with the following command)

General Options is a good place for it.

computer-metrics/eid-metrics/logon-summary/pivot-keywords-list/search/csv-timeline/json-timeline

I was only thinking about csv-timeline and json-timeline but it might be useful for these other commands as well. The problem is there will be many duplicate events so I think it will mess up the results. Until we can support ignoring duplicate entries in the other commands then we should probably ignore them for now and just implement it in csv-timeline and json-timeline.

It would also be nice to see what volumes were found and their creation date.

Before:

Hayabusa rules: 162
Sigma rules: 3,948
Total detection rules: 4,110

Creating the channel filter. Please wait.

After

Hayabusa rules: 162
Sigma rules: 3,948
Total detection rules: 4,110

No shadow copy volumes found.

Creating the channel filter. Please wait.

or

Hayabusa rules: 162
Sigma rules: 3,948
Total detection rules: 4,110

Shadow copy volumes found:
2024/05/09  13:15:39 : (C:)\\?\Volume{a005d58f-0000-0000-0000-100000000000}\ (HarddiskVolumeShadowCopy1)
2024/05/10  18:12:10 : (C:)\\?\Volume{3ea7e1a6-0976-4717-a681-53838c5bf39c}\ (HarddiskVolumeShadowCopy2)

Creating the channel filter. Please wait.

I am guessing that extracting this information should be done before creating the channel filter but if there is a better place/time to do it then please change it to where ever you think is good. I believe Original Volume is the same as the C: drive. I think we should use シャドウコピー ID: {8cfe7010-b6da-4c51-ac35-6f7b209806b7} to scan the backup files. シャドウコピーセット ID: {bc538b53-b2bf-4dd0-8182-dcb42f1fed6a} is used as an ID for a set of copies if multiple snapshots were taken at the same time.

fukusuket commented 2 months ago

@YamatoSecurity Thank you for comment! I see!

I think we should use シャドウコピー ID: {8cfe7010-b6da-4c51-ac35-6f7b209806b7} to scan the backup files.

I tried シャドウコピー ID , but I couldn't scan it with hayabusa ... :( ([ERROR] No .evtx files were found.) Is there a way to scan by シャドウコピー ID?

YamatoSecurity commented 2 months ago

@fukusuket Ah, that is my mistake. I was using the original volume ID of the C: thinking it was the snapshot. It doesn't work for me directly on the command line. Only if the C: volume ID is used. I think we can do this by mounting the snapshot to a folder. (I don't want to add another volume to mount if possible).

mklink /d c:\Tools\hayabusa-2.15.0-win-x64\volumeshadowcopytest \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1\
.\hayabusa-2.15.0-win-x64.exe csv-timeline -w -d c:\Tools\hayabusa-2.15.0-win-x64\volumeshadowcopytest

seems to work... however, unfortunately it gives these errors:

[00:00:00] 285 / 363 ⠙ [===============================>        ] 79%

"volumeshadowcopytest/Windows/System32/winevt/Logs/Microsoft-Windows-TCPIP%4Operational.evtx"                                                                                                                      An error occurred while trying to read input.
An error occurred while trying to read input.
An error occurred while trying to read input.
An error occurred while trying to read input.
An error occurred while trying to read input.
An error occurred while trying to read input.

When I tried to open the backed up Security.evtx from Event Viewer, it gives the error: This media is write protected

If i copy out the files to a different directory then it works but trying to make copies of all the evtx files for live analysis is not ideal as evidence may be overwritten..

When i just type a text file inside the mounted folder, it outputs without any error so maybe Hayabusa is not specifying that it is trying to open the file in read-only mode? By default, it should open the file in read-only mode so I don't know why this would be the case though.. https://doc.rust-lang.org/rust-by-example/std_misc/file/open.html Do you know if it is possible to add the reason for why the error happened like in the link above? Right now we just get An error occurred while trying to read input. which is too vague.

Could you check the code to see if Hayabusa is explicitly trying to open the .evtx files in read-only mode?

fukusuket commented 2 months ago

Could you check the code to see if Hayabusa is explicitly trying to open the .evtx files in read-only mode?

The file is opened at the following code, but it is Read Only. https://github.com/Yamato-Security/hayabusa-evtx/blob/main/src/evtx_parser.rs#L273

Also, in my environment, when I opened mounted folder's evtx in Event Viewer after mklink, I got an error saying that the file was corrupted... :( Therefore, even after running mklink, there may still be missing information to restore the file...?🤔

fukusuket commented 2 months ago

I tried wevtutil command then following result😢

C:\tmp\hayabusa-2.15.0-win-x64\volumeshadowcopytest>wevtutil qe "C:\tmp\hayabusa-2.15.0-win-x64\volumeshadowcopytest\Windows\System32\winevt\Logs\Application.evtx" /lf:true /f:text
イベント ログ ファイルが壊れています。

イベント クエリを開けませんでした。
イベント ログ ファイルが壊れています。

YamatoSecurity commented 2 months ago

@fukusuket Humm.. if you copy the file to a different directory and then scan it, does it work?

fukusuket commented 2 months ago

@YamatoSecurity
I tried copying it and got the same error :( I also checked another file (normal text, python script, exe... etc) then I found sometimes there are files that are partially filled with NULL characters, and this seems to be the cause of the corruption ...🤔

YamatoSecurity commented 2 months ago

I got to work with copying with robocopy:

robocopy c:\test\Windows\System32\winevt\Logs c:\testlogs
.\hayabusa-2.15.0-win-x64.exe csv-timeline -w -d c:\testlogs

Can you test if it works with copying with robocopy?

YamatoSecurity commented 2 months ago

Memo: I can run Hayabusa against the volume shadow backup with .\hayabusa-2.15.0-win-x64.exe csv-timeline -w -d \\.\HarddiskVolumeShadowCopy2\Windows\System32\winevt\Logs without having to make a link first. Unfortunately, I still get the file read errors. I am guessing that it is because the files only contain the block-level differential data which corrupts them unless they are copied out..

YamatoSecurity commented 2 months ago

So i copied out the Security.evtx file to the temp directory and the hashes don't match up so I am guessing that directly accessing the files is only good for metadata analysis and that it is necessary to copy out the files to check the content. Although I haven't read any article that talks about this and it is weird that you still can't read the file when you copy it out (but it works for me..)

While not ideal, one possibility would be to copy files out one at a time to a temp directory, scan the file, delete it then copy another file, etc... which at least would better than copying the entire directory (although not as easy to implement...)

fukusuket commented 2 months ago

@YamatoSecurity In my environment, most files could not be read by hayabusa, regardless of robocopy or not. Therefore, copying does not seem to solve this problem😥 I think it will be difficult to implement unless we find a procedure that allows the files to be completely restored.

-l option

Results Summary:

Events with hits / Total events: 26,026 / 236,081 (Data reduction: 210,055 events (88.98%))

Total | Unique detections: 26,156 | 59
Total | Unique critical detections: 190 (0.73%) | 3 (0.00%)
Total | Unique high detections: 144 (0.55%) | 8 (42.37%)
Total | Unique medium detections: 328 (1.25%) | 11 (20.34%)
Total | Unique low detections: 23,684 (90.55%) | 12 (18.64%)
Total | Unique informational detections: 1,810 (6.92%) | 25 (13.56%)

Dates with most total detections:
critical: 2024-04-27 (38), high: 2024-01-26 (22), medium: 2023-10-12 (48), low: 2024-05-03 (4,279), informational: 2024-05-11 (158)

Top 5 computers with most unique detections:
critical: mouse (3)
high: mouse (7), DESKTOP-CNG7416 (1)
medium: mouse (10), MyComputer (2)
low: mouse (12), DESKTOP-CNG7416 (1)
informational: mouse (25), DESKTOP-CNG7416 (7), MyComputer (1), DESKTOP-9HFNL0J (1)

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Top critical alerts:                                            Top high alerts:                                                                │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Defender Alert (Severe) (186)                                   Antivirus Relevant File Paths Alerts (106)                                      │
│ Antivirus Password Dumper Detection (3)                         Microsoft Defender Blocked from Loading Unsigned DLL (24)                       │
│ Antivirus Exploitation Framework Detection (1)                  Antivirus Hacktool Detection (4)                                                │
│ n/a                                                             Microsoft Defender Tamper Protection Trigger (4)                                │
│ n/a                                                             Defender Alert (High) (2)                                                       │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Top medium alerts:                                              Top low alerts:                                                                 │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Potentially Malicious PwSh (229)                                Credential Manager Enumerated (22,908)                                          │
│ Uncommon PowerShell Hosts (28)                                  Credential Manager Accessed (367)                                               │
│ Suspicious Non PowerShell WSMAN COM Provider (24)               CodeIntegrity - Unmet Signing Level Requirements By File Under Validation (322) │
│ Uncommon AppX Package Locations (15)                            Volume Shadow Copy Mount (49)                                                   │
│ BITS Transfer Job With Uncommon Or Suspicious Remote TLD (14)   Application Uninstalled (20)                                                    │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Top informational alerts:                                                                                                                       │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Svc Installed (248)                                             RDS Sess Logoff (138)                                                           │
│ Bits Job Created (218)                                          RDS Sess Disconnect (86)                                                        │
│ PwSh Engine Started (185)                                       Logoff (66)                                                                     │
│ WMI Provider Started (183)                                      Device Conn (64)                                                                │
│ RDS Sess Logon (141)                                            Event Log Svc Started (62)                                                      │
╰───────────────────────────────────────────────────────────────╌─────────────────────────────────────────────────────────────────────────────────╯

copy byexplorer

Results Summary:

Events with hits / Total events: 117 / 1,390 (Data reduction: 1,273 events (91.58%))

Total | Unique detections: 118 | 4
Total | Unique critical detections: 0 (0.00%) | 0 (0.00%)
Total | Unique high detections: 0 (0.00%) | 0 (50.00%)
Total | Unique medium detections: 0 (0.00%) | 0 (50.00%)
Total | Unique low detections: 116 (98.31%) | 2 (0.00%)
Total | Unique informational detections: 2 (1.69%) | 2 (0.00%)

Dates with most total detections:
critical: n/a, high: n/a, medium: n/a, low: 2024-04-27 (116), informational: 2024-03-30 (2)

Top 5 computers with most unique detections:
critical: n/a
high: n/a
medium: n/a
low: mouse (2)
informational: mouse (2)

╭─────────────────────────────────────────────────────────────────╮
│ Top critical alerts:        Top high alerts:                    │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Top medium alerts:          Top low alerts:                     │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ n/a                         Credential Manager Enumerated (115) │
│ n/a                         Credential Manager Accessed (1)     │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Top informational alerts:                                       │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ RDP Conn Attempt (1)        n/a                                 │
│ RDP Attempt (1)             n/a                                 │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
╰───────────────────────────╌─────────────────────────────────────╯

copy by robocopy

Results Summary:

Events with hits / Total events: 117 / 1,390 (Data reduction: 1,273 events (91.58%))

Total | Unique detections: 118 | 4
Total | Unique critical detections: 0 (0.00%) | 0 (0.00%)
Total | Unique high detections: 0 (0.00%) | 0 (50.00%)
Total | Unique medium detections: 0 (0.00%) | 0 (50.00%)
Total | Unique low detections: 116 (98.31%) | 2 (0.00%)
Total | Unique informational detections: 2 (1.69%) | 2 (0.00%)

Dates with most total detections:
critical: n/a, high: n/a, medium: n/a, low: 2024-04-27 (116), informational: 2024-03-30 (2)

Top 5 computers with most unique detections:
critical: n/a
high: n/a
medium: n/a
low: mouse (2)
informational: mouse (2)

╭─────────────────────────────────────────────────────────────────╮
│ Top critical alerts:        Top high alerts:                    │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Top medium alerts:          Top low alerts:                     │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ n/a                         Credential Manager Enumerated (115) │
│ n/a                         Credential Manager Accessed (1)     │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Top informational alerts:                                       │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ RDP Conn Attempt (1)        n/a                                 │
│ RDP Attempt (1)             n/a                                 │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
│ n/a                         n/a                                 │
╰───────────────────────────╌─────────────────────────────────────╯

fukusuket commented 2 months ago

According to the following article, the ScopeSnapshots feature introduced in Win8 seems to be the cause of the file corruption (the NULL bytes situation is the same with my Win11 data) ... :(

https://www.iij.ad.jp/dev/report/iir/pdf/iir_vol37_focus1.pdf

However, from the verification results in the above article, it seems that there is a possibility that vss can be used on Server OS without corruption🤔

fukusuket commented 2 months ago

Although verification is required, if vss can be read without corruption on Server OS, one option is to implement it as a feature supported only on Server OS🤔

YamatoSecurity commented 2 months ago

@fukusuket That's a good article you found! Even if we can only use it on server OSes, that still would be a good feature..

fukusuket commented 2 months ago

I also think it would be great if support could be provided even if it is only for Server OS :) If we were to implement it, I think the flow would be as follows.

Check if it is Server OS or not(also check -l, --live-analysis); if not Server OS, exit with a message
Execute vssadmin to obtain the volume shadow ID.
Execute mklink to create a temporary symbolic link.(to the dir obtained from the ent variable TMP)
Perform a Sigma scan
Delete the symbolic link with mklink

Concern is, 🤔

executing mklink (processing equivalent to WRITE without asking the user) is a bit suprising behavior.
mklink cannot be run by wmi, so simply run it as an external process
- However, since mklink's stdout is not used, it probably does not cause OS language-dependent problems

fukusuket commented 2 months ago

There are several options(Each has its pros and cons), which one do you think is better?

Implemented as an option for Hayabusa
- pros: Users can easily scan vss
- cons: OS-dependent problems are likely to occur(e.g. encoding)
Implemented as separate PowerShell or BAT
- pros: OS-dependent problems are less likely to occur
- cons: Users must operate script other than hayabusa.exe
Publish documentation of scanning procedures when using VSS
- pros: Operations related to vss are performed at the user's own risk. (So it's easy to isolate problem)
- cons: Users unaware of the document cannot scan vss

IMHO, I think it would be good to start with documentation above(Hoping to find better means of realization during writing document and research ... 😇)

YamatoSecurity commented 2 months ago

Memo: I can run Hayabusa against the volume shadow backup with .\hayabusa-2.15.0-win-x64.exe csv-timeline -w -d \\.\HarddiskVolumeShadowCopy2\Windows\System32\winevt\Logs without having to make a link first.

@fukusuket I think we can run against backup logs without making a link like this.

I think we should go with number 1 and implement in Hayabusa. If it doesn't work for some reason due to a difference environment then hopefully someone will create an issue and we can go from there.

fukusuket commented 2 months ago

Sorry, I missed the above comment. I see! If mklink is not needed, then my concerns are addressed. I'll check with Server OS and try to implement following specification💪

-l, --live-analysis option is required
Check if it is Server OS or not(also check -l, --live-analysis); if not Server OS, exit with a message
Add --scan-vss-backups in General Options(Only csv-timeline/json-timeline)
If --scan-vss-backups specified, then scan all volume shadow in addition to C:\Windows\System32\winevt\Logs\

YamatoSecurity commented 2 months ago

@fukusuket Since this is not necessarily dependent on whether it is a server OS, I think we should leave out the server check. That is, some people may be using Hayabusa on Windows 7, also some people may change the registry settings to get proper backups on Windows 8/10/11. We should though print some warning message that this feature may not work depending on the setting. I will think about what message we should tell the user. Before you implement this, we should check if it is possible to scan directly or if we need to copy out the files first, like in my case. If we have to copy out the files, then things will be a little bit more tricky... I'll do some testing and let you know, so please hold on for now.

YamatoSecurity commented 2 months ago

@fukusuket I found just twos articles (from the same person) talking about this in English: https://dfir.ru/2020/02/29/scoped-shadow-copies/ and the IIJ article in English: https://www.iij.ad.jp/en/dev/iir/pdf/iir_vol37_focused1_EN.pdf So it seems that it is not so widely known.

One thing to watch out for is that vssadmin list shadows may not show all of the available shadows. (Explained in the article above). I am not sure if WMI gives all of the correct information. If not, we could probably just guess that path and iterate the last number looking for .evtx files until we get an error.

To determine whether a Windows system may have usable VSS backups, we probably should check the registry key HKLM\Software\Microsoft\ Windows NT\CurrentVersion\SystemRestore to see if it is 0 or not. If it is 1, we should probably warn the user that this function will probably not work and there will be parsing errors. (In some cases, user data does seem to be backed up... so depends on luck..)

Also, for server OSes, SystemRestore is set to 0 by default, but VSS snapshots are disabled by default so in most cases server OSes probably won't have snapshots to begin with... 😞

msuhanov commented 2 months ago

@fukusuket I found just twos articles (from the same person) talking about this in English: https://dfir.ru/2020/02/29/scoped-shadow-copies/ and the IIJ article in English: https://www.iij.ad.jp/en/dev/iir/pdf/iir_vol37_focused1_EN.pdf

JFYI: these articles are from different authors. (In fact, these are independent discoveries.)

YamatoSecurity commented 2 months ago

@msuhanov Sorry I forgot to post the second link. I meant the same author for these two articles: https://dfir.ru/2020/02/29/scoped-shadow-copies/ and https://dfir.ru/2019/07/29/things-you-probably-didnt-know-about-shadow-copies/ Thanks for the writeups! It helps out alot.

YamatoSecurity commented 2 months ago

So some weird things going on... When I try to run hayabusa-2.15.0-win-x64.exe csv-timeline -w -d "\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy4\Windows\System32\winevtx\Logs" it says that no .evtx files were found, but if I run hayabusa-2.15.0-win-x64.exe csv-timeline -w -d "\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy4\Windows\System32\" it will find them inside... =(

However, I still get the read errors and no detections. This is after setting the registry to turn off ScopeSnapshots, rebooting and creating new snapshots... It does work if I copy the evtx files out to a different directory though...

Yamato-Security / hayabusa