[Security Solution] Rule Update failure on 8.13 from 7.17.18

[ghost]

Epics: https://github.com/elastic/security-team/issues/1974 (internal), https://github.com/elastic/kibana/issues/174168

Summary

Describe the bug: Rule Update failure on 8.13 from 7.17.18

Kibana / Elastic search Stack version Version: 7.17.18 to 8.13.0 BC2

Browser and Browser OS Version: Chrome for macOS Version 122.0.6261.94 (Official Build) (x86_64)

Functional Area: Rule Update

precondition

• Kibana build should be available by Upgrading 7.17.18 to 8.13 BC2

Steps to reproduce

• Navigate to kibana//app/security/rules/updates
• On our upgrade path we see 4 rule available for update

Microsoft Exchange Worker Spawning Suspicious Processes

Potential DLL Side-Loading via Microsoft Antimalware Service Executable

Peripheral Device Discovery

AWS S3 Bucket Configuration Deletion

• Click on update rule
• Observed that rule is not getting update

Additional Result

• No error message is received to get reason of rule update failure
• tick mark is shown in rule update notification which is conflicting to the actuall outcome

Current Result

• Rule Update failure on 8.13 from 7.17.18

Expected Result

• rule update should be successful

Screen-Cast:

https://github.com/elastic/kibana/assets/59917825/6eadb902-b20e-4fa0-b7c3-0e27ee521c8e

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[amolnater-qasource]

Reviewed & assigned to @MadameSheema

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[banderror]

I was given access to the environment where this bug can be reproduced. There are 4 rules for which the upgrade fails:

• Microsoft Exchange Worker Spawning Suspicious Processes
• Potential DLL Side-Loading via Microsoft Antimalware Service Executable
• Peripheral Device Discovery
• AWS S3 Bucket Configuration Deletion

Here's a response from the /internal/detection_engine/prebuilt_rules/upgrade/_review endpoint that is used to populate the Rule Updates table:

```json { "stats": { "num_rules_to_upgrade_total": 4, "tags": [ "Asset Visibility", "auto_disabled_8.0", "AWS", "Cloud", "Continuous Monitoring", "Defense Evasion", "Discovery", "Elastic", "Host", "Initial Access", "SecOps", "Threat Detection", "Windows" ] }, "rules": [ { "id": "76528ff1-d32f-11ee-a96b-438c4c44df86", "rule_id": "f81ee52c-297e-46d9-9205-07e66931df26", "revision": 6, "current_rule": { "id": "76528ff1-d32f-11ee-a96b-438c4c44df86", "updated_at": "2024-02-24T16:58:56.299Z", "updated_by": "elastic", "created_at": "2024-02-24T16:12:11.572Z", "created_by": "elastic", "name": "Microsoft Exchange Worker Spawning Suspicious Processes", "tags": [ "Elastic", "Host", "Windows", "Threat Detection", "Initial Access", "auto_disabled_8.0" ], "interval": "5m", "enabled": false, "revision": 6, "description": "Identifies suspicious processes being spawned by the Microsoft Exchange Server worker process (w3wp). This activity may indicate exploitation activity or access to an existing web shell backdoor.", "risk_score": 73, "severity": "high", "note": "## Setup\n\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2, events will not define `event.ingested` and default fallback for EQL rules was not added until 8.2, so you will need to add a custom pipeline to populate `event.ingested` to @timestamp for this rule to work.\n", "license": "Elastic License v2", "output_index": "", "timestamp_override": "event.ingested", "author": [ "Elastic" ], "false_positives": [], "from": "now-9m", "rule_id": "f81ee52c-297e-46d9-9205-07e66931df26", "max_signals": 100, "risk_score_mapping": [], "severity_mapping": [], "threat": [ { "framework": "MITRE ATT&CK", "technique": [ { "reference": "https://attack.mitre.org/techniques/T1190/", "name": "Exploit Public-Facing Application", "id": "T1190" } ], "tactic": { "reference": "https://attack.mitre.org/tactics/TA0001/", "name": "Initial Access", "id": "TA0001" } } ], "to": "now", "references": [ "https://www.microsoft.com/security/blog/2021/03/02/hafnium-targeting-exchange-servers", "https://www.volexity.com/blog/2021/03/02/active-exploitation-of-microsoft-exchange-zero-day-vulnerabilities", "https://discuss.elastic.co/t/detection-and-response-for-hafnium-activity/266289" ], "version": 6, "exceptions_list": [], "immutable": true, "related_integrations": [], "required_fields": [], "setup": "", "type": "eql", "language": "eql", "index": [ "logs-endpoint.events.*", "winlogbeat-*", "logs-windows.*" ], "query": "process where event.type == \"start\" and\n process.parent.name : \"w3wp.exe\" and process.parent.args : \"MSExchange*AppPool\" and\n (process.name : (\"cmd.exe\", \"powershell.exe\", \"pwsh.exe\", \"powershell_ise.exe\") or\n process.pe.original_file_name in (\"cmd.exe\", \"powershell.exe\", \"pwsh.dll\", \"powershell_ise.exe\"))\n", "actions": [], "execution_summary": { "last_execution": { "date": "2024-02-26T04:53:11.486Z", "status": "failed", "status_order": 30, "message": "authenticationStart is not registered!", "metrics": {} } } }, "target_rule": { "name": "Microsoft Exchange Worker Spawning Suspicious Processes", "description": "Identifies suspicious processes being spawned by the Microsoft Exchange Server worker process (w3wp). This activity may indicate exploitation activity or access to an existing web shell backdoor.", "risk_score": 73, "severity": "high", "timestamp_override": "event.ingested", "license": "Elastic License v2", "note": "", "version": 107, "tags": [ "Domain: Endpoint", "OS: Windows", "Use Case: Threat Detection", "Tactic: Initial Access", "Tactic: Execution", "Data Source: Elastic Endgame", "Data Source: Elastic Defend" ], "enabled": false, "risk_score_mapping": [], "severity_mapping": [], "interval": "5m", "from": "now-9m", "to": "now", "actions": [], "exceptions_list": [], "author": [ "Elastic" ], "false_positives": [], "references": [ "https://www.microsoft.com/security/blog/2021/03/02/hafnium-targeting-exchange-servers", "https://www.volexity.com/blog/2021/03/02/active-exploitation-of-microsoft-exchange-zero-day-vulnerabilities", "https://discuss.elastic.co/t/detection-and-response-for-hafnium-activity/266289" ], "max_signals": 100, "threat": [ { "framework": "MITRE ATT&CK", "tactic": { "id": "TA0001", "name": "Initial Access", "reference": "https://attack.mitre.org/tactics/TA0001/" }, "technique": [ { "id": "T1190", "name": "Exploit Public-Facing Application", "reference": "https://attack.mitre.org/techniques/T1190/" } ] }, { "framework": "MITRE ATT&CK", "tactic": { "id": "TA0002", "name": "Execution", "reference": "https://attack.mitre.org/tactics/TA0002/" }, "technique": [ { "id": "T1059", "name": "Command and Scripting Interpreter", "reference": "https://attack.mitre.org/techniques/T1059/", "subtechnique": [ { "id": "T1059.001", "name": "PowerShell", "reference": "https://attack.mitre.org/techniques/T1059/001/" }, { "id": "T1059.003", "name": "Windows Command Shell", "reference": "https://attack.mitre.org/techniques/T1059/003/" } ] } ] } ], "id": "76528ff1-d32f-11ee-a96b-438c4c44df86", "rule_id": "f81ee52c-297e-46d9-9205-07e66931df26", "immutable": true, "updated_at": "2024-02-29T12:32:31.820Z", "updated_by": "elastic", "created_at": "2024-02-24T16:12:11.572Z", "created_by": "elastic", "revision": 7, "related_integrations": [ { "package": "endpoint", "version": "^8.2.0" }, { "package": "windows", "version": "^1.5.0" } ], "required_fields": [ { "name": "event.type", "type": "keyword", "ecs": true }, { "name": "host.os.type", "type": "keyword", "ecs": true }, { "name": "process.name", "type": "keyword", "ecs": true }, { "name": "process.parent.args", "type": "keyword", "ecs": true }, { "name": "process.parent.name", "type": "keyword", "ecs": true }, { "name": "process.pe.original_file_name", "type": "keyword", "ecs": true } ], "setup": "\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2,\nevents will not define `event.ingested` and default fallback for EQL rules was not added until version 8.2.\nHence for this rule to work effectively, users will need to add a custom ingest pipeline to populate\n`event.ingested` to @timestamp.\nFor more details on adding a custom ingest pipeline refer - https://www.elastic.co/guide/en/fleet/current/data-streams-pipeline-tutorial.html\n", "type": "eql", "query": "process where host.os.type == \"windows\" and event.type == \"start\" and\n process.parent.name : \"w3wp.exe\" and process.parent.args : \"MSExchange*AppPool\" and\n (process.name : (\"cmd.exe\", \"powershell.exe\", \"pwsh.exe\", \"powershell_ise.exe\") or\n ?process.pe.original_file_name in (\"cmd.exe\", \"powershell.exe\", \"pwsh.dll\", \"powershell_ise.exe\"))\n", "language": "eql", "index": [ "logs-endpoint.events.*", "winlogbeat-*", "logs-windows.*", "endgame-*" ] }, "diff": { "fields": { "version": { "current_version": 6, "target_version": 107, "merged_version": 107, "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "tags": { "current_version": [ "Elastic", "Host", "Windows", "Threat Detection", "Initial Access", "auto_disabled_8.0" ], "target_version": [ "Domain: Endpoint", "OS: Windows", "Use Case: Threat Detection", "Tactic: Initial Access", "Tactic: Execution", "Data Source: Elastic Endgame", "Data Source: Elastic Defend" ], "merged_version": [ "Domain: Endpoint", "OS: Windows", "Use Case: Threat Detection", "Tactic: Initial Access", "Tactic: Execution", "Data Source: Elastic Endgame", "Data Source: Elastic Defend" ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "threat": { "current_version": [ { "framework": "MITRE ATT&CK", "technique": [ { "reference": "https://attack.mitre.org/techniques/T1190/", "name": "Exploit Public-Facing Application", "id": "T1190" } ], "tactic": { "reference": "https://attack.mitre.org/tactics/TA0001/", "name": "Initial Access", "id": "TA0001" } } ], "target_version": [ { "framework": "MITRE ATT&CK", "tactic": { "id": "TA0001", "name": "Initial Access", "reference": "https://attack.mitre.org/tactics/TA0001/" }, "technique": [ { "id": "T1190", "name": "Exploit Public-Facing Application", "reference": "https://attack.mitre.org/techniques/T1190/" } ] }, { "framework": "MITRE ATT&CK", "tactic": { "id": "TA0002", "name": "Execution", "reference": "https://attack.mitre.org/tactics/TA0002/" }, "technique": [ { "id": "T1059", "name": "Command and Scripting Interpreter", "reference": "https://attack.mitre.org/techniques/T1059/", "subtechnique": [ { "id": "T1059.001", "name": "PowerShell", "reference": "https://attack.mitre.org/techniques/T1059/001/" }, { "id": "T1059.003", "name": "Windows Command Shell", "reference": "https://attack.mitre.org/techniques/T1059/003/" } ] } ] } ], "merged_version": [ { "framework": "MITRE ATT&CK", "tactic": { "id": "TA0001", "name": "Initial Access", "reference": "https://attack.mitre.org/tactics/TA0001/" }, "technique": [ { "id": "T1190", "name": "Exploit Public-Facing Application", "reference": "https://attack.mitre.org/techniques/T1190/" } ] }, { "framework": "MITRE ATT&CK", "tactic": { "id": "TA0002", "name": "Execution", "reference": "https://attack.mitre.org/tactics/TA0002/" }, "technique": [ { "id": "T1059", "name": "Command and Scripting Interpreter", "reference": "https://attack.mitre.org/techniques/T1059/", "subtechnique": [ { "id": "T1059.001", "name": "PowerShell", "reference": "https://attack.mitre.org/techniques/T1059/001/" }, { "id": "T1059.003", "name": "Windows Command Shell", "reference": "https://attack.mitre.org/techniques/T1059/003/" } ] } ] } ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "note": { "current_version": "## Setup\n\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2, events will not define `event.ingested` and default fallback for EQL rules was not added until 8.2, so you will need to add a custom pipeline to populate `event.ingested` to @timestamp for this rule to work.\n", "target_version": "", "merged_version": "", "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "setup": { "current_version": "", "target_version": "\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2,\nevents will not define `event.ingested` and default fallback for EQL rules was not added until version 8.2.\nHence for this rule to work effectively, users will need to add a custom ingest pipeline to populate\n`event.ingested` to @timestamp.\nFor more details on adding a custom ingest pipeline refer - https://www.elastic.co/guide/en/fleet/current/data-streams-pipeline-tutorial.html\n", "merged_version": "\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2,\nevents will not define `event.ingested` and default fallback for EQL rules was not added until version 8.2.\nHence for this rule to work effectively, users will need to add a custom ingest pipeline to populate\n`event.ingested` to @timestamp.\nFor more details on adding a custom ingest pipeline refer - https://www.elastic.co/guide/en/fleet/current/data-streams-pipeline-tutorial.html\n", "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "related_integrations": { "current_version": [], "target_version": [ { "package": "endpoint", "version": "^8.2.0" }, { "package": "windows", "version": "^1.5.0" } ], "merged_version": [ { "package": "endpoint", "version": "^8.2.0" }, { "package": "windows", "version": "^1.5.0" } ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "required_fields": { "current_version": [], "target_version": [ { "name": "event.type", "type": "keyword", "ecs": true }, { "name": "host.os.type", "type": "keyword", "ecs": true }, { "name": "process.name", "type": "keyword", "ecs": true }, { "name": "process.parent.args", "type": "keyword", "ecs": true }, { "name": "process.parent.name", "type": "keyword", "ecs": true }, { "name": "process.pe.original_file_name", "type": "keyword", "ecs": true } ], "merged_version": [ { "name": "event.type", "type": "keyword", "ecs": true }, { "name": "host.os.type", "type": "keyword", "ecs": true }, { "name": "process.name", "type": "keyword", "ecs": true }, { "name": "process.parent.args", "type": "keyword", "ecs": true }, { "name": "process.parent.name", "type": "keyword", "ecs": true }, { "name": "process.pe.original_file_name", "type": "keyword", "ecs": true } ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "eql_query": { "current_version": { "query": "process where event.type == \"start\" and\n process.parent.name : \"w3wp.exe\" and process.parent.args : \"MSExchange*AppPool\" and\n (process.name : (\"cmd.exe\", \"powershell.exe\", \"pwsh.exe\", \"powershell_ise.exe\") or\n process.pe.original_file_name in (\"cmd.exe\", \"powershell.exe\", \"pwsh.dll\", \"powershell_ise.exe\"))\n", "language": "eql", "filters": [] }, "target_version": { "query": "process where host.os.type == \"windows\" and event.type == \"start\" and\n process.parent.name : \"w3wp.exe\" and process.parent.args : \"MSExchange*AppPool\" and\n (process.name : (\"cmd.exe\", \"powershell.exe\", \"pwsh.exe\", \"powershell_ise.exe\") or\n ?process.pe.original_file_name in (\"cmd.exe\", \"powershell.exe\", \"pwsh.dll\", \"powershell_ise.exe\"))\n", "language": "eql", "filters": [] }, "merged_version": { "query": "process where host.os.type == \"windows\" and event.type == \"start\" and\n process.parent.name : \"w3wp.exe\" and process.parent.args : \"MSExchange*AppPool\" and\n (process.name : (\"cmd.exe\", \"powershell.exe\", \"pwsh.exe\", \"powershell_ise.exe\") or\n ?process.pe.original_file_name in (\"cmd.exe\", \"powershell.exe\", \"pwsh.dll\", \"powershell_ise.exe\"))\n", "language": "eql", "filters": [] }, "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "data_source": { "current_version": { "type": "index_patterns", "index_patterns": [ "logs-endpoint.events.*", "winlogbeat-*", "logs-windows.*" ] }, "target_version": { "type": "index_patterns", "index_patterns": [ "logs-endpoint.events.*", "winlogbeat-*", "logs-windows.*", "endgame-*" ] }, "merged_version": { "type": "index_patterns", "index_patterns": [ "logs-endpoint.events.*", "winlogbeat-*", "logs-windows.*", "endgame-*" ] }, "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false } }, "has_conflict": false } }, { "id": "76530526-d32f-11ee-a96b-438c4c44df86", "rule_id": "053a0387-f3b5-4ba5-8245-8002cca2bd08", "revision": 6, "current_rule": { "id": "76530526-d32f-11ee-a96b-438c4c44df86", "updated_at": "2024-02-24T16:58:57.003Z", "updated_by": "elastic", "created_at": "2024-02-24T16:12:11.222Z", "created_by": "elastic", "name": "Potential DLL Side-Loading via Microsoft Antimalware Service Executable", "tags": [ "Elastic", "Host", "Windows", "Threat Detection", "Defense Evasion", "auto_disabled_8.0" ], "interval": "5m", "enabled": false, "revision": 6, "description": "Identifies a Windows trusted program that is known to be vulnerable to DLL Search Order Hijacking starting after being renamed or from a non-standard path. This is uncommon behavior and may indicate an attempt to evade defenses via side-loading a malicious DLL within the memory space of one of those processes.", "risk_score": 73, "severity": "high", "note": "## Setup\n\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2, events will not define `event.ingested` and default fallback for EQL rules was not added until 8.2, so you will need to add a custom pipeline to populate `event.ingested` to @timestamp for this rule to work.\n", "license": "Elastic License v2", "output_index": "", "timestamp_override": "event.ingested", "author": [ "Elastic", "Dennis Perto" ], "false_positives": [ "Microsoft Antimalware Service Executable installed on non default installation path." ], "from": "now-9m", "rule_id": "053a0387-f3b5-4ba5-8245-8002cca2bd08", "max_signals": 100, "risk_score_mapping": [], "severity_mapping": [], "threat": [ { "framework": "MITRE ATT&CK", "technique": [ { "reference": "https://attack.mitre.org/techniques/T1574/", "name": "Hijack Execution Flow", "subtechnique": [ { "reference": "https://attack.mitre.org/techniques/T1574/002/", "name": "DLL Side-Loading", "id": "T1574.002" } ], "id": "T1574" } ], "tactic": { "reference": "https://attack.mitre.org/tactics/TA0005/", "name": "Defense Evasion", "id": "TA0005" } } ], "to": "now", "references": [ "https://news.sophos.com/en-us/2021/07/04/independence-day-revil-uses-supply-chain-exploit-to-attack-hundreds-of-businesses/" ], "version": 6, "exceptions_list": [], "immutable": true, "related_integrations": [], "required_fields": [], "setup": "", "type": "eql", "language": "eql", "index": [ "winlogbeat-*", "logs-endpoint.events.*", "logs-windows.*" ], "query": "process where event.type == \"start\" and\n (process.pe.original_file_name == \"MsMpEng.exe\" and not process.name : \"MsMpEng.exe\") or\n (process.name : \"MsMpEng.exe\" and not\n process.executable : (\"?:\\\\ProgramData\\\\Microsoft\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files (x86)\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files\\\\Microsoft Security Client\\\\*.exe\",\n \"?:\\\\Program Files (x86)\\\\Microsoft Security Client\\\\*.exe\"))\n", "actions": [], "execution_summary": { "last_execution": { "date": "2024-02-26T04:53:11.485Z", "status": "failed", "status_order": 30, "message": "authenticationStart is not registered!", "metrics": {} } } }, "target_rule": { "name": "Potential DLL Side-Loading via Microsoft Antimalware Service Executable", "description": "Identifies a Windows trusted program that is known to be vulnerable to DLL Search Order Hijacking starting after being renamed or from a non-standard path. This is uncommon behavior and may indicate an attempt to evade defenses via side-loading a malicious DLL within the memory space of one of those processes.", "risk_score": 73, "severity": "high", "timestamp_override": "event.ingested", "license": "Elastic License v2", "note": "", "version": 108, "tags": [ "Domain: Endpoint", "OS: Windows", "Use Case: Threat Detection", "Tactic: Defense Evasion", "Data Source: Elastic Endgame", "Tactic: Execution", "Data Source: Elastic Defend" ], "enabled": false, "risk_score_mapping": [], "severity_mapping": [], "interval": "5m", "from": "now-9m", "to": "now", "actions": [], "exceptions_list": [], "author": [ "Elastic", "Dennis Perto" ], "false_positives": [ "Microsoft Antimalware Service Executable installed on non default installation path." ], "references": [ "https://news.sophos.com/en-us/2021/07/04/independence-day-revil-uses-supply-chain-exploit-to-attack-hundreds-of-businesses/" ], "max_signals": 100, "threat": [ { "framework": "MITRE ATT&CK", "tactic": { "id": "TA0005", "name": "Defense Evasion", "reference": "https://attack.mitre.org/tactics/TA0005/" }, "technique": [ { "id": "T1574", "name": "Hijack Execution Flow", "reference": "https://attack.mitre.org/techniques/T1574/", "subtechnique": [ { "id": "T1574.002", "name": "DLL Side-Loading", "reference": "https://attack.mitre.org/techniques/T1574/002/" } ] } ] } ], "id": "76530526-d32f-11ee-a96b-438c4c44df86", "rule_id": "053a0387-f3b5-4ba5-8245-8002cca2bd08", "immutable": true, "updated_at": "2024-02-29T12:32:31.820Z", "updated_by": "elastic", "created_at": "2024-02-24T16:12:11.222Z", "created_by": "elastic", "revision": 7, "related_integrations": [ { "package": "endpoint", "version": "^8.2.0" }, { "package": "windows", "version": "^1.5.0" } ], "required_fields": [ { "name": "event.type", "type": "keyword", "ecs": true }, { "name": "host.os.type", "type": "keyword", "ecs": true }, { "name": "process.executable", "type": "keyword", "ecs": true }, { "name": "process.name", "type": "keyword", "ecs": true }, { "name": "process.pe.original_file_name", "type": "keyword", "ecs": true } ], "setup": "\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2,\nevents will not define `event.ingested` and default fallback for EQL rules was not added until version 8.2.\nHence for this rule to work effectively, users will need to add a custom ingest pipeline to populate\n`event.ingested` to @timestamp.\nFor more details on adding a custom ingest pipeline refer - https://www.elastic.co/guide/en/fleet/current/data-streams-pipeline-tutorial.html\n", "type": "eql", "query": "process where host.os.type == \"windows\" and event.type == \"start\" and\n(\n (process.pe.original_file_name == \"MsMpEng.exe\" and not process.name : \"MsMpEng.exe\") or\n (process.name : \"MsMpEng.exe\" and not\n process.executable : (\"?:\\\\ProgramData\\\\Microsoft\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files (x86)\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files\\\\Microsoft Security Client\\\\*.exe\",\n \"?:\\\\Program Files (x86)\\\\Microsoft Security Client\\\\*.exe\"))\n)\n", "language": "eql", "index": [ "winlogbeat-*", "logs-endpoint.events.*", "logs-windows.*", "endgame-*" ] }, "diff": { "fields": { "version": { "current_version": 6, "target_version": 108, "merged_version": 108, "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "tags": { "current_version": [ "Elastic", "Host", "Windows", "Threat Detection", "Defense Evasion", "auto_disabled_8.0" ], "target_version": [ "Domain: Endpoint", "OS: Windows", "Use Case: Threat Detection", "Tactic: Defense Evasion", "Data Source: Elastic Endgame", "Tactic: Execution", "Data Source: Elastic Defend" ], "merged_version": [ "Domain: Endpoint", "OS: Windows", "Use Case: Threat Detection", "Tactic: Defense Evasion", "Data Source: Elastic Endgame", "Tactic: Execution", "Data Source: Elastic Defend" ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "note": { "current_version": "## Setup\n\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2, events will not define `event.ingested` and default fallback for EQL rules was not added until 8.2, so you will need to add a custom pipeline to populate `event.ingested` to @timestamp for this rule to work.\n", "target_version": "", "merged_version": "", "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "setup": { "current_version": "", "target_version": "\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2,\nevents will not define `event.ingested` and default fallback for EQL rules was not added until version 8.2.\nHence for this rule to work effectively, users will need to add a custom ingest pipeline to populate\n`event.ingested` to @timestamp.\nFor more details on adding a custom ingest pipeline refer - https://www.elastic.co/guide/en/fleet/current/data-streams-pipeline-tutorial.html\n", "merged_version": "\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2,\nevents will not define `event.ingested` and default fallback for EQL rules was not added until version 8.2.\nHence for this rule to work effectively, users will need to add a custom ingest pipeline to populate\n`event.ingested` to @timestamp.\nFor more details on adding a custom ingest pipeline refer - https://www.elastic.co/guide/en/fleet/current/data-streams-pipeline-tutorial.html\n", "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "related_integrations": { "current_version": [], "target_version": [ { "package": "endpoint", "version": "^8.2.0" }, { "package": "windows", "version": "^1.5.0" } ], "merged_version": [ { "package": "endpoint", "version": "^8.2.0" }, { "package": "windows", "version": "^1.5.0" } ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "required_fields": { "current_version": [], "target_version": [ { "name": "event.type", "type": "keyword", "ecs": true }, { "name": "host.os.type", "type": "keyword", "ecs": true }, { "name": "process.executable", "type": "keyword", "ecs": true }, { "name": "process.name", "type": "keyword", "ecs": true }, { "name": "process.pe.original_file_name", "type": "keyword", "ecs": true } ], "merged_version": [ { "name": "event.type", "type": "keyword", "ecs": true }, { "name": "host.os.type", "type": "keyword", "ecs": true }, { "name": "process.executable", "type": "keyword", "ecs": true }, { "name": "process.name", "type": "keyword", "ecs": true }, { "name": "process.pe.original_file_name", "type": "keyword", "ecs": true } ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "eql_query": { "current_version": { "query": "process where event.type == \"start\" and\n (process.pe.original_file_name == \"MsMpEng.exe\" and not process.name : \"MsMpEng.exe\") or\n (process.name : \"MsMpEng.exe\" and not\n process.executable : (\"?:\\\\ProgramData\\\\Microsoft\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files (x86)\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files\\\\Microsoft Security Client\\\\*.exe\",\n \"?:\\\\Program Files (x86)\\\\Microsoft Security Client\\\\*.exe\"))\n", "language": "eql", "filters": [] }, "target_version": { "query": "process where host.os.type == \"windows\" and event.type == \"start\" and\n(\n (process.pe.original_file_name == \"MsMpEng.exe\" and not process.name : \"MsMpEng.exe\") or\n (process.name : \"MsMpEng.exe\" and not\n process.executable : (\"?:\\\\ProgramData\\\\Microsoft\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files (x86)\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files\\\\Microsoft Security Client\\\\*.exe\",\n \"?:\\\\Program Files (x86)\\\\Microsoft Security Client\\\\*.exe\"))\n)\n", "language": "eql", "filters": [] }, "merged_version": { "query": "process where host.os.type == \"windows\" and event.type == \"start\" and\n(\n (process.pe.original_file_name == \"MsMpEng.exe\" and not process.name : \"MsMpEng.exe\") or\n (process.name : \"MsMpEng.exe\" and not\n process.executable : (\"?:\\\\ProgramData\\\\Microsoft\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files (x86)\\\\Windows Defender\\\\*.exe\",\n \"?:\\\\Program Files\\\\Microsoft Security Client\\\\*.exe\",\n \"?:\\\\Program Files (x86)\\\\Microsoft Security Client\\\\*.exe\"))\n)\n", "language": "eql", "filters": [] }, "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "data_source": { "current_version": { "type": "index_patterns", "index_patterns": [ "winlogbeat-*", "logs-endpoint.events.*", "logs-windows.*" ] }, "target_version": { "type": "index_patterns", "index_patterns": [ "winlogbeat-*", "logs-endpoint.events.*", "logs-windows.*", "endgame-*" ] }, "merged_version": { "type": "index_patterns", "index_patterns": [ "winlogbeat-*", "logs-endpoint.events.*", "logs-windows.*", "endgame-*" ] }, "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false } }, "has_conflict": false } }, { "id": "76510959-d32f-11ee-a96b-438c4c44df86", "rule_id": "0c7ca5c2-728d-4ad9-b1c5-bbba83ecb1f4", "revision": 8, "current_rule": { "id": "76510959-d32f-11ee-a96b-438c4c44df86", "updated_at": "2024-02-24T16:58:56.308Z", "updated_by": "elastic", "created_at": "2024-02-24T16:12:10.434Z", "created_by": "elastic", "name": "Peripheral Device Discovery", "tags": [ "Elastic", "Host", "Windows", "Threat Detection", "Discovery", "auto_disabled_8.0" ], "interval": "5m", "enabled": false, "revision": 8, "description": "Identifies use of the Windows file system utility (fsutil.exe) to gather information about attached peripheral devices and components connected to a computer system.", "risk_score": 21, "severity": "low", "note": "## Triage and analysis\n\n### Investigating Peripheral Device Discovery\n\nAfter successfully compromising an environment, attackers may try to gain situational awareness to plan their next steps.\nThis can happen by running commands to enumerate network resources, users, connections, files, and installed security\nsoftware.\n\nThis rule looks for the execution of the `fsutil` utility with the `fsinfo` subcommand to enumerate drives attached to\nthe computer, which can be used to identify secondary drives used for backups, mapped network drives, and removable\nmedia. These devices can contain valuable information for attackers.\n\n#### Possible investigation steps\n\n- Investigate the process execution chain (parent process tree) for unknown processes. Examine their executable files\nfor prevalence, whether they are located in expected locations, and if they are signed with valid digital signatures.\n- Identify the user account that performed the action and whether it should perform this kind of action.\n- Investigate other alerts associated with the user/host during the past 48 hours.\n- Investigate abnormal behaviors observed using the account, such as commands executed, files created or modified, and\nnetwork connections.\n- Determine whether this activity was followed by suspicious file access/copy operations or uploads to file storage\nservices.\n\n### False positive analysis\n\n- Discovery activities are not inherently malicious if they occur in isolation. As long as the analyst did not identify\nsuspicious activity related to the user or host, such alerts can be dismissed.\n\n### Response and remediation\n\n- Initiate the incident response process based on the outcome of the triage.\n- Isolate the involved hosts to prevent further post-compromise behavior.\n- Investigate credential exposure on systems compromised or used by the attacker to ensure all compromised accounts are\nidentified. Reset passwords for these accounts and other potentially compromised credentials, such as email, business\nsystems, and web services.\n- Run a full antimalware scan. This may reveal additional artifacts left in the system, persistence mechanisms, and\nmalware components.\n- Determine the initial vector abused by the attacker and take action to prevent reinfection via the same vector.\n- Using the incident response data, update logging and audit policies to improve the mean time to detect (MTTD) and the\nmean time to respond (MTTR).\n\n## Setup\n\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2, events will not define `event.ingested` and default fallback for EQL rules was not added until 8.2, so you will need to add a custom pipeline to populate `event.ingested` to @timestamp for this rule to work.\n", "license": "Elastic License v2", "output_index": "", "timestamp_override": "event.ingested", "author": [ "Elastic" ], "false_positives": [], "from": "now-9m", "rule_id": "0c7ca5c2-728d-4ad9-b1c5-bbba83ecb1f4", "max_signals": 100, "risk_score_mapping": [], "severity_mapping": [], "threat": [ { "framework": "MITRE ATT&CK", "technique": [ { "reference": "https://attack.mitre.org/techniques/T1120/", "name": "Peripheral Device Discovery", "id": "T1120" } ], "tactic": { "reference": "https://attack.mitre.org/tactics/TA0007/", "name": "Discovery", "id": "TA0007" } } ], "to": "now", "references": [], "version": 8, "exceptions_list": [], "immutable": true, "related_integrations": [], "required_fields": [], "setup": "", "type": "eql", "language": "eql", "index": [ "winlogbeat-*", "logs-endpoint.events.*", "logs-windows.*" ], "query": "process where event.type in (\"start\", \"process_started\") and\n (process.name : \"fsutil.exe\" or process.pe.original_file_name == \"fsutil.exe\") and\n process.args : \"fsinfo\" and process.args : \"drives\"\n", "actions": [], "execution_summary": { "last_execution": { "date": "2024-02-26T04:53:11.487Z", "status": "failed", "status_order": 30, "message": "authenticationStart is not registered!", "metrics": {} } } }, "target_rule": { "name": "Peripheral Device Discovery", "description": "Identifies use of the Windows file system utility (fsutil.exe) to gather information about attached peripheral devices and components connected to a computer system.", "risk_score": 21, "severity": "low", "timestamp_override": "event.ingested", "license": "Elastic License v2", "note": "## Triage and analysis\n\n### Investigating Peripheral Device Discovery\n\nAfter successfully compromising an environment, attackers may try to gain situational awareness to plan their next steps. This can happen by running commands to enumerate network resources, users, connections, files, and installed security software.\n\nThis rule looks for the execution of the `fsutil` utility with the `fsinfo` subcommand to enumerate drives attached to the computer, which can be used to identify secondary drives used for backups, mapped network drives, and removable media. These devices can contain valuable information for attackers.\n\n#### Possible investigation steps\n\n- Investigate the process execution chain (parent process tree) for unknown processes. Examine their executable files for prevalence, whether they are located in expected locations, and if they are signed with valid digital signatures.\n- Identify the user account that performed the action and whether it should perform this kind of action.\n- Investigate other alerts associated with the user/host during the past 48 hours.\n- Investigate any abnormal account behavior, such as command executions, file creations or modifications, and network connections.\n- Determine whether this activity was followed by suspicious file access/copy operations or uploads to file storage services.\n\n### False positive analysis\n\n- Discovery activities are not inherently malicious if they occur in isolation. As long as the analyst did not identify suspicious activity related to the user or host, such alerts can be dismissed.\n\n### Response and remediation\n\n- Initiate the incident response process based on the outcome of the triage.\n- Isolate the involved hosts to prevent further post-compromise behavior.\n- Investigate credential exposure on systems compromised or used by the attacker to ensure all compromised accounts are identified. Reset passwords for these accounts and other potentially compromised credentials, such as email, business systems, and web services.\n- Run a full antimalware scan. This may reveal additional artifacts left in the system, persistence mechanisms, and malware components.\n- Determine the initial vector abused by the attacker and take action to prevent reinfection through the same vector.\n- Using the incident response data, update logging and audit policies to improve the mean time to detect (MTTD) and the mean time to respond (MTTR).\n\n", "version": 107, "tags": [ "Domain: Endpoint", "OS: Windows", "Use Case: Threat Detection", "Tactic: Discovery", "Resources: Investigation Guide", "Data Source: Elastic Endgame", "Data Source: Elastic Defend" ], "enabled": false, "risk_score_mapping": [], "severity_mapping": [], "interval": "5m", "from": "now-9m", "to": "now", "actions": [], "exceptions_list": [], "author": [ "Elastic" ], "false_positives": [], "references": [], "max_signals": 100, "threat": [ { "framework": "MITRE ATT&CK", "tactic": { "id": "TA0007", "name": "Discovery", "reference": "https://attack.mitre.org/tactics/TA0007/" }, "technique": [ { "id": "T1120", "name": "Peripheral Device Discovery", "reference": "https://attack.mitre.org/techniques/T1120/" } ] } ], "id": "76510959-d32f-11ee-a96b-438c4c44df86", "rule_id": "0c7ca5c2-728d-4ad9-b1c5-bbba83ecb1f4", "immutable": true, "updated_at": "2024-02-29T12:32:31.821Z", "updated_by": "elastic", "created_at": "2024-02-24T16:12:10.434Z", "created_by": "elastic", "revision": 9, "related_integrations": [ { "package": "endpoint", "version": "^8.2.0" }, { "package": "windows", "version": "^1.5.0" } ], "required_fields": [ { "name": "event.type", "type": "keyword", "ecs": true }, { "name": "host.os.type", "type": "keyword", "ecs": true }, { "name": "process.args", "type": "keyword", "ecs": true }, { "name": "process.name", "type": "keyword", "ecs": true }, { "name": "process.pe.original_file_name", "type": "keyword", "ecs": true } ], "setup": "\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2,\nevents will not define `event.ingested` and default fallback for EQL rules was not added until version 8.2.\nHence for this rule to work effectively, users will need to add a custom ingest pipeline to populate\n`event.ingested` to @timestamp.\nFor more details on adding a custom ingest pipeline refer - https://www.elastic.co/guide/en/fleet/current/data-streams-pipeline-tutorial.html\n", "type": "eql", "query": "process where host.os.type == \"windows\" and event.type == \"start\" and\n (process.name : \"fsutil.exe\" or process.pe.original_file_name == \"fsutil.exe\") and\n process.args : \"fsinfo\" and process.args : \"drives\"\n", "language": "eql", "index": [ "winlogbeat-*", "logs-endpoint.events.*", "logs-windows.*", "endgame-*" ] }, "diff": { "fields": { "version": { "current_version": 8, "target_version": 107, "merged_version": 107, "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "tags": { "current_version": [ "Elastic", "Host", "Windows", "Threat Detection", "Discovery", "auto_disabled_8.0" ], "target_version": [ "Domain: Endpoint", "OS: Windows", "Use Case: Threat Detection", "Tactic: Discovery", "Resources: Investigation Guide", "Data Source: Elastic Endgame", "Data Source: Elastic Defend" ], "merged_version": [ "Domain: Endpoint", "OS: Windows", "Use Case: Threat Detection", "Tactic: Discovery", "Resources: Investigation Guide", "Data Source: Elastic Endgame", "Data Source: Elastic Defend" ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "note": { "current_version": "## Triage and analysis\n\n### Investigating Peripheral Device Discovery\n\nAfter successfully compromising an environment, attackers may try to gain situational awareness to plan their next steps.\nThis can happen by running commands to enumerate network resources, users, connections, files, and installed security\nsoftware.\n\nThis rule looks for the execution of the `fsutil` utility with the `fsinfo` subcommand to enumerate drives attached to\nthe computer, which can be used to identify secondary drives used for backups, mapped network drives, and removable\nmedia. These devices can contain valuable information for attackers.\n\n#### Possible investigation steps\n\n- Investigate the process execution chain (parent process tree) for unknown processes. Examine their executable files\nfor prevalence, whether they are located in expected locations, and if they are signed with valid digital signatures.\n- Identify the user account that performed the action and whether it should perform this kind of action.\n- Investigate other alerts associated with the user/host during the past 48 hours.\n- Investigate abnormal behaviors observed using the account, such as commands executed, files created or modified, and\nnetwork connections.\n- Determine whether this activity was followed by suspicious file access/copy operations or uploads to file storage\nservices.\n\n### False positive analysis\n\n- Discovery activities are not inherently malicious if they occur in isolation. As long as the analyst did not identify\nsuspicious activity related to the user or host, such alerts can be dismissed.\n\n### Response and remediation\n\n- Initiate the incident response process based on the outcome of the triage.\n- Isolate the involved hosts to prevent further post-compromise behavior.\n- Investigate credential exposure on systems compromised or used by the attacker to ensure all compromised accounts are\nidentified. Reset passwords for these accounts and other potentially compromised credentials, such as email, business\nsystems, and web services.\n- Run a full antimalware scan. This may reveal additional artifacts left in the system, persistence mechanisms, and\nmalware components.\n- Determine the initial vector abused by the attacker and take action to prevent reinfection via the same vector.\n- Using the incident response data, update logging and audit policies to improve the mean time to detect (MTTD) and the\nmean time to respond (MTTR).\n\n## Setup\n\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2, events will not define `event.ingested` and default fallback for EQL rules was not added until 8.2, so you will need to add a custom pipeline to populate `event.ingested` to @timestamp for this rule to work.\n", "target_version": "## Triage and analysis\n\n### Investigating Peripheral Device Discovery\n\nAfter successfully compromising an environment, attackers may try to gain situational awareness to plan their next steps. This can happen by running commands to enumerate network resources, users, connections, files, and installed security software.\n\nThis rule looks for the execution of the `fsutil` utility with the `fsinfo` subcommand to enumerate drives attached to the computer, which can be used to identify secondary drives used for backups, mapped network drives, and removable media. These devices can contain valuable information for attackers.\n\n#### Possible investigation steps\n\n- Investigate the process execution chain (parent process tree) for unknown processes. Examine their executable files for prevalence, whether they are located in expected locations, and if they are signed with valid digital signatures.\n- Identify the user account that performed the action and whether it should perform this kind of action.\n- Investigate other alerts associated with the user/host during the past 48 hours.\n- Investigate any abnormal account behavior, such as command executions, file creations or modifications, and network connections.\n- Determine whether this activity was followed by suspicious file access/copy operations or uploads to file storage services.\n\n### False positive analysis\n\n- Discovery activities are not inherently malicious if they occur in isolation. As long as the analyst did not identify suspicious activity related to the user or host, such alerts can be dismissed.\n\n### Response and remediation\n\n- Initiate the incident response process based on the outcome of the triage.\n- Isolate the involved hosts to prevent further post-compromise behavior.\n- Investigate credential exposure on systems compromised or used by the attacker to ensure all compromised accounts are identified. Reset passwords for these accounts and other potentially compromised credentials, such as email, business systems, and web services.\n- Run a full antimalware scan. This may reveal additional artifacts left in the system, persistence mechanisms, and malware components.\n- Determine the initial vector abused by the attacker and take action to prevent reinfection through the same vector.\n- Using the incident response data, update logging and audit policies to improve the mean time to detect (MTTD) and the mean time to respond (MTTR).\n\n", "merged_version": "## Triage and analysis\n\n### Investigating Peripheral Device Discovery\n\nAfter successfully compromising an environment, attackers may try to gain situational awareness to plan their next steps. This can happen by running commands to enumerate network resources, users, connections, files, and installed security software.\n\nThis rule looks for the execution of the `fsutil` utility with the `fsinfo` subcommand to enumerate drives attached to the computer, which can be used to identify secondary drives used for backups, mapped network drives, and removable media. These devices can contain valuable information for attackers.\n\n#### Possible investigation steps\n\n- Investigate the process execution chain (parent process tree) for unknown processes. Examine their executable files for prevalence, whether they are located in expected locations, and if they are signed with valid digital signatures.\n- Identify the user account that performed the action and whether it should perform this kind of action.\n- Investigate other alerts associated with the user/host during the past 48 hours.\n- Investigate any abnormal account behavior, such as command executions, file creations or modifications, and network connections.\n- Determine whether this activity was followed by suspicious file access/copy operations or uploads to file storage services.\n\n### False positive analysis\n\n- Discovery activities are not inherently malicious if they occur in isolation. As long as the analyst did not identify suspicious activity related to the user or host, such alerts can be dismissed.\n\n### Response and remediation\n\n- Initiate the incident response process based on the outcome of the triage.\n- Isolate the involved hosts to prevent further post-compromise behavior.\n- Investigate credential exposure on systems compromised or used by the attacker to ensure all compromised accounts are identified. Reset passwords for these accounts and other potentially compromised credentials, such as email, business systems, and web services.\n- Run a full antimalware scan. This may reveal additional artifacts left in the system, persistence mechanisms, and malware components.\n- Determine the initial vector abused by the attacker and take action to prevent reinfection through the same vector.\n- Using the incident response data, update logging and audit policies to improve the mean time to detect (MTTD) and the mean time to respond (MTTR).\n\n", "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "setup": { "current_version": "", "target_version": "\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2,\nevents will not define `event.ingested` and default fallback for EQL rules was not added until version 8.2.\nHence for this rule to work effectively, users will need to add a custom ingest pipeline to populate\n`event.ingested` to @timestamp.\nFor more details on adding a custom ingest pipeline refer - https://www.elastic.co/guide/en/fleet/current/data-streams-pipeline-tutorial.html\n", "merged_version": "\nIf enabling an EQL rule on a non-elastic-agent index (such as beats) for versions <8.2,\nevents will not define `event.ingested` and default fallback for EQL rules was not added until version 8.2.\nHence for this rule to work effectively, users will need to add a custom ingest pipeline to populate\n`event.ingested` to @timestamp.\nFor more details on adding a custom ingest pipeline refer - https://www.elastic.co/guide/en/fleet/current/data-streams-pipeline-tutorial.html\n", "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "related_integrations": { "current_version": [], "target_version": [ { "package": "endpoint", "version": "^8.2.0" }, { "package": "windows", "version": "^1.5.0" } ], "merged_version": [ { "package": "endpoint", "version": "^8.2.0" }, { "package": "windows", "version": "^1.5.0" } ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "required_fields": { "current_version": [], "target_version": [ { "name": "event.type", "type": "keyword", "ecs": true }, { "name": "host.os.type", "type": "keyword", "ecs": true }, { "name": "process.args", "type": "keyword", "ecs": true }, { "name": "process.name", "type": "keyword", "ecs": true }, { "name": "process.pe.original_file_name", "type": "keyword", "ecs": true } ], "merged_version": [ { "name": "event.type", "type": "keyword", "ecs": true }, { "name": "host.os.type", "type": "keyword", "ecs": true }, { "name": "process.args", "type": "keyword", "ecs": true }, { "name": "process.name", "type": "keyword", "ecs": true }, { "name": "process.pe.original_file_name", "type": "keyword", "ecs": true } ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "eql_query": { "current_version": { "query": "process where event.type in (\"start\", \"process_started\") and\n (process.name : \"fsutil.exe\" or process.pe.original_file_name == \"fsutil.exe\") and\n process.args : \"fsinfo\" and process.args : \"drives\"\n", "language": "eql", "filters": [] }, "target_version": { "query": "process where host.os.type == \"windows\" and event.type == \"start\" and\n (process.name : \"fsutil.exe\" or process.pe.original_file_name == \"fsutil.exe\") and\n process.args : \"fsinfo\" and process.args : \"drives\"\n", "language": "eql", "filters": [] }, "merged_version": { "query": "process where host.os.type == \"windows\" and event.type == \"start\" and\n (process.name : \"fsutil.exe\" or process.pe.original_file_name == \"fsutil.exe\") and\n process.args : \"fsinfo\" and process.args : \"drives\"\n", "language": "eql", "filters": [] }, "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "data_source": { "current_version": { "type": "index_patterns", "index_patterns": [ "winlogbeat-*", "logs-endpoint.events.*", "logs-windows.*" ] }, "target_version": { "type": "index_patterns", "index_patterns": [ "winlogbeat-*", "logs-endpoint.events.*", "logs-windows.*", "endgame-*" ] }, "merged_version": { "type": "index_patterns", "index_patterns": [ "winlogbeat-*", "logs-endpoint.events.*", "logs-windows.*", "endgame-*" ] }, "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false } }, "has_conflict": false } }, { "id": "76501ef6-d32f-11ee-a96b-438c4c44df86", "rule_id": "227dc608-e558-43d9-b521-150772250bae", "revision": 9, "current_rule": { "id": "76501ef6-d32f-11ee-a96b-438c4c44df86", "updated_at": "2024-02-24T16:58:56.315Z", "updated_by": "elastic", "created_at": "2024-02-24T16:12:11.182Z", "created_by": "elastic", "name": "AWS S3 Bucket Configuration Deletion", "tags": [ "Elastic", "Cloud", "AWS", "Continuous Monitoring", "SecOps", "Asset Visibility", "auto_disabled_8.0" ], "interval": "10m", "enabled": false, "revision": 9, "description": "Identifies the deletion of various Amazon Simple Storage Service (S3) bucket configuration components.", "risk_score": 21, "severity": "low", "note": "## Setup\n\nThe AWS Fleet integration, Filebeat module, or similarly structured data is required to be compatible with this rule.", "license": "Elastic License v2", "output_index": "", "timestamp_override": "event.ingested", "author": [ "Elastic" ], "false_positives": [ "Bucket components may be deleted by a system or network administrator. Verify whether the user identity, user agent, and/or hostname should be making changes in your environment. Bucket component deletions by unfamiliar users or hosts should be investigated. If known behavior is causing false positives, it can be exempted from the rule." ], "from": "now-60m", "rule_id": "227dc608-e558-43d9-b521-150772250bae", "max_signals": 100, "risk_score_mapping": [], "severity_mapping": [], "threat": [ { "framework": "MITRE ATT&CK", "technique": [ { "reference": "https://attack.mitre.org/techniques/T1070/", "name": "Indicator Removal on Host", "id": "T1070" } ], "tactic": { "reference": "https://attack.mitre.org/tactics/TA0005/", "name": "Defense Evasion", "id": "TA0005" } } ], "to": "now", "references": [ "https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketPolicy.html", "https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketReplication.html", "https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketCors.html", "https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketEncryption.html", "https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketLifecycle.html" ], "version": 9, "exceptions_list": [], "immutable": true, "related_integrations": [], "required_fields": [], "setup": "", "type": "query", "language": "kuery", "index": [ "filebeat-*", "logs-aws*" ], "query": "event.dataset:aws.cloudtrail and event.provider:s3.amazonaws.com and\n event.action:(DeleteBucketPolicy or DeleteBucketReplication or DeleteBucketCors or\n DeleteBucketEncryption or DeleteBucketLifecycle)\n and event.outcome:success\n", "actions": [], "execution_summary": { "last_execution": { "date": "2024-02-26T04:53:11.488Z", "status": "failed", "status_order": 30, "message": "authenticationStart is not registered!", "metrics": {} } } }, "target_rule": { "name": "AWS S3 Bucket Configuration Deletion", "description": "Identifies the deletion of various Amazon Simple Storage Service (S3) bucket configuration components.", "risk_score": 21, "severity": "low", "timestamp_override": "event.ingested", "license": "Elastic License v2", "note": "", "version": 206, "tags": [ "Domain: Cloud", "Data Source: AWS", "Data Source: Amazon Web Services", "Use Case: Asset Visibility", "Tactic: Defense Evasion" ], "enabled": false, "risk_score_mapping": [], "severity_mapping": [], "interval": "10m", "from": "now-60m", "to": "now", "actions": [], "exceptions_list": [], "author": [ "Elastic" ], "false_positives": [ "Bucket components may be deleted by a system or network administrator. Verify whether the user identity, user agent, and/or hostname should be making changes in your environment. Bucket component deletions by unfamiliar users or hosts should be investigated. If known behavior is causing false positives, it can be exempted from the rule." ], "references": [ "https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketPolicy.html", "https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketReplication.html", "https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketCors.html", "https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketEncryption.html", "https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketLifecycle.html" ], "max_signals": 100, "threat": [ { "framework": "MITRE ATT&CK", "tactic": { "id": "TA0005", "name": "Defense Evasion", "reference": "https://attack.mitre.org/tactics/TA0005/" }, "technique": [ { "id": "T1070", "name": "Indicator Removal", "reference": "https://attack.mitre.org/techniques/T1070/" } ] } ], "id": "76501ef6-d32f-11ee-a96b-438c4c44df86", "rule_id": "227dc608-e558-43d9-b521-150772250bae", "immutable": true, "updated_at": "2024-02-29T12:32:31.821Z", "updated_by": "elastic", "created_at": "2024-02-24T16:12:11.182Z", "created_by": "elastic", "revision": 10, "related_integrations": [ { "package": "aws", "version": "^2.0.0", "integration": "cloudtrail" } ], "required_fields": [ { "name": "event.action", "type": "keyword", "ecs": true }, { "name": "event.dataset", "type": "keyword", "ecs": true }, { "name": "event.outcome", "type": "keyword", "ecs": true }, { "name": "event.provider", "type": "keyword", "ecs": true } ], "setup": "The AWS Fleet integration, Filebeat module, or similarly structured data is required to be compatible with this rule.", "type": "query", "index": [ "filebeat-*", "logs-aws*" ], "query": "event.dataset:aws.cloudtrail and event.provider:s3.amazonaws.com and\n event.action:(DeleteBucketPolicy or DeleteBucketReplication or DeleteBucketCors or\n DeleteBucketEncryption or DeleteBucketLifecycle)\n and event.outcome:success\n", "language": "kuery" }, "diff": { "fields": { "version": { "current_version": 9, "target_version": 206, "merged_version": 206, "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "tags": { "current_version": [ "Elastic", "Cloud", "AWS", "Continuous Monitoring", "SecOps", "Asset Visibility", "auto_disabled_8.0" ], "target_version": [ "Domain: Cloud", "Data Source: AWS", "Data Source: Amazon Web Services", "Use Case: Asset Visibility", "Tactic: Defense Evasion" ], "merged_version": [ "Domain: Cloud", "Data Source: AWS", "Data Source: Amazon Web Services", "Use Case: Asset Visibility", "Tactic: Defense Evasion" ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "threat": { "current_version": [ { "framework": "MITRE ATT&CK", "technique": [ { "reference": "https://attack.mitre.org/techniques/T1070/", "name": "Indicator Removal on Host", "id": "T1070" } ], "tactic": { "reference": "https://attack.mitre.org/tactics/TA0005/", "name": "Defense Evasion", "id": "TA0005" } } ], "target_version": [ { "framework": "MITRE ATT&CK", "tactic": { "id": "TA0005", "name": "Defense Evasion", "reference": "https://attack.mitre.org/tactics/TA0005/" }, "technique": [ { "id": "T1070", "name": "Indicator Removal", "reference": "https://attack.mitre.org/techniques/T1070/" } ] } ], "merged_version": [ { "framework": "MITRE ATT&CK", "tactic": { "id": "TA0005", "name": "Defense Evasion", "reference": "https://attack.mitre.org/tactics/TA0005/" }, "technique": [ { "id": "T1070", "name": "Indicator Removal", "reference": "https://attack.mitre.org/techniques/T1070/" } ] } ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "note": { "current_version": "## Setup\n\nThe AWS Fleet integration, Filebeat module, or similarly structured data is required to be compatible with this rule.", "target_version": "", "merged_version": "", "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "setup": { "current_version": "", "target_version": "The AWS Fleet integration, Filebeat module, or similarly structured data is required to be compatible with this rule.", "merged_version": "The AWS Fleet integration, Filebeat module, or similarly structured data is required to be compatible with this rule.", "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "related_integrations": { "current_version": [], "target_version": [ { "package": "aws", "version": "^2.0.0", "integration": "cloudtrail" } ], "merged_version": [ { "package": "aws", "version": "^2.0.0", "integration": "cloudtrail" } ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false }, "required_fields": { "current_version": [], "target_version": [ { "name": "event.action", "type": "keyword", "ecs": true }, { "name": "event.dataset", "type": "keyword", "ecs": true }, { "name": "event.outcome", "type": "keyword", "ecs": true }, { "name": "event.provider", "type": "keyword", "ecs": true } ], "merged_version": [ { "name": "event.action", "type": "keyword", "ecs": true }, { "name": "event.dataset", "type": "keyword", "ecs": true }, { "name": "event.outcome", "type": "keyword", "ecs": true }, { "name": "event.provider", "type": "keyword", "ecs": true } ], "diff_outcome": "BASE=A, CURRENT=A, TARGET=B", "merge_outcome": "TARGET", "has_update": true, "has_conflict": false } }, "has_conflict": false } } ] } ```

When you try to upgrade the "Potential DLL Side-Loading via Microsoft Antimalware Service Executable" rule, you can see a few issues:

First, the upgrade fails and is shown in the toast as a "green", successful result, although it says "failed":

Second, this is an error that the /internal/detection_engine/prebuilt_rules/upgrade/_perform endpoint returns when you click "Update". Something is wrong with the lastRun object which is an internal rule field used by the Alerting Framework.

{
    "summary": {
        "total": 1,
        "skipped": 0,
        "succeeded": 0,
        "failed": 1
    },
    "results": {
        "updated": [],
        "skipped": []
    },
    "errors": [
        {
            "message": "[attributes.lastRun]: types that failed validation:\n- [attributes.lastRun.0.outcomeMsg]: types that failed validation:\n - [attributes.lastRun.outcomeMsg.0]: could not parse array value from json input\n - [attributes.lastRun.outcomeMsg.1]: expected value to equal [null]\n- [attributes.lastRun.1]: expected value to equal [null]: Bad Request",
            "rules": [
                {
                    "rule_id": "053a0387-f3b5-4ba5-8245-8002cca2bd08",
                    "name": "Potential DLL Side-Loading via Microsoft Antimalware Service Executable"
                }
            ]
        }
    ]
}

Third, we incorrectly pass the rule version and revision in the request body to this endpoint. We pass this:

{"mode":"SPECIFIC_RULES","rules":[{"rule_id":"053a0387-f3b5-4ba5-8245-8002cca2bd08","version":108,"revision":6}],"pick_version":"TARGET"}

but the versions and revisions of the current and target rule versions are these:

Current:
revision: 6
version: 6

Target:
revision: 7
version: 108

I think we should be passing this instead:

{"mode":"SPECIFIC_RULES","rules":[{"rule_id":"053a0387-f3b5-4ba5-8245-8002cca2bd08","version":6,"revision":6}],"pick_version":"TARGET"}

The same happens with the other 3 rules.

[jpdjere]

Just for the record @banderror:

I think we should be passing this instead: {"mode":"SPECIFIC_RULES","rules":[{"rule_id":"053a0387-f3b5-4ba5-8245-8002cca2bd08","version":6,"revision":6}],"pick_version":"TARGET"}

The check for the revision in the handler checks that what's passed in equals the current revision.

The version that should be passed is the next version.

So the payload looks correct to me.

[jpdjere]

@vgomez-el @banderror @karanbirsingh-qasource and @elastic/response-ops team

I'm investigating the issue and have reached some conclusions:

• I have tried multiple times reproducing the error both in Cloud and locally, updating the stack from 7.18.17 to 8.13 BC 2. I could not reproduce it.
• I logged-in in to the affected instance with the credentials provided in the Slack thread and see that the error still persists for those affected rules.
• Navigating to the Rules Detail page of these 4 affected rules you can see that the rule execution has failed with an authenticationStart is not registered! error message.
• Checking the SO for those rules, noticed that that error message is saved to the lastRun.outcomeMsg property as a string:

  "lastRun": {
  "outcome": "failed",
  "outcomeMsg": "authenticationStart is not registered!",
  "warning": "unknown",
  "alertsCount": {},
  "outcomeOrder": 20
  },

  However, lastRun.outcomeMsg is of type string[] | null.

• The authenticationStart is not registered! error message is thrown during Kibana startup, due to some failure in authentication. This apparently happened, in this specific case, during the Kibana upgrade, and cannot be reproduced (at least not easily) in similar upgrade processes.
• The error message in the plugin.ts file of the Alerting Framework.. This error bubbles up from the execution_log and ends up written in the lastRun.outcomeMsg.
• It looks like the origin of the error of wrong type (a string) being written into lastRun.outcomeMsg comes out from a decision taken when the Rule Result Service was created. The migration does not apply the change from string to string[]; instead, that type transformation takes place in the endpoint: https://github.com/elastic/kibana/blob/main/x-pack/plugins/alerting/server/task_runner/fixtures.ts#L118
• Therefore, my guess of what went on in this case is: since the error was thrown during the Kibana start-up process, the authentication error was written directly as a string to lastRun.outcomeMsg instead of being converted to string[], since this only happens during API requests.

Therefore:

• I wouldn't consider this as a blocker for the next release. It's not a regression, but a very specific edge case during upgrades, when an authentication error happens. Restarting Kibana or deleting and reinstalling the rules will probably fix the issue.
• I'd take some time to figure out how to fix this issue, and prevent lastRun.outcomeMsg to be written with the wrong type, reaching out to the Response Ops team.

TL;DR for ResponseOps team:

• During upgrade of Kibana, some error in authentication is produced which ends up written in the rule's lastRun.outcomeMsg property as a string, although the property is acutally an array of strings. This causes rule validation to fail; breaking the affected rule's execution and management capabilities like upgrade (which is what this ticket reported).
• When the Rule Result Service was created (I wrote the PR) we decided that the legacy type of lastRun.outcomeMsg -which was a string- would not be migrated using a normal migration to its new type of string[]; instead, that type transformation takes place in the endpoint: https://github.com/elastic/kibana/blob/main/x-pack/plugins/alerting/server/task_runner/fixtures.ts#L118
• However, that causes that when the error is thrown during the Kibana start-up process, the authentication error is written directly as a string to lastRun.outcomeMsg instead of being converted to string[], since this only happens during API requests.

I would like to know how we can move forward to fixing this issue.

1. Does it make sense to create a new migration now, to fix that migration of the lastRun.outcomeMsg that we didn't do back then?
2. Is there any other way we can catch error produced during start-up and prevent them from being written as strings directly to lastRun.outcomeMsg?

[pmuellr]

My guess is that the string (vs string[]) version of the field is being populated here, but just a quick guess based on searching the code and finding this in migrations ...

https://github.com/elastic/kibana/blob/b93081823acce53cd22f66c7d6f6a25db5ad4db0/x-pack/plugins/alerting/server/saved_objects/migrations/8.6/index.ts#L18-L43

Seems like we can just fix that, right?

Oh, that code is already in a migration. So, we'd need another migration to fix it?

[jpdjere]

Hi @pmuellr thanks for taking a look at this.

Yes, doing another migration to fix this was my initial thought, but my understanding was that the migration mechanism that ResponseOps maintained was deprecated. Indeed, I see that the last migration file that was created was for 8.8.

How do you currently carry out migrations? Is there a replacing mechanism that I can use to fix this issue?

[pmuellr]

Our migration story got complicated with serverless, but I think it was mainly about additions / removals of fields, vs just wacking some data, like what I think we need to do here. Also, I'm not sure we've done a migration since 8.8, since we were asked to not make migration changes while they were working on changes to encrypted saved objects.

Let me ask the team about this ...

[banderror]

Since we don't have a working migration mechanism for rules that could be used for fixing the broken data that causes this bug, but there's a simple workaround:

• if it's a prebuilt rule, user can delete and reinstall it
• if it's a custom rule, user can duplicate it and delete the original

we will postpone fixing this bug until after 8.17 to be able to ship https://github.com/elastic/kibana/issues/174168 earlier.

cc @jpdjere @approksiu