[Meta] Prepare 20 Linux ES|QL Hunts

[Aegrah]

Meta Summary

The goal of this meta is to create ~20 Linux ES|QL hunts.

Estimated Time to Complete

1 sprint - 2 weeks

Tasklist

### Meta Tasks
- [ ] Provide Week 1 Update Comment
- [ ] Provide Week 2 Update or Closeout Comment

Resources / References

https://github.com/elastic/ia-trade-team/issues/302

[Aegrah]

Initial ideas pastebin:

• Capitalized process executable execution
• Unusual amount of data exfiltration (should check whether we can do something for UDP here as well)
• Persistence via any of the common persistence methods;
  • Systemd (timer), init.d, rc.local, motd, cron, bashrc, LKM, LD_preload, etc.
• Execution from suspicious location (/tmp, /dev/shm/, /var/run/, etc.)
• Execution from different web roots or by different default web users (www-data, etc.)
• Base64 executions (or other evasive executions)
• Suspicious file downloads from an IP instead of a domain (local & public?)
• Long-lasting connections from suspicious utilities (capable of creating rev shells) (https://github.com/elastic/ia-trade-team/issues/302#issuecomment-1960048584)
• Unusual command proxying for defense evasion (identify unusual process parents for uncommon processes?) (https://github.com/elastic/ia-trade-team/issues/302)
• Network Discovery via sensitive ports by unusual process (https://github.com/elastic/ia-trade-team/issues/302#issuecomment-1957216751)
• Unusual SSH root connections by host
• Unusual execution of GTFObins
• Bufferoverflow attack based on X amount of segfaults --> check whether ES|QL can query message field correctly.
• ...

[Aegrah]

Uncommon process execution from suspicious directory

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 7 day
| WHERE host.os.type == "linux" and event.type == "start" and event.action == "exec" and (
// Add paths to monitor from your environment here
  (process.executable like "/dev/shm/*") or
  (process.executable like "/var/www/*") or
  (process.executable like "/boot/*") or
  (process.executable like "/srv/*") or
  ((process.executable rlike "/tmp/[^/]+" or process.executable rlike "/var/tmp/[^/]+")) or
  (process.executable rlike "/run/[^/]+") or
  (process.executable rlike "/var/run/[^/]+")
) and not (
  // Exclude noisy (parent) processes, users or directories from your environment here
  (process.parent.executable in ("/usr/sbin/dpkg-preconfigure")) or
  // Exclude /tmp and /var/tmp instances starting or ending with digits (usually benign files)
  (process.executable rlike "/tmp/[0-9].*" or process.executable rlike "/tmp/.*[0-9]/?") or
  (process.executable rlike "/var/tmp/[0-9].*" or process.executable rlike "/var/tmp/.*[0-9]/?")
)
| STATS process_count = COUNT(process.executable), parent_process_count = COUNT(process.parent.executable), host_count = COUNT(host.name) by process.executable, process.parent.executable, host.name, user.id
// Alter this threshold to make sense for your environment 
| WHERE (process_count <= 3 or parent_process_count <= 3) and host_count <= 3
| SORT process_count asc
| LIMIT 100

Notes:

• Excluded /tmp, /var/tmp, /run, /var/run subdirectories to exclude noise
• Excluded /tmp, /var/tmp files starting or ending with digits to exclude real temporary files
• Include a process or parent process count of <= 3, and a host count of <= 3 to eliminate common processes across different hosts.
• Screenshot was taken from last 365 days of detonate data, only TP malware data is visible in the screenshot. Telemetry was analyzed as well, only 7 hits last 75 days; which are FPs that are easily tuned in customer environments, but should not be tuned for a general threat hunting query.

[Aegrah]

Hidden process execution

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 180 day
| WHERE host.os.type == "linux" and event.type == "start" and event.action == "exec" and (
  (process.executable rlike "/[^/]+/\\.[^/]+")
) 
| STATS process_count = COUNT(process.executable), parent_process_count = COUNT(process.parent.executable), host_count = COUNT(host.name) by process.executable, process.parent.executable, host.name, user.id
// Alter this threshold to make sense for your environment 
| WHERE (process_count <= 3 or parent_process_count <= 3) and host_count <= 3
| SORT process_count asc
| LIMIT 100

• Included only hidden files, excluded hidden directories, as this is common in Unix
• Include a process or parent process count of <= 3, and a host count of <= 3 to eliminate common processes across different hosts.
• Only TPs in testing stack, 0 FPs in telemetry last 75 days and detonate last 365 days.

[Aegrah]

Potential defense evasion via multi-dot process execution

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 7 day
| WHERE host.os.type == "linux" and event.type == "start" and event.action == "exec" and process.executable rlike """.*\.{3,}.*"""
| STATS process_count = COUNT(process.executable), host_count = COUNT(host.name) by process.executable
// Alter this threshold to make sense for your environment 
| WHERE process_count <= 10
| SORT process_count asc
| LIMIT 100

• This one probably does not make sense for ES|QL hunt --> might be an interesting detection rule; probably not noisy and therefore doesn't specifically need a process_count to be valuable. Will leave it here for reference.

[Aegrah]

Defense evasion via capitalized process execution

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 10 day
| WHERE host.os.type == "linux" and event.type == "start" and event.action == "exec" and (
  (process.name rlike """[A-Z]{2,}[a-z]{1,}[0-9]{0,}""") or
  (process.name rlike """[A-Z]{1,}[0-9]{0,}""")
)
| STATS process_count = COUNT(process.name), host_count = COUNT(host.name) by process.name
// Alter this threshold to make sense for your environment 
| WHERE process_count <= 3 and host_count <= 3
| LIMIT 100

• Detects processes that have 2 or more capital letters within its process name, and 0 or more digits. Using 2 or more capital letters will exclude most common processes starting with a capital letter
• Will detect defense evasion techniques where binaries are capitalized to evade detections.
• Many default payloads contain random capitalization, such as Metasploit payloads (as seen in the screenshot).
• Include a process count of <= 3, and a host count of <= 3 to eliminate common processes across different hosts.

[Aegrah]

Unusual process command lines for web server user

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.type == "start" and user.name in ("www-data", "apache", "nginx", "httpd", "tomcat", "lighttpd", "glassfish", "weblogic")
| STATS process_cli_count = COUNT(process.command_line), host_count = COUNT(host.name) by process.command_line, process.name, user.name, host.name
| WHERE process_cli_count <= 3 and host_count <= 2
| SORT process_cli_count asc
| LIMIT 100

• Detects process command executions through a commonly used web server user account.
• This was never doable in EQL without creating noise, as many web servers execute commands by default, and without explicitly leveraging whitelisting or a huge blacklist for every single web server technology, we would either be overly or underly exclusive. Leveraging process command line counting in conjunction with host counting, we can easily minimize the number of FPs caused by common web server command executions, while still capturing true TPs.
• Some FPs remain, however, these are easily mitigated on an on-environment basis.

[Aegrah]

Unusual file creations by web server user

FROM logs-endpoint.events.file-*
| WHERE @timestamp > NOW() - 50 day
| WHERE host.os.type == "linux" and event.type == "creation" and user.name in ("www-data", "apache", "nginx", "httpd", "tomcat", "lighttpd", "glassfish", "weblogic") and (
  file.path like "/var/www/*" or
  file.path like "/var/tmp/*" or
  file.path like "/tmp/*" or
  file.path like "/dev/shm/*"
)
| STATS file_count = COUNT(file.path), host_count = COUNT(host.name) by file.path, host.name, process.name, user.name
// Alter this threshold to make sense for your environment 
| WHERE file_count <= 5
| SORT file_count asc
| LIMIT 100

• Might not be a useful ES|QL query. Might be good as a detection rule, or as an endpoint rule if made more specific. Will keep it here as a reference.

[Aegrah]

Unusual file downloads from ....

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 7 day
| WHERE host.os.type == "linux" and event.type == "start" and process.name in ("curl", "wget") and process.command_line rlike """.*[0-9]{1,3}(\.[0-9]{1,3}){3}.*"""
| STATS process_cli_count = COUNT(process.command_line), host_count = COUNT(host.name) by process.command_line, process.executable, host.name
| WHERE process_cli_count <= 10 and host_count <= 5
| SORT process_cli_count asc
| LIMIT 100

[Aegrah]

Segmentation Fault & Potential Buffer Overflow Hunting

FROM logs-system.syslog*
| WHERE @timestamp > NOW() - 12 hour
| WHERE host.os.type == "linux" and process.name == "kernel" and message like "*segfault*"
| GROK message "\\[%{NUMBER:timestamp}\\] %{WORD:process}\\[%{NUMBER:pid}\\]: segfault at %{BASE16NUM:segfault_address} ip %{BASE16NUM:instruction_pointer} sp %{BASE16NUM:stack_pointer} error %{NUMBER:error_code} in %{DATA:so_file}\\[%{BASE16NUM:so_base_address}\\+%{BASE16NUM:so_offset}\\]"
| KEEP timestamp, process, pid, so_file, segfault_address, instruction_pointer, stack_pointer, error_code, so_base_address, so_offset

• Detects segfaults, parses the syslog related to the segfault, and shows which process segfaulted in conjunction with which shared object file crashed and some additional information regarding the crash.

FROM logs-system.syslog*
| WHERE host.os.type == "linux" and process.name == "kernel" and message like "*segfault*"
| WHERE @timestamp > NOW() - 12 hour
| GROK message "\\[%{DATA:timestamp}\\] %{WORD:process}\\[%{NUMBER:pid}\\]: segfault at %{BASE16NUM:segfault_address} ip %{BASE16NUM:instruction_pointer} sp %{BASE16NUM:stack_pointer} error %{NUMBER:error_code} in %{DATA:so_name}\\[%{BASE16NUM:so_base_address}\\+%{BASE16NUM:so_offset}\\] likely on CPU %{NUMBER:cpu} \\(core %{NUMBER:core}, socket %{NUMBER:socket}\\)"
| EVAL timestamp = REPLACE(timestamp, "\\s+", "")
| KEEP timestamp, process, pid, segfault_address, instruction_pointer, stack_pointer, error_code, so_name, so_base_address, so_offset, cpu, core, socket
| STATS process_count = COUNT(process), so_count = COUNT(so_name) by process, so_name
// Alter this threshold to make sense for your environment 
| WHERE process_count > 100
| LIMIT 10

• Syslog doesn't log all the same every time, sometimes it adds prepending spaces. We can use EVAL to remove those
• Using GROK and STATS, we can count occurences of segfaults within a plain text message field, potentially detecting buffer overflow attacks
• We can detect unsuccessful process injection attempts

[Aegrah]

Persistence via Cron

File

FROM logs-endpoint.events.file-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.type in ("creation", "change") and (
    file.path in ("/etc/cron.allow", "/etc/cron.deny", "/etc/crontab") or
    file.path like "/etc/cron.*/*" or
    file.path like "/var/spool/cron/crontabs/*" or 
    file.path like "/var/spool/anacron/*" or 
    file.path like "/var/spool/cron/atjobs/*" or
    file.path like "/var/spool/fcron/*" or
    file.path like "/home/*/.tsp/*"
) and not (
    process.name in ("dpkg", "dockerd", "yum", "dnf", "snapd", "pacman", "pamac-daemon", "anacron") or
    file.extension in ("dpkg-remove", "swx", "swp") or
    file.name like "tmp.*"
)
| EVAL persistence = CASE(
    file.path in ("/etc/cron.allow", "/etc/cron.deny", "/etc/crontab") or
    file.path like "/etc/cron.*/*" or
    file.path like "/var/spool/cron/crontabs/*" or 
    file.path like "/var/spool/anacron/*" or 
    file.path like "/var/spool/cron/atjobs/*" or
    file.path like "/var/spool/fcron/*" or
    file.path like "/home/*/.tsp/*",
    process.name,
    null
)
| STATS cc = COUNT(*), pers_count = COUNT(persistence), agent_count = COUNT(agent.id) by process.executable, file.path, host.name, user.name
| WHERE pers_count > 0 and pers_count <= 20 and agent_count <= 3
| SORT cc asc
| LIMIT 100

Process

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.action == “exec” and event.type == "start" and process.parent.name in ("cron", "fcron", "atd")
| STATS process_cli_count = COUNT(process.command_line), host_count = COUNT_DISTINCT(host.name) by process.command_line, process.executable, process.parent.executable
| WHERE host_count <= 3
| SORT process_cli_count asc
| LIMIT 100

OSQuery

File

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
WHERE 
    f.path IN ("/etc/cron.allow", "/etc/cron.deny", "/etc/crontab")
    OR f.path LIKE "/etc/cron.%/*"
    OR f.path LIKE "/var/spool/cron/crontabs/%"
    OR f.path LIKE "/var/spool/anacron/%"
    OR f.path LIKE "/var/spool/cron/atjobs/%"
    OR f.path LIKE "/var/spool/fcron/%"
    OR f.path LIKE "/home/%/.tsp/%"
    OR f.path LIKE "/etc/cron.allow.d/%" 
    OR f.path LIKE "/etc/cron.d/%" 
    OR f.path LIKE "/etc/cron.hourly/%" 
    OR f.path LIKE "/etc/cron.daily/%" 
    OR f.path LIKE "/etc/cron.weekly/%" 
    OR f.path LIKE "/etc/cron.monthly/%"

File

SELECT * FROM crontab

[Aegrah]

Persistence via Systemd (timers)

FROM logs-endpoint.events.file-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.type in ("creation", "change") and (

    // System-wide/user-specific services/timers (root permissions required)
    file.path like "/run/systemd/system/*" or
    file.path like "/etc/systemd/system/*" or
    file.path like "/etc/systemd/user/*" or
    file.path like "/usr/local/lib/systemd/system/*" or
    file.path like "/lib/systemd/system/*" or
    file.path like "/usr/lib/systemd/system/*" or
    file.path like "/usr/lib/systemd/user/*" or

    // user-specific services/timers (user permissions required)
    file.path like "/home/*/.config/systemd/user/*" or
    file.path like "/home/*/.local/share/systemd/user/*" or

    // System-wide generators (root permissions required)
    file.path like "/etc/systemd/system-generators/*" or
    file.path like "/usr/local/lib/systemd/system-generators/*" or
    file.path like "/lib/systemd/system-generators/*" or
    file.path like "/etc/systemd/user-generators/*" or
    file.path like "/usr/local/lib/systemd/user-generators/*" or
    file.path like "/usr/lib/systemd/user-generators/*"

) and not (
    process.name in (
      "dpkg", "dockerd", "yum", "dnf", "snapd", "pacman", "pamac-daemon",
      "netplan", "systemd", "generate"
    ) or
    process.executable == "/proc/self/exe" or
    process.executable like "/dev/fd/*" or
    file.extension in ("dpkg-remove", "swx", "swp")
)
| EVAL persistence = CASE(

    // System-wide/user-specific services/timers (root permissions required)
    file.path like "/run/systemd/system/*" or
    file.path like "/etc/systemd/system/*" or
    file.path like "/etc/systemd/user/*" or
    file.path like "/usr/local/lib/systemd/system/*" or
    file.path like "/lib/systemd/system/*" or
    file.path like "/usr/lib/systemd/system/*" or
    file.path like "/usr/lib/systemd/user/*" or

    // user-specific services/timers (user permissions required)
    file.path like "/home/*/.config/systemd/user/*" or
    file.path like "/home/*/.local/share/systemd/user/*" or

    // System-wide generators (root permissions required)
    file.path like "/etc/systemd/system-generators/*" or
    file.path like "/usr/local/lib/systemd/system-generators/*" or
    file.path like "/lib/systemd/system-generators/*" or
    file.path like "/etc/systemd/user-generators/*" or
    file.path like "/usr/local/lib/systemd/user-generators/*" or
    file.path like "/usr/lib/systemd/user-generators/*",
    process.name,
    null
)
| STATS cc = COUNT(*), pers_count = COUNT(persistence), agent_count = COUNT(agent.id) by process.executable, file.path, host.name, user.name
| WHERE pers_count > 0 and pers_count <= 20 and agent_count <= 3
| SORT cc asc
| LIMIT 100

OSQuery Service entries

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
WHERE 
    (f.path LIKE "/run/systemd/system/%"
    OR f.path LIKE "/etc/systemd/system/%"
    OR f.path LIKE "/etc/systemd/user/%"
    OR f.path LIKE "/usr/local/lib/systemd/system/%"
    OR f.path LIKE "/lib/systemd/system/%"
    OR f.path LIKE "/usr/lib/systemd/system/%"
    OR f.path LIKE "/usr/lib/systemd/user/%"
    OR f.path LIKE "/home/%/.config/systemd/user/%"
    OR f.path LIKE "/home/%/.local/share/systemd/user/%")
    AND f.filename LIKE "%.service"

OSQuery Timer entries

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes,
    h.md5 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
LEFT JOIN 
    hash h ON f.path = h.path 
WHERE 
    f.directory IN (
        '/run/systemd/system',
        '/etc/systemd/system',
        '/etc/systemd/user',
        '/usr/local/lib/systemd/system',
        '/lib/systemd/system',
        '/usr/lib/systemd/system',
        '/usr/lib/systemd/user',
        '/home/.config/systemd/user',
        '/home/.local/share/systemd/user'
    )
    AND f.filename LIKE "%.timer"
ORDER BY 
    f.mtime DESC;

OSQuery generator entries

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes,
    h.md5 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
LEFT JOIN 
    hash h ON f.path = h.path 
WHERE 
    f.directory IN (
        '/etc/systemd/system-generators/',
        '/usr/local/lib/systemd/system-generators/',
        '/lib/systemd/system-generators/',
        '/etc/systemd/user-generators/',
        '/usr/local/lib/systemd/user-generators/',
        '/usr/lib/systemd/user-generators/'
    )
ORDER BY 
    f.mtime DESC;

OSQuery Active Systemd Services/Timers/Generators

SELECT name, path, source, status, type FROM startup_items
WHERE type == "systemd unit" AND status == "active" AND
name LIKE "%.service" OR name LIKE  "%.timer"

[Aegrah]

Persistence via message-of-the-day

FROM logs-endpoint.events.file-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.type in ("creation", "change") and file.path like "/etc/update-motd.d/*" and
not (
    process.name in ("dpkg", "dockerd", "yum", "dnf", "snapd", "pacman")
)
| EVAL persistence = CASE(file.path like "/etc/update-motd.d/*", process.name, null)
| STATS cc = COUNT(*), pers_count = COUNT(persistence), agent_count = COUNT(agent.id) by process.executable, file.path, host.name, user.name
| WHERE pers_count > 0 and pers_count <= 20 and agent_count <= 5
| SORT cc asc
| LIMIT 100

Process

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.action == "exec" and event.type == "start" and process.parent.executable like "/etc/update-motd.d/*" and
not process.args like "/tmp/tmp.*"
| STATS process_cli_count = COUNT(process.command_line), host_count = COUNT_DISTINCT(host.name) by process.command_line, process.executable, process.parent.executable
| WHERE host_count <= 5
| SORT process_cli_count asc
| LIMIT 100

OSQuery File Hunt

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes,
    h.md5 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
LEFT JOIN 
    hash h ON f.path = h.path 
WHERE 
    f.directory IN ('/etc/update-motd.d/')
ORDER BY 
    f.mtime DESC;

[Aegrah]

Persistence via rc.local

FROM logs-endpoint.events.file-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.type in ("creation", "change") and (file.path == "/etc/rc.local" or file.path == "/etc/rc.common")
| EVAL persistence = CASE(file.path == "/etc/rc.local" or file.path == "/etc/rc.common", process.name, null)
| STATS cc = COUNT(*), rc_pers_count = COUNT(persistence), agent_count = COUNT(agent.id) by process.executable
| WHERE rc_pers_count > 0 and rc_pers_count <= 3 and agent_count <= 3
| SORT cc asc
| LIMIT 100

Syslog hunting

FROM logs-system.syslog-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and process.name in ("rc.local", "rc.common")
| STATS message_count = COUNT(message), host_count = COUNT_DISTINCT(host.name) by message
| WHERE host_count <= 3 AND message_count < 10
| SORT message_count asc
| LIMIT 100

OSQuery systemd_unit state

SELECT * FROM systemd_units  WHERE id = "rc-local.service"

OSQuery Startup item

SELECT * FROM startup_items WHERE name = "rc-local.service"

OSQuery File information

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
WHERE 
    f.path in ('/etc/rc.local', '/etc/rc.common')

[Aegrah]

Persistence via init.d

FROM logs-endpoint.events.file-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.type in ("creation", "change") and file.path like "/etc/init.d/*" and
not (
    process.name in ("dpkg", "dockerd", "yum", "dnf", "snapd", "pacman")
)
| EVAL persistence = CASE(file.path like "/etc/init.d/*", process.name, null)
| STATS cc = COUNT(*), pers_count = COUNT(persistence), agent_count = COUNT(agent.id) by process.executable, file.path, host.name, user.name
| WHERE pers_count > 0 and pers_count <= 20 and agent_count <= 3
| SORT cc asc
| LIMIT 100

Process

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.action == "exec" and event.type == "start" and process.parent.executable like "/etc/init.d/*"
| STATS process_cli_count = COUNT(process.command_line), host_count = COUNT_DISTINCT(host.name) by process.command_line, process.executable, process.parent.executable
| WHERE host_count <= 3
| SORT process_cli_count asc
| LIMIT 100

OSQuery Services started from init.d source

SELECT name, path, source, status, type FROM startup_items
WHERE type == "systemd unit" AND status == "active" AND
source LIKE "/etc/init.d/%"

OSQuery Files

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes,
    h.md5 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
LEFT JOIN 
    hash h ON f.path = h.path 
WHERE 
    f.directory IN ('/etc/init.d/')
ORDER BY 
    f.mtime DESC;

[Aegrah]

Drivers load with low occurrence frequency

FROM logs-auditd_manager.auditd-*, logs-auditd.log-*, auditbeat-*
| WHERE @timestamp > NOW() - 365 day
| WHERE host.os.type == "linux" and event.category == "driver" and event.action == "loaded-kernel-module" and auditd.data.syscall in ("init_module", "finit_module")
| STATS host_count = COUNT_DISTINCT(host.id), total_count = COUNT(*), ko_count = COUNT_DISTINCT(auditd.data.name) by auditd.data.name, process.executable, process.name
| WHERE host_count == 1 and total_count == 1 and ko_count == 1
| LIMIT 100 
| SORT auditd.data.name asc

• This event will soon be added to Elastic Defend as well, allowing me to rewrite this to enhance usage.

[Aegrah]

Network connections with low occurence frequency for unique agent.id

FROM logs-endpoint.events.network-*
| WHERE @timestamp > NOW() - 7 day
| WHERE host.os.type == "linux" and event.type == "start" and event.action in ("connection_attempted", "connection_accepted") and destination.ip IS NOT null and not CIDR_MATCH(destination.ip, "10.0.0.0/8", "127.0.0.0/8", "169.254.0.0/16", "172.16.0.0/12", "192.0.0.0/24", "192.0.0.0/29", "192.0.0.8/32", "192.0.0.9/32", "192.0.0.10/32", "192.0.0.170/32", "192.0.0.171/32", "192.0.2.0/24", "192.31.196.0/24", "192.52.193.0/24", "192.168.0.0/16", "192.88.99.0/24", "224.0.0.0/4", "100.64.0.0/10", "192.175.48.0/24","198.18.0.0/15", "198.51.100.0/24", "203.0.113.0/24", "224.0.0.0/4", "240.0.0.0/4", "::1","FE80::/10", "FF00::/8")
| STATS process_count = COUNT(process.name), agent_count = COUNT_DISTINCT(agent.id) by process.name
// Alter this threshold to make sense for your environment 
| WHERE agent_count == 1 and process_count > 0 and process_count <= 3
| LIMIT 100 
| SORT process_count asc

Taking into account GTFOBins

FROM logs-endpoint.events.network-*
| WHERE @timestamp > NOW() - 7 day
| WHERE host.os.type == "linux" and event.type == "start" and event.action in ("connection_attempted", "connection_accepted") and (
    // Add additional LoLbins here
    (process.name in ("bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish", "socat", "java", "awk", "gawk", "mawk", "nawk", "openssl", "nc", "ncat", "netcat", "telnet")) or
    (process.name like "python*") or
    (process.name like "perl*") or
    (process.name like "ruby*") or
    (process.name like "lua*") or
    (process.name like "php*")
) and
destination.ip IS NOT null and not CIDR_MATCH(destination.ip, "10.0.0.0/8", "127.0.0.0/8", "169.254.0.0/16", "172.16.0.0/12", "192.0.0.0/24", "192.0.0.0/29", "192.0.0.8/32", "192.0.0.9/32", "192.0.0.10/32", "192.0.0.170/32", "192.0.0.171/32", "192.0.2.0/24", "192.31.196.0/24", "192.52.193.0/24", "192.168.0.0/16", "192.88.99.0/24", "224.0.0.0/4", "100.64.0.0/10", "192.175.48.0/24","198.18.0.0/15", "198.51.100.0/24", "203.0.113.0/24", "224.0.0.0/4", "240.0.0.0/4", "::1","FE80::/10", "FF00::/8")
| STATS process_count = COUNT(process.name), agent_count = COUNT_DISTINCT(agent.id) by process.name
// Alter this threshold to make sense for your environment 
| WHERE agent_count <= 3 and process_count > 0 and process_count <= 5
| LIMIT 100 
| SORT process_count asc

Taking into account suspicious directories

FROM logs-endpoint.events.network-*
| WHERE @timestamp > NOW() - 90 day
| WHERE host.os.type == "linux" and event.type == "start" and event.action in ("connection_attempted", "connection_accepted") and (
    (process.executable like "./") or
    (process.executable like "/dev/shm/*") or
    (process.executable like "/var/www/*") or
    (process.executable like "/boot/*") or
    (process.executable like "/srv/*") or
    ((process.executable rlike "/tmp/[^/]+" or process.executable rlike "/var/tmp/[^/]+")) or
    (process.executable rlike "/run/[^/]+") or
    (process.executable rlike "/var/run/[^/]+")
) and
destination.ip IS NOT null and not CIDR_MATCH(destination.ip, "127.0.0.0/8", "169.254.0.0/16", "224.0.0.0/4", "::1")
| STATS process_count = COUNT(process.name), agent_count = COUNT_DISTINCT(agent.id) by process.executable
// Alter this threshold to make sense for your environment 
| WHERE agent_count <= 3 and process_count > 0 and process_count <= 5
| LIMIT 100 
| SORT process_count asc

[Aegrah]

Excessive SSH network activity to unique destinations

FROM logs-endpoint.events.network-*
| WHERE @timestamp > NOW() - 7 day
| WHERE host.os.type == "linux" and event.category == "network" and network.transport == "tcp" and destination.port == 22 and source.port >= 49152 
| KEEP destination.ip, host.id, user.name
| STATS count_unique_dst = COUNT_DISTINCT(destination.ip) by host.id, user.name
// Alter this threshold to make sense for your environment 
| WHERE count_unique_dst >= 10
| LIMIT 100 
| SORT user.name asc

[Aegrah]

Shell execution from low occurrence suspicious process parent

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.type == "start" and event.action == "exec" and process.name in ("bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish") and not process.parent.pid == 1 and (
    (process.parent.executable like "./") or
    (process.parent.executable like "/dev/shm/*") or
    (process.parent.executable like "/var/www/*") or
    (process.parent.executable like "/boot/*") or
    (process.parent.executable like "/srv/*") or
    ((process.parent.executable rlike "/tmp/[^/]+" or process.parent.executable rlike "/var/tmp/[^/]+")) or
    (process.parent.executable rlike "/run/[^/]+") or
    (process.parent.executable rlike "/var/run/[^/]+")
)
| STATS agent_count = COUNT_DISTINCT(agent.id), cc = COUNT(*) by process.parent.executable
// Alter this threshold to make sense for your environment 
| WHERE agent_count <= 3 and cc <= 5
| LIMIT 100 
| SORT cc asc

[Aegrah]

Logon activity by source IP

FROM logs-system.auth-*
| WHERE @timestamp > NOW() - 7 day
| WHERE host.os.type == "linux" and event.category == "authentication" and event.action in ("ssh_login", "user_login") and event.outcome == "failure" and source.ip IS NOT null and not CIDR_MATCH(source.ip, "127.0.0.0/8", "169.254.0.0/16", "224.0.0.0/4", "::1")
| EVAL failed = CASE(event.outcome == "failure", source.ip, null), success = CASE(event.outcome == "success", source.ip, null)
| STATS count_failed = COUNT(failed), count_success = COUNT(success), count_user = count_distinct(user.name) by source.ip
 /* below threshold should be adjusted to your env logon patterns */
| WHERE count_failed >= 100 and count_success <= 10 and count_user >= 20

[Aegrah]

Low volume external network connections from process by unique agent

FROM logs-endpoint.events.network-*
| WHERE  @timestamp > now() - 7 day 
| WHERE host.os.type == "linux" and event.category == "network" and event.type == "start" and event.action == "connection_attempted" and not process.name is null and
    not CIDR_MATCH(destination.ip, "10.0.0.0/8", "127.0.0.0/8", "169.254.0.0/16", "172.16.0.0/12", "192.0.0.0/24", "192.0.0.0/29", "192.0.0.8/32", "192.0.0.9/32", "192.0.0.10/32", "192.0.0.170/32", "192.0.0.171/32", "192.0.2.0/24", "192.31.196.0/24", "192.52.193.0/24", "192.168.0.0/16", "192.88.99.0/24", "224.0.0.0/4", "100.64.0.0/10", "192.175.48.0/24","198.18.0.0/15", "198.51.100.0/24", "203.0.113.0/24", "240.0.0.0/4", "::1","FE80::/10", "FF00::/8")
| STATS connection_count = COUNT(*), unique_agent_count = COUNT_DISTINCT(agent.id) by process.name
| WHERE connection_count <= 5 and unique_agent_count == 1
| LIMIT 100 
| SORT connection_count, unique_agent_count asc

Low volume root external network connections from process by unique agent

FROM logs-endpoint.events.network-*
| WHERE  @timestamp > now() - 7 day 
| WHERE host.os.type == "linux" and event.category == "network" and event.type == "start" and event.action == "connection_attempted" and user.id == "0" and not process.name is null and
    not CIDR_MATCH(destination.ip, "10.0.0.0/8", "127.0.0.0/8", "169.254.0.0/16", "172.16.0.0/12", "192.0.0.0/24", "192.0.0.0/29", "192.0.0.8/32", "192.0.0.9/32", "192.0.0.10/32", "192.0.0.170/32", "192.0.0.171/32", "192.0.2.0/24", "192.31.196.0/24", "192.52.193.0/24", "192.168.0.0/16", "192.88.99.0/24", "224.0.0.0/4", "100.64.0.0/10", "192.175.48.0/24","198.18.0.0/15", "198.51.100.0/24", "203.0.113.0/24", "240.0.0.0/4", "::1","FE80::/10", "FF00::/8")
| STATS connection_count = COUNT(*), unique_agent_count = COUNT_DISTINCT(agent.id) by process.name
| WHERE connection_count <= 5 and unique_agent_count == 1
| LIMIT 100 
| SORT connection_count, unique_agent_count asc

[Aegrah]

Low volume modifications to critical system binaries by unique host

FROM logs-endpoint.events.file-*
| WHERE @timestamp > NOW() - 7 day
| WHERE host.os.type == "linux" and (
  (file.path like "/bin/*") or
  (file.path like "/usr/bin/*") or
  (file.path like "/sbin/*") or
  (file.path like "/usr/sbin/*")
) and not (
  // Exclude expected update processes, e.g., package managers
  (process.executable in ("/usr/bin/apt", "/usr/bin/dpkg", "/usr/bin/yum", "/usr/bin/rpm", "/usr/bin/pacman", "/usr/bin/pamac-daemon", "/usr/bin/update-alternatives", "/usr/bin/dockerd", "/usr/bin/microdnf", "/sbin/apk")) or
  // Exclude certain benign or expected modification patterns, if applicable
  (file.path like "/usr/bin/gzip*") // Example exclusion, adjust based on your environment
)
| STATS modification_count = COUNT(file.path), unique_files_modified = COUNT_DISTINCT(file.path), host_count = COUNT(host.name) by process.executable, host.name, user.name
// Alter this threshold based on typical behavior in your environment 
| WHERE modification_count >= 1 and host_count == 1
| SORT modification_count asc
| LIMIT 100

[Aegrah]

Low volume process injection-related syscalls by process executable

FROM logs-auditd_manager.auditd-*, logs-auditd.log-*, auditbeat-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and auditd.data.syscall in ("ptrace", "memfd_create")
| STATS cc = COUNT(*) by process.executable, auditd.data.syscall
| WHERE cc <= 10
| LIMIT 100 
| SORT cc asc

[Aegrah]

Low volume GTFOBins external network connections

FROM logs-endpoint.events.network-*
| WHERE @timestamp > NOW() - 7 day
| WHERE host.os.type == "linux" and event.type == "start" and process.name in (
  "ab", "aria2c", "bash", "cpan", "curl", "easy_install", "finger", "ftp",
  "gdb", "gimp", "irb", "jjs", "jrunscript", "julia", "ksh", "lua", "lwp-download",
  "nc", "nmap", "node", "openssl", "php", "pip", "python", "ruby", "rview", "rvim",
  "scp", "sftp", "smbclient", "socat", "ssh", "tar", "tftp", "view", "vim", "vimdiff",
  "wget", "whois", "yum"
) and
destination.ip IS NOT null and not CIDR_MATCH(destination.ip, "10.0.0.0/8", "127.0.0.0/8", "169.254.0.0/16", "172.16.0.0/12", "192.0.0.0/24", "192.0.0.0/29", "192.0.0.8/32", "192.0.0.9/32", "192.0.0.10/32", "192.0.0.170/32", "192.0.0.171/32", "192.0.2.0/24", "192.31.196.0/24", "192.52.193.0/24", "192.168.0.0/16", "192.88.99.0/24", "224.0.0.0/4", "100.64.0.0/10", "192.175.48.0/24","198.18.0.0/15", "198.51.100.0/24", "203.0.113.0/24", "224.0.0.0/4", "240.0.0.0/4", "::1","FE80::/10", "FF00::/8") 
| KEEP process.name, destination.port, destination.ip, user.name, host.name
| STATS cc = COUNT(*) by destination.port, process.name, host.name, user.name
| WHERE cc <= 5
| SORT cc asc, destination.port

[botelastic[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Aegrah]

Shell Modification Persistence

ES|QL File creation/modification

FROM logs-endpoint.events.file-*
| WHERE @timestamp > NOW() - 90 day
| WHERE host.os.type == "linux" and event.type in ("creation", "change") and (

    // System-wide profile files
    file.path in ("/etc/profile", "/etc/bash.bashrc", "/etc/bash.bash_logout") or
    file.path like "/etc/profile.d/*" or

    // User-specific profile files
    file.path like "/home/*/.profile" or
    file.path like "/home/*/.bash_profile" or
    file.path like "/home/*/.bash_login" or
    file.path like "/home/*/.bash_logout" or
    file.path like "/home/*/.bashrc"
) and not (
    process.name in (
      "dpkg", "dockerd", "yum", "dnf", "snapd", "pacman", "pamac-daemon", "microdnf", "podman", "apk"
    ) or
    process.executable == "/proc/self/exe" or
    process.executable like "/dev/fd/*" or
    file.extension in ("dpkg-remove", "swx", "swp")
)
| EVAL persistence = CASE(
    // System-wide profile files
    file.path in ("/etc/profile", "/etc/bash.bashrc", "/etc/bash.bash_logout") or
    file.path like "/etc/profile.d/*" or

    // User-specific profile files
    file.path like "/home/*/.profile" or
    file.path like "/home/*/.bash_profile" or
    file.path like "/home/*/.bash_login" or
    file.path like "/home/*/.bash_logout" or
    file.path like "/home/*/.bashrc",
    process.name,
    null
)
| STATS cc = COUNT(*), pers_count = COUNT(persistence), agent_count = COUNT(agent.id) by process.executable, file.path, host.name, user.name
| WHERE pers_count > 0 and pers_count <= 20 and agent_count <= 4
| SORT cc asc
| LIMIT 500

ES|QL SSH Parent Execution

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 90 day
| WHERE host.os.type == "linux" and event.type == "start" and event.action == "exec" and process.parent.name == "sshd"
| STATS cc = COUNT(*), agent_count = COUNT(agent.id) by process.executable, process.command_line, host.name, user.name
| WHERE cc <= 20 and agent_count <= 4
| SORT cc asc
| LIMIT 100

OSQuery Files

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
WHERE 
    f.path IN ("/etc/profile", "/etc/bash.bashrc", "/etc/bash.bash_logout")
    OR f.path LIKE "/etc/profile.d/%"
    OR f.path LIKE "/home/%/.profile"
    OR f.path LIKE "/home/%/.bash_profile"
    OR f.path LIKE "/home/%/.bash_login"
    OR f.path LIKE "/home/%/.bash_logout"
    OR f.path LIKE "/home/%/.bashrc"

[Aegrah]

XDG persistence

ES|QL File creation / Modification

FROM logs-endpoint.events.file-*
| WHERE @timestamp > NOW() - 90 day
| WHERE host.os.type == "linux" and event.type in ("creation", "change") and (

    // System-wide autostart directories
    file.path like "/etc/xdg/autostart/*" or
    file.path like "/usr/share/autostart/*" or

    // User-specific autostart directories
    file.path like "/home/*/.config/autostart/*" or
    file.path like "/home/*/.local/share/autostart/*" or
    file.path like "/home/*/.config/autostart-scripts/*" or

    // Root-specific autostart directories
    file.path like "/root/.config/autostart/*" or
    file.path like "/root/.local/share/autostart/*" or
    file.path like "/root/.config/autostart-scripts/*"
) and not (
    process.name in (
      "dpkg", "dockerd", "yum", "dnf", "snapd", "pacman", "pamac-daemon", "microdnf", "podman", "apk"
    ) or
    process.executable == "/proc/self/exe" or
    process.executable like "/dev/fd/*" or
    file.extension in ("dpkg-remove", "swx", "swp")
)
| EVAL persistence = CASE(
    // System-wide autostart directories
    file.path like "/etc/xdg/autostart/*" or
    file.path like "/usr/share/autostart/*" or

    // User-specific autostart directories
    file.path like "/home/*/.config/autostart/*" or
    file.path like "/home/*/.local/share/autostart/*" or
    file.path like "/home/*/.config/autostart-scripts/*" or

    // Root-specific autostart directories
    file.path like "/root/.config/autostart/*" or
    file.path like "/root/.local/share/autostart/*" or
    file.path like "/root/.config/autostart-scripts/*",
    process.name,
    null
)
| STATS cc = COUNT(*), pers_count = COUNT(persistence), agent_count = COUNT(agent.id) by process.executable, file.path, host.name, user.name
| WHERE pers_count > 0 and pers_count <= 20 and agent_count <= 4
| SORT cc asc
| LIMIT 100

ES|QL Execution

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 90 day
| WHERE host.os.type == "linux" and event.type == "start" and event.action == "exec" and process.parent.name in (
  "plasmashell", "gnome-session", "xfce4-session", "gnome-session-binary", "mate-session", "cinnamon-session",
  "lxsession", "lxqt-session", "unity-session", "pantheon-session", "enlightenment_start"
)
| STATS cc = COUNT(*), agent_count = COUNT(agent.id) by process.executable, process.command_line, host.name, user.name, process.parent.executable
| WHERE cc <= 20 and agent_count <= 4
| SORT cc asc
| LIMIT 100

OSQuery Enabled XDG Startup Items

SELECT name, path, source, status, type FROM startup_items
WHERE type == "Startup Item" AND status == "enabled" AND (
    source LIKE "/etc/xdg/autostart/%"
    OR source LIKE "/usr/share/autostart/%"
    OR source LIKE "/home/%/.config/autostart/%"
    OR source LIKE "/home/%/.local/share/autostart/%"
    OR source LIKE "/home/%/.config/autostart-scripts/%"
    OR source LIKE "/root/.config/autostart/%"
    OR source LIKE "/root/.local/share/autostart/%"
    OR source LIKE "/root/.config/autostart-scripts/%"
)

OSQuery Files

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
WHERE 
    f.path LIKE "/etc/xdg/autostart/%"
    OR f.path LIKE "/usr/share/autostart/%"
    OR f.path LIKE "/home/%/.config/autostart/%"
    OR f.path LIKE "/home/%/.local/share/autostart/%"
    OR f.path LIKE "/home/%/.config/autostart-scripts/%"
    OR f.path LIKE "/root/.config/autostart/%"
    OR f.path LIKE "/root/.local/share/autostart/%"
    OR f.path LIKE "/root/.config/autostart-scripts/%"

[Aegrah]

OSQuery SUID Hunting 1

SELECT * FROM suid_bin

OSQuery SUID Hunting 2

SELECT 
    f.filename, 
    f.path,
    f.mode,
    f.uid,
    f.gid,
    f.type,
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
WHERE
f.type == "regular" AND
(f.uid == 0 or f.gid == 0) AND
(f.mode LIKE "2%" OR f.mode LIKE "4%") AND
(
  f.path LIKE "/%%" OR
  f.path LIKE "/%%/%%" OR
  f.path LIKE "/%%/%%/%%" OR
  f.path LIKE "/%%/%%/%%/%%"
)

OSQuery requires a path to be specified, and wildcards are not recursive. Adding more paths makes my OSQuery instance return 0 results, so the user can change this for his environment.

[Aegrah]

Sudoers

OSQuery Sudoers rule hunt

SELECT * FROM sudoers

[Aegrah]

User/Group Creation/Modification

OSQuery

SELECT * FROM shadow

SELECT * FROM shadow
WHERE password_status != "locked"

SELECT username, gid, uid, shell, description FROM users
WHERE username != 'root' AND uid LIKE "0"

uid LIKE "0" is needed due to some strange formatting in OSQuery

SELECT * FROM users WHERE username = "newuser"

All info from a specific user

SELECT * FROM logged_in_users WHERE user = "newuser"

Authentication status for a specific user

SELECT pid, username, name FROM processes p JOIN users u ON u.uid = p.uid ORDER BY username

Get processes ran per user

[Aegrah]

SSH Persistence

OSQuery SSH keys

SELECT * FROM user_ssh_keys

OSquery get authorized_keys information

SELECT authorized_keys.*   
FROM users   
JOIN authorized_keys   
USING(uid)

OSQuery get SSH Configuration

SELECT * FROM ssh_configs

OSQuery get SSH file information

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
WHERE 
    f.path LIKE "/root/.ssh/%"
    OR f.path LIKE "/home/%/.ssh/%"
    OR f.path LIKE "/etc/ssh/%"
    OR f.path LIKE "/etc/ssh/sshd_config.d/%"
    OR f.path LIKE "/etc/ssh/ssh_config.d/%"

[Aegrah]

Bind/Reverse shell hunting

OSQuery Listening Ports

SELECT pid, address, port, socket, protocol, path FROM listening_ports

OSQuery Open Sockets

SELECT * FROM process_open_sockets

OSQuery Get Running Processes

SELECT * FROM processes p JOIN users u ON u.uid = p.uid ORDER BY username

[IanLee1521]

This looks awesome @Aegrah Thanks for putting it together! @jamesspi sent me. :)

As someone that can't use Endpoint effectively in my environment, I wonder if you would consider including more of the rules / queries that use auditd data instead? I thought I'd seen something recently about the security tooling starting to support not just Endpoint but also auditd data. That is much more readily available for us.

I'll also just say (and would be happy to discuss this more) that I'm really interested in the idea of using some of these queries as benchmarks of real ESQL calls in our environment on real data (and for other reasons, using them to seed some drag races comparing to similar searches in Splunk).

[Aegrah]

Hello @IanLee1521, thank you!

I greatly appreciate the Auditd dataset as well, and attempt to use it to fill in gaps that we have with Elastic Defend. Unfortunately, Elastic Defend is used way more by our customers, and therefore our priority lies there.

However, some of these queries are likely compatible with Auditd data with some tweaking. I also would be interested in exploring this more, but that would be a project for a later point in time.

Regarding your second point, that would be great! Real customer feedback is something that we appreciate a lot, and if that helps us tune out our hunting ruleset that would be great.

I am continuing this hunting rule set today to add some OSQuery/ES|QL hunts for additional persistence mechanisms, and we are looking to fine-tune and release this to our detection rules repository soon as well.

[IanLee1521]

I actually had issues with auditd log handling at one point about a year ago with Elastic, and I've been overhauling our rule configuration recently, hence why I'm interested in that set. I'll have to see how well things work when I get there in the next few weeks hopefully.

Yeah, Defend is definitely of interest, there is a limitation that it doesn't deploy well in a diskless compute environment (https://github.com/elastic/endpoint/issues/69), which has paused our adoption in the HPC world.

As a question (I'm new to this particular repo), is the idea that these rules will eventually turn in to content in the hunting/ top level dir? I haven't contributed here before, but I wonder if there is room to include those "alternate" implementations of the queries when they could use Endpoint or Auditd data?

As far as performance testing, that was the originally pointed to this repo, to source queries that I could use to try to measure / compare performance, similar to what I did here. I admit though that I have some trouble with timing numbers due to lacking a "wall clock time" being reported in Kibana (that's a separate issue I finally just opened: https://github.com/elastic/kibana/issues/187051)

Looking forward to collaborating a bit on this!

[Aegrah]

Persistence via Udev

File

FROM logs-endpoint.events.file-*
| WHERE @timestamp > NOW() - 90 day
| WHERE host.os.type == "linux" and event.type in ("creation", "change") and (
    file.path like "/etc/udev/rules.d/*" or
    file.path like "/run/udev/rules.d/*" or
    file.path like "/usr/lib/udev/rules.d/*" or
    file.path like "/lib/udev/*"
) and not process.name in (
  "dpkg", "dockerd", "yum", "dnf", "snapd", "pacman", "pamac-daemon",
  "microdnf", "podman", "apk", "netplan", "generate"
)
| EVAL persistence = CASE(
    file.path like "/etc/udev/rules.d/*" or
    file.path like "/run/udev/rules.d/*" or
    file.path like "/usr/lib/udev/rules.d/*" or
    file.path like "/lib/udev/*",
    process.name,
    null
)
| STATS cc = COUNT(*), pers_count = COUNT(persistence), agent_count = COUNT(agent.id) by process.executable, file.path, host.name, user.name
| WHERE pers_count > 0 and pers_count <= 20 and agent_count <= 4
| SORT cc asc

Process

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.action == "exec" and event.type == "start" and process.parent.name == "udevadm" and
// Excluding these because this is typical udev behavior.
// If you suspect Udev persistence, remove this exclusion in order to do a more elaborate search
not (process.executable like "/lib/*" or process.executable like "/usr/lib/*")
| STATS process_cli_count = COUNT(process.command_line), process_count = COUNT(process.executable), host_count = COUNT_DISTINCT(host.name) by process.executable
// Tweak the process/host count if you suspect Udev persistence
| WHERE host_count <= 5 and process_count < 50
| SORT process_cli_count asc
| LIMIT 100

OSQuery File

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes,
    h.md5 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
LEFT JOIN 
    hash h ON f.path = h.path 
WHERE 
    f.directory IN (
        '/etc/udev/rules.d/',
        '/run/udev/rules.d/',
        '/usr/lib/udev/rules.d/',
        '/lib/udev/'
    )
ORDER BY 
    f.mtime DESC;

[Aegrah]

Package Manager Persistence

ESQL File

FROM logs-endpoint.events.file-*
| WHERE @timestamp > NOW() - 90 day
| WHERE host.os.type == "linux" and event.type in ("creation", "change") and (
    file.path like "/etc/apt/apt.conf.d/*" or
    file.path like "/usr/lib/python%/site-packages/dnf-plugins/*" or
    file.path like "/etc/dnf/plugins/*" or
    file.path like "/usr/lib/yum-plugins/*" or
    file.path like "/etc/yum/pluginconf.d/*"
) and not process.name in (
  "dpkg", "dockerd", "yum", "dnf", "snapd", "pacman", "pamac-daemon",
  "microdnf", "podman", "apk", "yumBackend.py"
)
| EVAL persistence = CASE(
    file.path like "/etc/apt/apt.conf.d/*" or
    file.path like "/usr/lib/python%/site-packages/dnf-plugins/*" or
    file.path like "/etc/dnf/plugins/*" or
    file.path like "/usr/lib/yum-plugins/*" or
    file.path like "/etc/yum/pluginconf.d/*",
    process.name,
    null
)
| STATS cc = COUNT(*), pers_count = COUNT(persistence), agent_count = COUNT(agent.id) by process.executable, file.path, host.name, user.name
| WHERE pers_count > 0 and pers_count <= 20 and agent_count <= 4
| SORT cc asc

ESQL Process

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.action == "exec" and event.type == "start" and process.parent.name in ("apt", "yum", "dnf")
| STATS process_cli_count = COUNT(process.command_line), process_count = COUNT(process.executable), host_count = COUNT_DISTINCT(host.name) by process.executable
// Tweak the process/host count if you suspect Udev persistence
| WHERE host_count <= 5 and process_count < 50
| SORT process_cli_count asc
| LIMIT 100

OSQuery File

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
WHERE 
    f.path LIKE '/etc/apt/apt.conf.d/%'
    OR f.path LIKE '/usr/lib/python%/site-packages/dnf-plugins/%'
    OR f.path LIKE '/etc/dnf/plugins/%'
    OR f.path LIKE '/usr/lib/yum-plugins/%'
    OR f.path LIKE '/etc/yum/pluginconf.d/%'

OSQuery Apt Sources

SELECT * FROM apt_sources

OSQuery YUM Sources

SELECT * FROM yum_sources

[Aegrah]

Git Hook/Pager Persistence

ESQL File Creations

FROM logs-endpoint.events.file-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.type in ("creation", "change") and (
    file.path == "/etc/gitconfig" or
    file.path like "*/.git/config" or
    file.path like "/home/*/.gitconfig" or
    file.path like "*/.git/hooks/*"
) and process.name != "git"
| EVAL persistence = CASE(
    file.path == "/etc/gitconfig" or
    file.path like "*/.git/config" or
    file.path like "/home/*/.gitconfig" or
    file.path like "*/.git/hooks/*",
    process.name,
    null
)
| STATS cc = COUNT(*), pers_count = COUNT(persistence), agent_count = COUNT(agent.id) by process.executable, file.path, host.name, user.name
| WHERE pers_count > 0 and pers_count <= 20 and agent_count <= 4
| SORT cc asc

ESQL Process

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.action == "exec" and event.type == "start" and process.parent.executable like "*.git/hooks/*"
| STATS process_cli_count = COUNT(process.command_line), process_count = COUNT(process.executable), host_count = COUNT_DISTINCT(host.name) by process.parent.executable, process.executable
| WHERE host_count <= 5 and process_count < 50
| SORT process_cli_count asc
| LIMIT 100

OSQuery File Creations

SELECT 
    f.filename, 
    f.path, 
    u.username AS file_owner, 
    g.groupname AS group_owner, 
    datetime(f.atime, 'unixepoch') AS file_last_access_time, 
    datetime(f.mtime, 'unixepoch') AS file_last_modified_time, 
    datetime(f.ctime, 'unixepoch') AS file_last_status_change_time, 
    datetime(f.btime, 'unixepoch') AS file_created_time, 
    f.size AS size_bytes 
FROM 
    file f 
LEFT JOIN 
    users u ON f.uid = u.uid 
LEFT JOIN 
    groups g ON f.gid = g.gid 
WHERE 
    f.path == '/etc/gitconfig'
    OR f.path LIKE '/%%/.git/config'
    OR f.path LIKE '/home/%/.gitconfig'
    OR f.path LIKE '/%%/.git/hooks/%'
    OR f.path LIKE '/%%/%%/.git/hooks/%'
    OR f.path LIKE '/%%/%%/%%/.git/hooks/%'
    OR f.path LIKE '/%%/%%/%%/%%/.git/hooks/%'

[Aegrah]

Process Capability Hunting

ESQL Process Execution with Process Capability Set

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.action == "exec" and event.type == "start" and (process.thread.capabilities.effective is not null or process.thread.capabilities.permitted is not null) and user.id != "0" and
not (
  // Remove these if you expect persistence through capabilities
  process.executable like "/var/lib/docker/*" or
  process.name == "gnome-keyring-daemon" or
  process.thread.capabilities.permitted == "CAP_WAKE_ALARM"
)
| STATS process_cli_count = COUNT(process.command_line), process_count = COUNT(process.executable), host_count = COUNT_DISTINCT(host.name) by process.parent.executable, process.executable, process.command_line, process.thread.capabilities.effective, process.thread.capabilities.permitted, user.id
| WHERE host_count <= 3 and process_count < 5
| SORT process_cli_count asc
| LIMIT 100

ESQL Processes with Dangerous Capabilities set

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 90 day
| WHERE host.os.type == "linux" and event.action == "exec" and event.type == "start" and (
  process.thread.capabilities.effective in ("CAP_SYS_MODULE", "CAP_SYS_PTRACE", "CAP_DAC_OVERRIDE", "CAP_DAC_READ_SEARCH", "CAP_SETUID", "CAP_SETGID", "CAP_SYS_ADMIN") or
  process.thread.capabilities.permitted in ("CAP_SYS_MODULE", "CAP_SYS_PTRACE", "CAP_DAC_OVERRIDE", "CAP_DAC_READ_SEARCH", "CAP_SETUID", "CAP_SETGID", "CAP_SYS_ADMIN")
) and user.id != "0"
| STATS process_cli_count = COUNT(process.command_line), process_count = COUNT(process.executable), host_count = COUNT_DISTINCT(host.name) by process.parent.executable, process.executable, process.command_line, process.thread.capabilities.effective, process.thread.capabilities.permitted, user.id
| WHERE host_count <= 3 and process_count < 5
| SORT process_cli_count asc
| LIMIT 100

[Aegrah]

Unsual System Binary Parent (potential system binary hijacking attempt)

FROM logs-endpoint.events.process-*
| WHERE @timestamp > NOW() - 30 day
| WHERE host.os.type == "linux" and event.action == "exec" and event.type == "start" and process.parent.name in ("ls", "cat", "mkdir", "touch", "mv", "cp")
| STATS process_cli_count = COUNT(process.command_line), process_count = COUNT(process.executable), host_count = COUNT_DISTINCT(host.name) by process.parent.executable, process.executable
| WHERE host_count <= 5
| SORT process_cli_count asc
| LIMIT 100