search

crowdsecurity / hub

Main repository for crowdsec scenarios/parsers
https://hub.crowdsec.net
160 stars 146 forks source link

Support for Sophos XG Firewall IPS (Snort) logs or generic security logs #754

Closed lordraiden closed 1 year ago

lordraiden commented 1 year ago

Sophos XG IPS logs have this format 2023-06-01 18:04:04IPSmessageid="07002" log_type="IDP" log_component="Signatures" log_subtype="Drop" ips_policy="" ips_policy_id="7" fw_rule_id="8" fw_rule_name="WAN to DMZ (50) - WebProxy" fw_rule_section="Local rule" user="" sig_id="2300357" message="SERVER-WEBAPP ThinkPHP 5.0.23/5.1.31 CVE-2018-20062 Remote Code Execution" classification="Unknown" rule_priority="1" src_ip="47.100.56.27" src_country="CHN" dst_ip="10.10.50.10" dst_country="R1" protocol="TCP" src_port="60253" dst_port="80" OS="Linux,Other,Solaris,Unix,Windows" category="server-webapp" victim="Server"

I just need to ban the IP's that are covered in [ src_ip="xxx.xxx.xxx.xxx" ]

Can someone develop a simple collection to do this? just for each log entry take the source ip and ban it... no correlation or anything else needed.

The use case is much more stronger It could serve for much more purposes, any security log just by changing the regex of the parser. So since the security tool has already taking a decision about that IP, no correlation or anything is needed by Crowdsec, just parsing the logs getting the attackers IP to configure later a response action like a ban

LaurenceJJones commented 1 year ago

So a simple parser for version 1.5 can be this

onsuccess: next_stage
#debug: false
name: crowdsecurity/sophos
description: "Parse sophos logs"
filter: "evt.Parsed.program == 'sophos'"
pattern_syntax:
  SOPHOS_TIMESTAMP: "%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}"
nodes:
  - grok:
      pattern: "%{SOPHOS_TIMESTAMP:timestamp}%{GREEDYDATA:kvmessage}"
      apply_on: message
statics:
   - meta: unused ## this is not used but must be set
     expression: "ParseKV(evt.Parsed.kvmessage, evt.Unmarshaled, 'sophos')"
   - target: evt.StrTime
     expression: evt.Parsed.timestamp
   - meta: source_ip
     expression: evt.Unmarshaled.sophos.src_ip
   - meta: sub_type
     value: sophos_log
line: 2023-06-01 18:04:04IPSmessageid="07002" log_type="IDP" log_component="Signatures" log_subtype="Drop" ips_policy="" ips_policy_id="7" fw_rule_id="8" fw_rule_name="WAN to DMZ (50) - WebProxy" fw_rule_section="Local rule" user="" sig_id="2300357" message="SERVER-WEBAPP ThinkPHP 5.0.23/5.1.31 CVE-2018-20062 Remote Code Execution" classification="Unknown" rule_priority="1" src_ip="47.100.56.27" src_country="CHN" dst_ip="10.10.50.10" dst_country="R1" protocol="TCP" src_port="60253" dst_port="80" OS="Linux,Other,Solaris,Unix,Windows" category="server-webapp" victim="Server"
├ s00-raw
|       ├ 🟢 crowdsecurity/non-syslog (+5 ~8)
|               ├ update evt.ExpectMode : %!s(int=0) -> 1
|               ├ update evt.Stage :  -> s01-parse
|               ├ update evt.Line.Raw :  -> 2023-06-01 18:04:04IPSmessageid="07002" log_type="IDP" log_component="Signatures" log_subtype="Drop" ips_policy="" ips_policy_id="7" fw_rule_id="8" fw_rule_name="WAN to DMZ (50) - WebProxy" fw_rule_section="Local rule" user="" sig_id="2300357" message="SERVER-WEBAPP ThinkPHP 5.0.23/5.1.31 CVE-2018-20062 Remote Code Execution" classification="Unknown" rule_priority="1" src_ip="47.100.56.27" src_country="CHN" dst_ip="10.10.50.10" dst_country="R1" protocol="TCP" src_port="60253" dst_port="80" OS="Linux,Other,Solaris,Unix,Windows" category="server-webapp" victim="Server"
|               ├ update evt.Line.Src :  -> /tmp/sophos.log
|               ├ update evt.Line.Time : 0001-01-01 00:00:00 +0000 UTC -> 2023-06-04 21:25:54.811962287 +0000 UTC
|               ├ create evt.Line.Labels.type : sophos
|               ├ update evt.Line.Process : %!s(bool=false) -> true
|               ├ update evt.Line.Module :  -> file
|               ├ create evt.Parsed.message : 2023-06-01 18:04:04IPSmessageid="07002" log_type="IDP" log_component="Signatures" log_subtype="Drop" ips_policy="" ips_policy_id="7" fw_rule_id="8" fw_rule_name="WAN to DMZ (50) - WebProxy" fw_rule_section="Local rule" user="" sig_id="2300357" message="SERVER-WEBAPP ThinkPHP 5.0.23/5.1.31 CVE-2018-20062 Remote Code Execution" classification="Unknown" rule_priority="1" src_ip="47.100.56.27" src_country="CHN" dst_ip="10.10.50.10" dst_country="R1" protocol="TCP" src_port="60253" dst_port="80" OS="Linux,Other,Solaris,Unix,Windows" category="server-webapp" victim="Server"
|               ├ create evt.Parsed.program : sophos
|               ├ update evt.Time : 0001-01-01 00:00:00 +0000 UTC -> 2023-06-04 21:25:54.812048402 +0000 UTC
|               ├ create evt.Meta.datasource_path : /tmp/sophos.log
|               ├ create evt.Meta.datasource_type : file
├ s01-parse
|       ├ 🟢 crowdsecurity/sophos (+4 ~2)
|               ├ update evt.Stage : s01-parse -> s02-enrich
|               ├ create evt.Parsed.kvmessage : IPSmessageid="07002" log_type="IDP" log_component="Signatures" log_subtype="Drop" ips_policy="" ips_policy_id="7" fw_rule_id="8" fw_rule_name="WAN to DMZ (50) - WebProxy" fw_rule_section="Local rule" user="" sig_id="2300357" message="SERVER-WEBAPP ThinkPHP 5.0.23/5.1.31 CVE-2018-20062 Remote Code Execution" classification="Unknown" rule_priority="1" src_ip="47.100.56.27" src_country="CHN" dst_ip="10.10.50.10" dst_country="R1" protocol="TCP" src_port="60253" dst_port="80" OS="Linux,Other,Solaris,Unix,Windows" category="server-webapp" victim="Server"
|               ├ create evt.Parsed.timestamp : 2023-06-01 18:04:04
|               ├ create evt.Unmarshaled.sophos : map[IPSmessageid:07002 OS:Linux,Other,Solaris,Unix,Windows category:server-webapp classification:Unknown dst_country:R1 dst_ip:10.10.50.10 dst_port:80 fw_rule_id:8 fw_rule_name:WAN to DMZ (50) - WebProxy fw_rule_section:Local rule ips_policy: ips_policy_id:7 log_component:Signatures log_subtype:Drop log_type:IDP message:SERVER-WEBAPP ThinkPHP 5.0.23/5.1.31 CVE-2018-20062 Remote Code Execution protocol:TCP rule_priority:1 sig_id:2300357 src_country:CHN src_ip:47.100.56.27 src_port:60253 user: victim:Server]
|               ├ update evt.StrTime :  -> 2023-06-01 18:04:04
|               ├ create evt.Meta.source_ip : 47.100.56.27
├ s02-enrich
|       ├ 🟢 crowdsecurity/dateparse-enrich (+2 ~2)
|               ├ create evt.Enriched.MarshaledTime : 2023-06-01T18:04:04Z
|               ├ update evt.Time : 2023-06-04 21:25:54.812048402 +0000 UTC -> 2023-06-01 18:04:04 +0000 UTC
|               ├ update evt.MarshaledTime :  -> 2023-06-01T18:04:04Z
|               ├ create evt.Meta.timestamp : 2023-06-01T18:04:04Z
|       ├ 🟢 crowdsecurity/geoip-enrich (+13)
|               ├ create evt.Enriched.ASNNumber : 37963
|               ├ create evt.Enriched.IsInEU : false
|               ├ create evt.Enriched.IsoCode : CN
|               ├ create evt.Enriched.Latitude : 31.222200
|               ├ create evt.Enriched.ASNOrg : Hangzhou Alibaba Advertising Co.,Ltd.
|               ├ create evt.Enriched.ASNumber : 37963
|               ├ create evt.Enriched.Longitude : 121.458100
|               ├ create evt.Enriched.SourceRange : 47.96.0.0/12
|               ├ create evt.Meta.SourceRange : 47.96.0.0/12
|               ├ create evt.Meta.ASNNumber : 37963
|               ├ create evt.Meta.ASNOrg : Hangzhou Alibaba Advertising Co.,Ltd.
|               ├ create evt.Meta.IsInEU : false
|               ├ create evt.Meta.IsoCode : CN
|       └ 🟢 crowdsecurity/whitelists (unchanged)
├-------- parser success 🟢├ Scenarios
LaurenceJJones commented 1 year ago

Did this managed to help @lordraiden ??

LaurenceJJones commented 1 year ago

Closing issue till either OP comes back or more user needs a parser for this use case

lordraiden commented 1 year ago

Closing issue till either OP comes back or more user needs a parser for this use case

Hi, sorry for not answering before I have been evolving my log stack to try to simplify the process. Right now all my logs are in graylog, different sources not only sophos, with graylog I'm able to filter whatever I want and send it to crowdsec via syslog.

I have seen how to configure syslog here: https://docs.crowdsec.net/docs/v1.3.4/data_sources/syslog

But I'm totally lost on how to configure crowdsec for this until now I had configure acquis.yaml and added collections to the docker compose file COLLECTIONS: "crowdsecurity/traefik crowdsecurity/home-assistant crowdsecurity/http-cve crowdsecurity/whitelist-good-actors"

But I think I will manage to do it following this example. https://docs.crowdsec.net/docs/parsers/create

But still after reading it I don't understand what is the difference between grok and statics? Why do you use grok to capture the timestamps but you don't use it to capture the src_ip.

If I only one to process logs with log_type="IDP" log_component="Signatures" this should be configured as a node?

what is the purpose of this?

Could you list all the files that I need to create or eddit to make this work? to block the IP's parsed