Cisco incomplete patterns

ragri8 commented 5 years ago

I've been working with the Elastic stack and cisco ASA logs for 2 months so far. I started parsing them with the logstash firewalls pattern used for grok match and now I switching to the ECS format. I'm not using Filebeat cisco module to transfer the data, but I need to follow the pattern to be compatible with SIEM.

From what I can see, some events doesn't translate pretty well. I won't look at all of them, but the event ASA-6-302021 is a pretty weird case which need more attention IMHO.

This is the grok pattern format:

# ASA-6-302020, ASA-6-302021
CISCOFW302020_302021 %{CISCO_ACTION:action}(?: %{CISCO_DIRECTION:direction})? %{WORD:protocol} connection for faddr %{IP:dst_ip}/%{INT:icmp_seq_num}(?:\(%{DATA:fwuser}\))? gaddr %{IP:src_xlated_ip}/%{INT:icmp_code_xlated} laddr %{IP:src_ip}/%{INT:icmp_code}( \(%{DATA:user}\))?

While this is the Filebeat ingest pipeline pattern format:

"Teardown %{NOTSPACE:network.transport} connection for faddr (:?%{NOTCOLON:cisco.asa.source_interface}:)?%{IP:source.ip}/%{NUMBER:source.port:int} (:?%{NOTSPACE:cisco.asa.source_username} )?gaddr (:?%{NOTCOLON}:)?%{IP}/%{NUMBER} laddr (:?%{NOTCOLON:cisco.asa.destination_interface}:)?%{IP:destination.ip}/%{NUMBER:destination.port:int}(:? %{NOTSPACE:cisco.asa.destination_username})?%{GREEDYDATA}"

The grok pattern format use the destination fields for the foreign host (faddr) and the source fields for the local host (laddr), while filebeat invert them. There is more conflict between field names, but I trust Filebeat patterns more to resolve them.

I want to make sure which parser is right, but I also want to improve it. The grok pattern assign the global host (gaddr) to a field, but the Filebeat pattern doesn't. I think that the gaddr holds a mapped address, but I don't know if they are more related to the source or the destination.

I'm suggesting a deeper parsing of the cisco patterns.

Below is my own grok pattern so far for the event, with more details:

The IP pattern is replaced by IPORHOST to also catch host type addresses, which will be copied from *.address to *.ip or *.domain field, like specified in the source field, or destination/client/server
Optional network.direction added to cover the event ASA-6-302020
The nested field cisco.cisco_type.* is a temporary field name because the pattern is also used for Cisco FWSM events, which are most of the time the same. Renaming cisco_type to asa or fwsm with an other filter is planned.
cisco.cisco_type.icmp_type and cisco.cisco_type.icmp_code optional fields are added
I dropped the destination_username because it's the same as the source_username but with less info

the gaddr part is still missing, I'm waiting for any suggestion

CISCOFWECS302020_302021 %{CISCO_ACTION:[event][outcome]}(?: %{CISCO_DIRECTION:[network][direction]})? %{WORD:[network][transport]} connection for faddr %{IPORHOST:[source][address]}/%{INT:[source][port]}(?:\(%{DATA:[cisco][cisco_type][source_username]}\))? gaddr %{IPORHOST}/%{INT} laddr %{IPORHOST:[destination][address]}/%{INT:[destination][port]}(?: type %{INT:[cisco][cisco_type][icmp_type]} code %{INT:[cisco][cisco_type][icmp_code]})?(?: \(%{DATA}\))?

The pattern can parse any of those logs:

ASA-6-302020: Built inbound ICMP connection for faddr 192.12.34.56/1(LOCAL\RAGRI8) gaddr 172.11.22.33/0 laddr 172.11.22.33/0 (RAGRI8)
ASA-6-302020: Built outbound ICMP connection for faddr 192.12.34.56/0 gaddr 10.0.0.0/10160 laddr 10.0.0.0/10160 type 8 code 0
ASA-6-302021: Teardown ICMP connection for faddr 10.0.0.0/10005 gaddr Prod-host.name.addr/0 laddr Prod-host.name.addr/0
FWSM-6-302021: Teardown ICMP connection for faddr 172.0.0.0/123 gaddr 172.0.1.0/123 laddr 172.0.1.0/0

If this works well, I'll suggest more patterns that I've written based on the logstash firewall patterns or created from scratch.

philippkahr commented 5 years ago

Hi,

I am also running filebeat 7.3.0 with the Cisco Plugin for ASA enabled. I also receive the following parsing errors.

Aug 16 09:09:17 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:17.747+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:20 myASA.firewall.com : %ASA-6-302016: Teardown UDP connection 8612565 for mylan:192.168.8.11/53 to VLAN1000:10.251.51.2/58451 duration 0:00:00 bytes 225\n"}
Aug 16 09:09:17 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:17.747+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:20 myASA.firewall.com : %ASA-6-302016: Teardown UDP connection 5488953 for mylan:192.168.8.11/53 to VLAN1000:10.251.51.2/47053 duration 0:00:00 bytes 0\n"}
Aug 16 09:09:17 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:17.754+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:20 myASA.firewall.com : %ASA-6-302016: Teardown UDP connection 5488954 for mylan:192.168.8.11/53 to VLAN1000:10.251.51.2/58451 duration 0:00:00 bytes 0\n"}
Aug 16 09:09:17 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:17.969+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<161>Aug 16 2019 09:09:20 myASA.firewall.com : %ASA-1-106021: Deny SCPS reverse path check from 10.251.1.249 to 10.251.1.250 on interface mylan\n"}
Aug 16 09:09:18 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:18.624+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:21 myASA.firewall.com : %ASA-6-302013: Built inbound TCP connection 8612566 for mylan:10.250.1.103/65079 (10.250.1.103/65079) to VLAN1002:10.251.52.2/20833 (10.251.52.2/20833)\n"}
Aug 16 09:09:18 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:18.627+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:21 myASA.firewall.com : %ASA-6-302014: Teardown TCP connection 8612566 for mylan:10.250.1.103/65079 to VLAN1002:10.251.52.2/20833 duration 0:00:00 bytes 0 TCP FINs from mylan\n"}
Aug 16 09:09:18 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:18.844+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:21 myASA.firewall.com : %ASA-6-302014: Teardown TCP connection 5488955 for mylan:10.250.1.103/65079 to VLAN1002:10.251.52.2/20833 duration 0:00:00 bytes 0 Failover primary closed\n"}
Aug 16 09:09:22 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:22.394+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:25 myASA.firewall.com : %ASA-6-302013: Built inbound TCP connection 8612567 for mylan:10.250.1.103/46483 (10.250.1.103/46483) to VLAN1002:10.251.52.1/20733 (10.251.52.1/20733)\n"}
Aug 16 09:09:22 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:22.395+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:25 myASA.firewall.com : %ASA-6-302014: Teardown TCP connection 8612567 for mylan:10.250.1.103/46483 to VLAN1002:10.251.52.1/20733 duration 0:00:00 bytes 0 TCP Reset-I from VLAN1002\n"}
Aug 16 09:09:22 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:22.616+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:25 myASA.firewall.com : %ASA-6-302014: Teardown TCP connection 5488956 for mylan:10.250.1.103/46483 to VLAN1002:10.251.52.1/20733 duration 0:00:00 bytes 0 Failover primary closed\n"}
Aug 16 09:09:22 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:22.945+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:25 myASA.firewall.com : %ASA-6-302014: Teardown TCP connection 8612554 for mylan:192.168.8.215/40553 to VLAN1000:10.251.51.2/80 duration 0:00:09 bytes 42184 TCP FINs from VLAN1000\n"}
Aug 16 09:09:22 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:22.969+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<161>Aug 16 2019 09:09:25 myASA.firewall.com : %ASA-1-106021: Deny SCPS reverse path check from 10.251.1.249 to 10.251.1.250 on interface mylan\n"}
Aug 16 09:09:23 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:23.625+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:26 myASA.firewall.com : %ASA-6-302013: Built inbound TCP connection 8612568 for mylan:10.250.1.103/54926 (10.250.1.103/54926) to VLAN1002:10.251.52.2/20833 (10.251.52.2/20833)\n"}
Aug 16 09:09:23 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:23.629+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:26 myASA.firewall.com : %ASA-6-302014: Teardown TCP connection 8612568 for mylan:10.250.1.103/54926 to VLAN1002:10.251.52.2/20833 duration 0:00:00 bytes 0 TCP Reset-I from VLAN1002\n"}
Aug 16 09:09:23 myFilebeat.hostname.com filebeat[6258]: 2019-08-16T09:09:23.856+0200        ERROR        [syslog]        syslog/input.go:132        can't parse event as syslog rfc3164        {"message": "<166>Aug 16 2019 09:09:26 myASA.firewall.com : %ASA-6-302014: Teardown TCP connection 5488957 for mylan:10.250.1.103/54926 to VLAN1002:10.251.52.2/20833 duration 0:00:00 bytes 0 Failover primary closed}

ragri8 commented 5 years ago

Except the event ASA-6-302013 because right now it's not part of the current ingest pipeline rules, your other events follow these patterns, so your problem isn't related.

elasticmachine commented 5 years ago

Pinging @elastic/siem

andrewkroh commented 5 years ago

the gaddr part is still missing, I'm waiting for any suggestion

Since the time when the module was first created ECS has added source.nat.ip and destination.nat.ip. So I think we can update the pipeline to make use of these new fields.

andrewkroh commented 5 years ago

@philippkahr That's a different problem which lies in the syslog input. I think it's this issue: https://github.com/elastic/beats/issues/6872.

ragri8 commented 5 years ago

I found fields already related to IP mapping in cisco module:

cisco.asa.mapped_source_ip
cisco.asa.mapped_source_port
cisco.asa.mapped_destination_ip
cisco.asa.mapped_destination_port

Those fields are mostly used for events between 338001 and 338204.

Fields like cisco.asa.mapped_source_ip and source.nat.ip aren't like the same? If so, shouldn't we stick to only one? And if not, could anyone explain to me what's the difference?

adriansr commented 5 years ago

Hi @ragri8

Thanks for raising this issue. I'm working on updating the pattern for 302021. We don't have patterns for 302020 or similar because we decided to focus on flows expirations which gave more information.

Currently I have this output for the line your shared:

{
    "@timestamp": "2019-10-10T10:21:36.000Z",
    "cisco.asa.icmp_code": 0,
    "cisco.asa.mapped_source_ip": "10.0.55.66",
    "cisco.asa.message_id": "302021",
    "destination.domain": "target.destination.hostname.local",
    "event.action": "flow-expiration",
    "event.code": 302021,
    "event.dataset": "cisco.asa",
    "event.module": "cisco",
    "event.original": "%ASA-6-302021: Teardown ICMP connection for faddr target.destination.hostname.local/10005 gaddr 10.0.55.66/0 laddr Prod-host.name.addr/0",
    "event.severity": 6,
    "event.timezone": "+00:00",
    "fileset.name": "asa",
    "host.hostname": "localhost",
    "input.type": "log",
    "log.level": "informational",
    "log.offset": 0,
    "network.iana_number": 1,
    "network.transport": "icmp",
    "service.type": "cisco",
    "source.domain": "Prod-host.name.addr",
    "source.nat.ip": "10.0.55.66",
    "tags": [
        "cisco-asa"
    ]
}

A few changes from your suggestion:

Not storing icmp_seq_num as we don't have a field for it. Is it really useful for anything?
Not storing translated icmp type. Same reason as above.
Set faddr to a domain to test that destination.domain would be populated.

As per your question, the cisco.asa.mapped_* fields were created before ECS added NAT fields. With 7.4, both fields will be populated, to maintain backwards compatibility, as you can see in the above event.

Edit: Here's the PR. Can you have a look @ragri8 ?

KevSex commented 4 years ago

@ragri8

Are you able to share your ECS formatted grok patterns for Cisco ASAs?

elastic / beats

Cisco incomplete patterns #13259