[fortinet_fortigate] fortinet.firewall.mac does not respect ECS via RFC 7042

[zez3]

Hello Elastics,

it seems that the fortinet_fortigate integration does not respect ECS for fortinet.firewall.mac fields

Can someone please correct this? Thank you

Also an side note: There is no related.mac in ECS https://www.elastic.co/guide/en/ecs/current/ecs-related.html

Perhaps there should be one added?

[zez3]

same thing for netflow integration e.g. netflow.post_destination_mac_address | 00:00:5e:00:00:00 netflow.source_mac_address | 00:10:f3:a2:00:00 source.mac | 00:10:f3:a2:00:00

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[andrewkroh]

@zez3 Please provide version information. What version of Elastic Agent? What version of the Fortinet integration package? What version of the netflow integration package?

[zez3]

Hey @andrewkroh I am always on the latest

Fortigate v1.2.3 Netflow 2.2.4

[andrewkroh]

Elastic Agent 8.5 will contain https://github.com/elastic/beats/pull/32622 which should address fields produced by the Filebeat netflow input.

[andrewkroh]

I don't see anything in the pipeline for the fortinet_fortigate.log that would format the MACs. So we'll need to add some gsub processors to format the macs.

Also we don't have any log samples that contain MACs. Would you be able to provide a sample of the event.original field (produced when the "Preserve original event" option is enabled)? We would add this to our test data for the package in https://github.com/elastic/integrations/tree/91300870e0a228e913583561950b9637960628fa/packages/fortinet_fortigate/data_stream/log/_dev/test/pipeline. If you can provide a sample, can you please indicate the fortigate version that it came from so we have know the lineage of the data?

Have samples should help us automatically detect if we drift from the RFC 7042 format.

[zez3]

I don't have the event.orig but I could get it. The macs are coming from OSPF neighbor changes

Click here to see an example

{
  "_index": ".ds-logs-fortinet_fortigate.log-ece_forti-2022.10.05-000019",
  "_id": "bXQCr4MBF-Kn07JyfISr",
  "_version": 1,
  "_score": 0,
  "_source": {
    "agent": {
      "name": "myhost.my.dom",
      "id": "4ba5a4aa-848d-4435-a13e-ace9584cddaa",
      "ephemeral_id": "4e84e4bd-7cdb-4010-a316-8f7929ab4012",
      "type": "filebeat",
      "version": "8.4.2"
    },
    "log": {
      "level": "information",
      "source": {
        "address": "1.1.1.1:23762"
      },
      "syslog": {
        "severity": {
          "code": 6
        },
        "priority": 190,
        "facility": {
          "code": 23
        }
      }
    },
    "elastic_agent": {
      "id": "4ba5a4aa-848d-4435-a13e-ace9584cddaa",
      "version": "8.4.2",
      "snapshot": false
    },
    "rule": {
      "description": "Neighbor table changed"
    },
    "source": {
      "geo": {
        "continent_name": "Europe",
        "region_iso_code": "MY-MY",
        "city_name": "mycity",
        "country_iso_code": "MY",
        "country_name": "myland",
        "region_name": "mycity",
        "location": {
          "lon": 7.4651,
          "lat": 46.9447
        }
      },
      "as": {
        "number": 559,
        "organization": {
          "name": "SWITCH"
        }
      },
      "ip": "1.1.1.1"
    },
    "message": "MAC address 00:0C:29:FF:FF:FF is added to neighbor table",
    "tags": [
      "fortinet-fortigate",
      "fortinet-firewall",
      "forwarded"
    ],
    "network": {
      "protocol": "kernel"
    },
    "input": {
      "type": "udp"
    },
    "observer": {
      "ingress": {
        "interface": {
          "name": "OUT_777"
        }
      },
      "product": "Fortigate",
      "vendor": "Fortinet",
      "name": "UNIFW77",
      "serial_number": "myserial",
      "type": "firewall"
    },
    "@timestamp": "2022-10-06T22:35:14.000+02:00",
    "ecs": {
      "version": "8.3.0"
    },
    "related": {
      "ip": [
        "1.1.1.1"
      ]
    },
    "data_stream": {
      "namespace": "ece_forti",
      "type": "logs",
      "dataset": "fortinet_fortigate.log"
    },
    "fortinet": {
      "firewall": {
        "subtype": "router",
        "action": "add",
        "type": "event",
        "vd": "UNIVD",
        "mac": "00:0C:29:FF:FF:FF"
      }
    },
    "event": {
      "agent_id_status": "verified",
      "ingested": "2022-10-06T20:35:19Z",
      "code": "0103051000",
      "timezone": "+0200",
      "kind": "event",
      "start": "2022-10-06T22:35:14.211+02:00",
      "dataset": "fortinet_fortigate.log"
    }
  },
  "fields": {
    "fortinet.firewall.mac": [
      "00:0C:29:FF:FF:FF"
    ],
    "elastic_agent.version": [
      "8.4.2"
    ],
    "observer.ingress.interface.name": [
      "OUT_777"
    ],
    "observer.vendor": [
      "Fortinet"
    ],
    "source.geo.region_name": [
      "mycity"
    ],
    "log.level": [
      "information"
    ],
    "source.ip": [
      "11.1.1"
    ],
    "agent.name": [
      "myhost.my.dom"
    ],
    "event.agent_id_status": [
      "verified"
    ],
    "source.geo.region_iso_code": [
      "MY-CT"
    ],
    "event.kind": [
      "event"
    ],
    "source.geo.city_name": [
      "myCity"
    ],
    "log.syslog.severity.code": [
      6
    ],
    "input.type": [
      "udp"
    ],
    "rule.description": [
      "Neighbor table changed"
    ],
    "data_stream.type": [
      "logs"
    ],
    "observer.serial_number": [
      "myserial"
    ],
    "tags": [
      "fortinet-fortigate",
      "fortinet-firewall",
      "forwarded"
    ],
    "fortinet.firewall.type": [
      "event"
    ],
    "event.code": [
      "0103051000"
    ],
    "agent.id": [
      "4ba5a4aa-848d-4435-a13e-ace9584cddaa"
    ],
    "observer.type": [
      "firewall"
    ],
    "ecs.version": [
      "8.3.0"
    ],
    "log.source.address": [
      "1.1.1.1:23762"
    ],
    "agent.version": [
      "8.4.2"
    ],
    "event.start": [
      "2022-10-06T20:35:14.211Z"
    ],
    "fortinet.firewall.subtype": [
      "router"
    ],
    "fortinet.firewall.vd": [
      "UNIVD"
    ],
    "source.as.number": [
      559
    ],
    "observer.name": [
      "UNIFW12"
    ],
    "source.geo.location": [
      {
        "coordinates": [
          7.4651,
          46.9447
        ],
        "type": "Point"
      }
    ],
    "agent.type": [
      "filebeat"
    ],
    "network.protocol": [
      "kernel"
    ],
    "event.module": [
      "fortinet"
    ],
    "related.ip": [
      "1.1.1.1"
    ],
    "source.geo.country_iso_code": [
      "CH"
    ],
    "observer.product": [
      "Fortigate"
    ],
    "elastic_agent.snapshot": [
      false
    ],
    "log.syslog.priority": [
      190
    ],
    "event.timezone": [
      "+0200"
    ],
    "source.as.organization.name.text": [
      "SWITCH"
    ],
    "elastic_agent.id": [
      "4ba5a4aa-848d-4435-a13e-ace9584cddaa"
    ],
    "data_stream.namespace": [
      "ece_forti"
    ],
    "fortinet.firewall.action": [
      "add"
    ],
    "source.as.organization.name": [
      "SWITCH"
    ],
    "source.geo.continent_name": [
      "Europe"
    ],
    "message": [
      "MAC address 00:0C:29:FF:FF:FF is added to neighbor table"
    ],
    "event.ingested": [
      "2022-10-06T20:35:19.000Z"
    ],
    "@timestamp": [
      "2022-10-06T20:35:14.000Z"
    ],
    "data_stream.dataset": [
      "fortinet_fortigate.log"
    ],
    "agent.ephemeral_id": [
      "4e84e4bd-7cdb-4010-a316-8f7929ab4012"
    ],
    "source.geo.country_name": [
      "Switzerland"
    ],
    "log.syslog.facility.code": [
      23
    ],
    "event.dataset": [
      "fortinet_fortigate.log"
    ]
  }
}            

[zez3]

https://docs.fortinet.com/document/fortigate/7.2.2/fortios-log-message-reference/51000/51000-log-id-nb-tbl-chg

"event.code": [ "0103051000" ],

[andrewkroh]

I don't have the event.orig but I could get it.

@zez3 If you could get the event.original that would be really helpful in validating #4426. Thanks!

[zez3]

@andrewkroh

<190>date=2022-12-15 time=14:48:08 devname="FirewallName" devid="ID_of_my_Device" eventtime=1671112088064754895 tz="+0100" logid="0103051000" type="event" subtype="router" level="information" vd="myvdom" logdesc="Neighbor table changed" service="kernel" action="delete" mac="48:9E:BD:11:11:11" src_int="ZB_VPN_4" srcip=fe80::34d3:4160:e502:cf44 msg="MAC address 48:9E:BD:11:11:1 is deleted from neighbor table"

does this help? but I think we don't need to do this anymore since Fortinet has changed the MAC format and it respect the RFC now.

[botelastic[bot]]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[zez3]

The issue was fixed @source Fortinet corrected the MAC format