[Fortinet Fortigate Traffic]: Wrong calculation of network.bytes

Integration Name

Fortinet FortiGate Firewall Logs [fortinet_fortigate]

Dataset Name

fortinet-fortigate.log

Integration Version

1.25.5

Agent Version

8.15.0

Agent Output Type

elasticsearch

Elasticsearch Version

8.15.0

OS Version and Architecture

Ubuntu 22.04

Software/API Version

7.x

Error Message

Hi @all!

We investigated some issue in counting / sum some traffic data over the fields sentbyte and rcvdbyte. We investigated that the fields are more statistics than metrics. The real metrics are written to the fields sentdelta and rcvddelta, which are more usable than the *bytes fields.

Event Original

timestamp="1724322947" devname="myfgt" devid="FG5H0E5899999999" vd="root" date="2024-08-22" time="12:35:47" eventtime="1724322947595266411" tz="+0200" logid="0000000013" type="traffic" subtype="forward" level="notice" srcip="10.100.208.12" srcport="51396" srcintf="srv" srcintfrole="dmz" dstip="10.200.100.112" dstport="47106" dstintf="lan" dstintfrole="lan" srccountry="Reserved" dstcountry="Reserved" sessionid="1660293071" proto="6" action="success" policyid="15" policytype="policy" poluuid="476097ca-7fe5-51ee-1867-9a120be36b89" policyname="push-to-client" service="tcp/47106" trandisp="noop" duration="5" sentbyte="487" rcvdbyte="310" sentpkt="9" rcvdpkt="1" appcat="unscanned" dsthwvendor="Brocade" masterdstmac="ff:ff:ff:ff:ff:aa" dstmac="ff:ff:ff:ff:ff:ff" dstserver="1"

What did you do?

It a snipped from agent-policy

id: udp-fortinet_fortigate-86972f9d-2345-4867-9dcd-ae0b8f5eb9ae name: test-fgt-port-9004 revision: 5 type: udp use_output: default meta: package: name: fortinet_fortigate version: 1.25.5 data_stream: namespace: test package_policy_id: 86972f9d-2345-4867-9dcd-ae0b8f5eb9ae streams:
- id: udp-fortinet_fortigate.log-86972f9d-2345-4867-9dcd-ae0b8f5eb9ae data_stream: dataset: fortinet_fortigate.log type: logs host: '0.0.0.0:9004' tags:
  - preserve_original_event
  - fortinet-fortigate
  - fortinet-firewall
  - forwarded publisher_pipeline.disable_host: true processors:
  - add_fields: target: _temp fields: internal_networks:
    - private

What did you see?

Hte prob

What did you expect to see?

The calculated sum in network.bytes (sendbyte (=source.bytes) + rcvdbyte (=(destination.bytes)) which is done over the ingest pipeline logs-fortinet_fortigate.log-1.25.5 and a script processor:

ctx.network.bytes = ctx.source.bytes + ctx.destination.bytes

is using only sendbyte and rcvdbyte, but should use under conditions senddelta and rcvddelta instead of the byte fields.

Maybe something like this:

[ { "convert": { "field": "fortinet.firewall.sentdelta", "type": "long", "if": "ctx?.fortinet?.firewall?.sentdelta != null", "ignore_failure": true } }, { "convert": { "field": "fortinet.firewall.rcvddelta", "type": "long", "if": "ctx?.fortinet?.firewall?.rcvddelta != null", "ignore_failure": true } }, { "script": { "lang": "painless", "source": "ctx.network.bytesdelta = ctx.fortinet.firewall.sentdelta + ctx.fortinet.firewall.rcvddelta", "if": "ctx?.fortinet?.firewall?.sentdelta != null && ctx?.fortinet?.firewall?.rcvddelta != null", "ignore_failure": true } }, ]

This is just a workaround over the custom pipeline from the integration. Is it possible to integrate this somehow?

Anything else?

No response

elastic / integrations