[Meta] Filebeat NetFlow input

[adriansr]

Filebeat NetFlow input release checklist

This checklist is intended to track progress of NetFlow support in Filebeat (#8434).

For this input and associated modules to go GA, the following criterias should be met:

• [x] Filebeat netflow input. (#9365)
• [x] Documentation for netflow input. (#9388)
• [x] Community ID support. (#9538)
• [x] Fields yml updated
• [x] Fields follow ECS and naming conventions (#9537)
• [x] Add BiFlow support (#9537)
• [x] Dashboards exists (#10180)

[elasticmachine]

Pinging @elastic/secops

[adriansr]

ECS fields for Biflow support: Non-Biflow flows: event.kind: event event.category: network_traffic event.action: netflow_flow

Biflow flows: event.kind: event event.category: network_session event.action: netflow_flow

Netflow options: event.kind: event event.category: network_traffic event.action: netflow_options

https://tools.ietf.org/html/rfc5103#section-6.3 shows how we would map source.* and destination.* to client.* and server.* Information Element Field = 0x00, 0x01, 0x03 source.* -> copy_to -> client.* destination.* -> copy_to -> server.*

Information Element Field = 0x02 source.* -> copy_to -> server.* destination.* -> copy_to -> client.*

( @MikePaquette suggestion )

[adriansr]

6.6 Release checklist

• [x] Merge #9537
• [x] Decide what to do about ECS flow. ( See this comment)
  • Andrew opened https://github.com/elastic/ecs/pull/288 for the *.locality fields.
  • flow.id can be removed later while in beta, but it's kind of useful for visualizing to have a single field with the flow ID even if it's unique to an application (e.g. packetbeat flow IDs can't correlate with Suricata or Bro, but they all have their own ID concept).
• [x] Rebase & submit PR adriansr/beats:netflow-ecs. This has the above changes and some other last-minute ECS changes. If flow is to be removed, it can be done in this PR. #9609
• [x] CHANGELOG entry. #9609
• [x] Merge #9609
• [x] Add netflow fields to binary. #9686

Backports to 6.x / 6.6. Merge in given order

• [x] Backport #9365 #9584
• [x] Backport #9538 merge #9614
• [x] Backport #9537 // Replace usage of source with ecs_source // ---> #9646
• [x] (docs) Backport #9544 #9585
• [x] (docs) Update elastic/docs to look for docs in x-pack/filebeat/docs for 6.x / 6.6 branch. Same as https://github.com/elastic/docs/pull/507 (@dedemorton can assist). ---> #https://github.com/elastic/docs/pull/511
• [x] (docs) Backport #9567 #9586
• [x] Backport ECS commit from above. (6.x) #9660
• [x] Backport ECS commit from above. (6.6) #9677
• [x] Backport 6.x [Add netflow fields to binary. #9686]
• [x] Backport 6.6 [Add netflow fields to binary. #9686]