[SIEM] [Maps] Absence of source or destination bytes prevents line layer from drawing

[spong]

There has been feedback that the line layer on the SIEM Network Map will not draw when either source.bytes or destination.bytes is not present (same will go for client/server bytes for APM data). These fields are used within the tooltip for the line, and are specified as metrics when we generate the map config.

Example error / Example Tooltip

As @andrewkroh pointed out:

In unidirectional flows there is never a destination.bytes value. The host doing the sending is always the source.

So long as there is data for both source.geo.location and destination.geo.location, the line layer should be visible, and we can just gracefully degrade the content in the tooltip based on whatever fields are available.

Relevant slack conversation here.

@nreese -- We should be able fix this on the SIEM side by introspecting each index pattern to ensure our metric fields are present before generating the map config, but wanted to get your thoughts on if this would be better handled at the map layer.

cc @jasonslater2000

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[spong]

@MadameSheema this should still be relevant as of the most recent 7.9.0-snapshot.

[jpittiglio]

We recently encountered this issue in a 7.12 deployment, and confirmed it still impacts 7.17. While I wasn't able to review the linked Slack conversation, I did want to post a few quick notes around what we found. Note screenshots were generated from a quick 7.12.1 deployment with a simple document using a basic GeoIP ingest processor for both source and destination:

POST sectest-1/_doc?pipeline=geolookup
{
  "@timestamp": "2022-05-19T12:00:00.000Z",
  "ecs": {
    "version": "1.0.0"
  },
  "source": {
    "ip": "1.2.3.4"
  },
  "destination": {
    "ip": "9.8.7.6"
  }
}

In particular, the issue originated from the fact that the index template in question only had source.geo.x, source.ip, destination.geo.x, and destination.ip present, as they were the only fields being populated by the original producer. This would render the source and destination points on the map, but not the line, as described in the original issue report - e.g.:

However, simply updating the index mapping with the source.bytes and destination.bytes fields was sufficient to get the line to render - note the documents themselves did not need to be updated with these values, only that these were in the mapping. Doing this presented the expected line:

While not directly related to this issue, a similar issue was still present when hovering over the source/destination, resulting in an error message - e.g:

Looking at what was being referenced, by simply adding source.as.x, source.domain, destination.as.x, destination.domain and host.name to the mappings, the expected window would display - e.g.:

Not saying this is necessarily resolved, but hope this can act as a workaround for anyone that similarly stumbles across this!