forwarded_ip - no place to put geo ... fields

[vbohata]

We have often original IP/X-Forwarded-for IP address in logs. Sometimes together with its geo information, sometimes we call geoip filter on it. Currently there is just network.forwarded_ip field which is very confusing and can not be used to store additional fields like geo. I think this should be replaced with something client.original., server.original.. The original field is the original (x-forwarded-for) IP address and can also store other fields as well (client.original.geo.*, ...). Also it is very clear to which it is related to - client.original is related to client, server.original to server.

[jeffrysleddens]

I agree. We also have a lot of logs with X-Forwarded-For and the ip address stored in network.forwarded_ip is much more interesting to run geoip on than the source.ip (which is our load balancer), but we have nowhere to store the geo information for network.forwarded_ip according to the ECS specs.

My preference would be to store the actual IP of the client in the source.ip field, it is after all the real IP address where the event/request originated from and store the forwarder ip in a field like network.forwarded_ip. And reading the following comment from @MikePaquette I think this is also how network.forwarded_ip should be used: https://github.com/elastic/ecs/pull/51#issuecomment-413033718 The description for network.forwarded_ip should really be improved as it is very confusing at the moment.

[ebeahan]

Thanks @vbohata @jeffrysleddens.

Better modeling of load balancers, proxies, and other intermediary devices has been a topic recently (e.g. https://github.com/elastic/ecs/issues/895#issuecomment-670200246), and evaluating better field nesting opportunities will definitely be part of that design discussion.

The `network.forwarded_ip' field description is something we can reevaluate in the near term to help remove some confusion. If you have any other more thoughts about how the field could be improved, feedback is always welcome. 😃

Also, in the ECS docs, there's upcoming work to include a place for examples which better demonstrate a field's intended usage. The aim is to provide better guidance around mapping fields and how those fields relate to each other in real-world usage.

[webmat]

I just opened the #938 meta-issue to address both proxies and the XFF/IP forwarding. Perhaps we should address network.forwarded_ip separately first.