logstash-plugins / logstash-input-beats

Apache License 2.0
87 stars 81 forks source link

Include peer IP address and port information in beats events #117

Open magnusbaeck opened 8 years ago

magnusbaeck commented 8 years ago

The tcp and udp inputs provide peer IP address and port information in the host and port fields, but this is missing from events emitted by the beats input. This is obviously a limitation if you want to save the client’s IP address as a field but also if you want the client’s FQDN. The hostname in the host field comes from [beat][hostname] field which typically wouldn’t be a FQDN. With the IP address available you’ve got a reasonable chance to obtain the FQDN by a reverse lookup of the IP address (using the dns filter).

Right now I’m working around the FQDN part by changing the Filebeat configuration file to include host as a custom field with the FQDN as value. I’m obtaining the FQDN value from Ansible so there’s no maintenance burden, but it’d be nice to not have to do this. Obtaining the IP address in the same manner would be harder but probably has rough corner cases for hosts with multiple network interfaces.

How about creating [beat][ip] and [beat][port] to complement [beat][hostname]? This might sound like a potential Beats feature, but I suspect it’s easier to implement on the input side since the IP address and port would be tied to the specific output in Beats (and wouldn’t make sense at all for e.g. file outputs).

ph commented 8 years ago

@magnusbaeck It totally make sense for me to add these in the input. +1

ruflin commented 8 years ago

@urso Could we have this as potential meta data that is added by "some" outputs?

jeanfabrice commented 7 years ago

I'm very interesting by this feature too. i.e having the host field populated with the real source ip of the beats tcp connection (like it's done in the logstash-input-tcp plugin) instead of a useless and craftable beat.hostname.

Could this be an option in the plugin configuration to avoid breaking existing setup ?

jeanfabrice commented 7 years ago

related to #180