Add "X-Forwarded-For" field support for IIS module in Filebeat

[blarsonrp]

Describe the enhancement: Currently the "X-Forwarded-For" field is not able to be used with the IIS module with Filebeat.

Describe a specific use case for the enhancement or feature: Any HTTP request served by a CDN, reverse proxy, or a load balancer will see the IP of the device forwarding the request as the remote IP. The actual requesting IP is not logged unless you enable custom logging and include the "X-Forwarded-For" header.

Without this header being logged, searchable, and aggregatable there is no good way to view the originating IP and this severely limits the usefulness of having these logs. Thank you!

[elasticmachine]

Pinging @elastic/infrastructure

[veccsolutions]

X-Forwarded-For is not a standard IIS header, I modified my grok expressions to include it (and a number of other IIS log fields), updated the pipeline in elastic search and it worked. I wrote it all up in the following 2 posts. https://www.frakkingsweet.com/x-forwarded-for-in-iis-and-elk/ and https://www.frakkingsweet.com/filebeat-and-iis-with-multiple-sites/

The grok expressions in those are for 6.5.2, now that 7.2 is out, there was a number of changes in the fields. I log every field in IIS, including the custom X-Forwarded-For, except for the cookie: %{TIMESTAMP_ISO8601:iis.access.time} %{NOTSPACE:iis.access.site_name} %{NOTSPACE:iis.access.server_name} %{IPORHOST:destination.address} %{WORD:http.request.method} %{NOTSPACE:url.path} %{NOTSPACE:url.query} %{NUMBER:destination.port:long} %{NOTSPACE:user.name} %{IPORHOST:proxy.address} HTTP/%{NUMBER:http.version} %{NOTSPACE:user_agent.original} %{NOTSPACE:http.request.referrer} %{NOTSPACE:destination.domain} %{NUMBER:http.response.status_code:long} %{NUMBER:iis.access.sub_status:long} %{NUMBER:iis.access.win32_status:long} %{NUMBER:http.response.body.bytes:long} %{NUMBER:http.request.body.bytes:long} %{NUMBER:temp.duration:long} %{IPORHOST:source.address}

This one also fixed the issue when URL's had a bunch of random (but valid) characters in them by changing the url.path to use NOTSPACE.

Hope it helps.

[botelastic[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jeffrysleddens]

This issue is still very relevant. We also have lots of IIS webservers behind loadbalancers, so the X-Forwarded-For field is very valuable.

[epsilon-jpage]

This is absolutely ridiculous the X-Forwarded-For field is not supported in IIS. I need to use grok patterns in Logstash because this isn't there and it's essentially double work every time I update my agents.