Packetbeat - Incorrect HTTP traffic reporting

[nul]

Hi all,

I'm running into a weird issue with Packetbeat whereby the HTTP traffic reported has incorrect details logged.

The setup is this: packetbeat > local logstash > remote redis installation > logstash > elasticsearch. Note that I have no processing of the data from packetbeat on any logstash instance, they're simply shippers. In addition, I'm using the same setup for topbeat and winlogbeat as well and have not ran into any issues.

Digging through data logged in ES (using Kibana) I ran into some weird data whereby, according to packetbeat data, requests for certain JS resources were redirected to image files. I've crosschecked the entries logged by IIS with the packetbeat entries and I can see that there's a mismatch in event data:

• packetbeat recorded data:
• request details:

• IIS recorded data:

  2016-03-28 13:48:44 GET /resources/SiteResources/My+Folder/static_pages/media_partnerships/slides/google.jpg - 80 - HTTP/1.1 Mozilla/5.0+(Windows+NT+6.1;+WOW64;+rv:45.0)+Gecko/20100101+Firefox/45.0 http://_____________________/eng/extraordinary-marketing 301 0

  2016-03-28 13:48:44 GET /eng/dynamic/dd70914a882447e2fd124b94648d6c17.js - 80 - HTTP/1.1 Mozilla/5.0+(Windows+NT+6.1)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/49.0.2623.87+Safari/537.36 http://_____________________/eng/sales/detail/180-l-697-mf5p5m/contemporary-estate-with-panoramic-sea-views-on-the-bay-of-golfe-juan-golfe-juan-pr-06220 200 0

The WEB server is doing redirects (301) for lowercasing so /resources/SiteResources/My+Folder/static_pages/media_partnerships/slides/google.jpg becomes /resources/siteresources/my+folder/static_pages/media_partnerships/slides/google.jpg which is correctly reported in the IIS logs as a 301 for the image call, with the correct referrer. The JS call is separately and correctly reported in IIS as a 200 status request also with the correct referrer.

However, it looks like packetbeat combined these two requests into a single request and took bits of each (i.e. the code, referrer, user-agent and the response location from one, and the path from the other). In addition, packetbeat never recorded the 200 status request for the JS.

Below are the configuration files for packetbeat setup:

• packetbeat config: packetbeat.txt
• logstash shipper (from packetbeat to redis): logstash_shipper_beats.txt
• logstash indexer (from redis to ES): logstash_indexer.txt

[botelastic[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic[bot]]

This issue doesn't have a Team:<team> label.