search

elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
111 stars 4.93k forks source link

Support wildcard to "include_body_for" to include body for all content types #34528

Open alaa-mallah opened 1 year ago

alaa-mallah commented 1 year ago

Describe the enhancement: User wants the ability to include body for all content types by specifying wildcard in the include_body_for configuration parameters in http protocols

Describe a specific use case for the enhancement or feature: User uses packetbeat to ingest HTTP traffic, right now they are specifying specific content types to be included in the response and request body, however they want to be able to include body for all content types by having the ability to add wildcard to include_body_for in packetbeat configuration in the http protocol section Ex: include_body_for: ["*"]

elasticmachine commented 1 year ago

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

efd6 commented 1 year ago

This looks like it should currently be possible given the way that the assessment for whether to retain the body is determined. The code for this is here.

The way that it's done can be abused to either include all valid mime-type specs (which must contain a /) (playground) or all types irrespective of whether the spec is valid of not (since all strings contain ""). Subclasses of mimetypes can be specified by e.g. "application/" (all application type) or "/jpeg" ("video/jpeg" and "image/jpeg")

The two wild-cards would be expressed in a config as include_body_for: ["/"] and include_body_for: [""]. I think that the empty string case will not work for the integration, but the slash should.

A demonstration of this using code based on the packetbeat protos code is here.

elasticmachine commented 10 months ago

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)