[Filebeat][AWS S3] file_selectors overwrite parsers configuration

[chemamartinez]

Kibana/Elasticsearch Stack version

8.14.3

Bug description

According to the input docs, global parsers can be overwritten by particular parsers for each file selector. However, this is not exactly how it works.

Currently, when there is any entry in file_selectors, global parsers are overwritten even though there is no parsers in that entry. If this is intentional I think the documentation should warn about it.

Steps to reproduce

Following the next config schema:

filebeat.inputs:
- type: aws-s3
  ...

  file_selectors:
    - regex: '/CloudTrail/'
    - regex: '/CloudTrail-Digest/'
  parsers:
    - multiline:
        pattern: "^<Event"
        negate:  true
        match:   after

You can check that the multiline is not applied as it is overwritten by parsers options set for each file selector (in this case empty so it is just ignored).

The right config to make it work would be:

filebeat.inputs:
- type: aws-s3
  ...

  file_selectors:
    - regex: '/CloudTrail/'
       parsers:
           - multiline:
                pattern: "^<Event"
                negate:  true
                match:   after
    - regex: '/CloudTrail-Digest/'
       parsers:
           - multiline:
                pattern: "^<Event"
                negate:  true
                match:   after

This can lead to confusions as the expected behaviour would be to overwrite the parsers settings only when it is defined for each file selector.

[botelastic[bot]]

This issue doesn't have a Team:<team> label.