[8.14](backport #39873) Add Offset to `libbeat/reader.Message`

mergify[bot] commented 1 week ago

Proposed commit message

libbeat: add Offset to libbeat/reader.Message

This commit introduces the Offset property to libbeat/reader.Message, which stores the total number of bytes read and discarded before generating the message. The Offset field allows inputs to accurately determine how much data has been read up to the message, calculated as Message.Bytes + Message.Offset.

With this new Offset field, the filestream input correctly updates its state to account for data read but discarded by the include_message parser.

Checklist

[x] My code follows the style guidelines of this project
[x] I have commented my code, particularly in hard-to-understand areas
~~[ ] I have made corresponding changes to the documentation~~ -~~ [ ] I have made corresponding change to the default configuration files~~
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

None

How to test this PR locally

Follow the instructions on #39653

Related issues

Closes #39653

This is an automatic backport of pull request #39873 done by Mergify.

mergify[bot] commented 1 week ago

Cherry-pick of 535a17418d5a59583ba6ea5ea7d13e0f693ed468 has failed:

On branch mergify/bp/8.14/pr-39873
Your branch is up to date with 'origin/8.14'.

You are currently cherry-picking commit 535a17418d.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
    modified:   CHANGELOG-developer.next.asciidoc
    modified:   CHANGELOG.next.asciidoc
    modified:   filebeat/input/filestream/input.go
    modified:   filebeat/input/filestream/parsers_integration_test.go
    modified:   libbeat/reader/filter/filter.go
    modified:   libbeat/reader/message.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
    both modified:   filebeat/input/filestream/environment_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

elasticmachine commented 6 days ago

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

elastic / beats