[filebeat][websocket] - Added runtime URL modification support based on state and cursor values

ShourieG commented 5 days ago

Type of change

Enhancement

Proposed commit message

Till now it was not possible to modify the Websocket URL dynamically via the CEL program as the Websocket logic exists outside the scope of the CEL engine instance. To make this action possible a new instance of the CEL engine was added as a one time execution process to prime the URL before the main CEL program executes. This new CEL instance shares the same state hence its possible to modify and use cursor and state values and in turn modify the URL based off of these values.

Checklist

[x] My code follows the style guidelines of this project
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation ~~- [] I have made corresponding change to the default configuration files~~
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

Author's Checklist

[ ]

How to test this PR locally

Related issues

Closes #39858

Use cases

Screenshots

Logs

mergify[bot] commented 5 days ago

This pull request does not have a backport label. If this is a bug or security fix, could you label this PR @ShourieG? 🙏. For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed branches, such as:

backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

elasticmachine commented 5 days ago

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

elasticmachine commented 4 days ago

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

pierrehilbert commented 4 days ago

Hey @ShourieG I added the necessary label to get your PR reviewed by our team. Could you please make the linter happy by excluding those lines from the linter check? As this is not real credential leak, we should make it aware of it.

ShourieG commented 4 days ago

Hey @ShourieG I added the necessary label to get your PR reviewed by our team. Could you please make the linter happy by excluding those lines from the linter check? As this is not real credential leak, we should make it aware of it.

Hey @pierrehilbert, I think your team got tagged due to an unintended update to the filebeat.yml file which I have reverted now. Fixed the linter btw :)

pierrehilbert commented 4 days ago

run docs-build

ShourieG commented 4 days ago

@efd6, I've addressed all the PR changes and removed the negative syntax tests. @belimawr, your concerns got addressed as the conditions were removed with the latest updates.

elastic / beats