EnvoyProxy Integration

[SubhrataK]

This issue is to migrate the EnvoyProxy module from beats to agent-based integration.

[tehbooom]

Started development of this integration. For metrics will be following a similar model as the Airflow package.

PR to follow.

[tehbooom]

Metrics currently a blocker per

https://github.com/elastic/beats/issues/39473

[ritalwar]

Just a quick question: I see we already have an EnvoyProxy Beats module, and we just want to create an agent-based integration for it. Why are we planning to use StatsD instead of using the existing EnvoyProxy Beats module for creating this integration?

[tehbooom]

@ritalwar The way our module was collecting the logs was simply doing a request to /stats with maybe 20 or so fields.

Envoy does support Statsd and states it's typically how the metrics are collected.

I did try to use the metric beat module but I noticed I was missing a lot of fields and they were in "Statsd format". So I thought it might be best to simply redo the metrics using Statsd or recreate the module using CEL input.

I think Statsd is the way to forward here as it's what envoy is supporting and other applications are using Statsd as an input like DataDog.

[ritalwar]

If statsd seems like the best way forward based on your analysis, let's go with that, although we can update this schema to include more/missing fields if necessary.

[tehbooom]

I agree that we could update that schema however some of the fields are already histograms like in statsd. I think that would be more work in the long run to basically recreate our statsd module to parse out "statsd like" fields.

[ritalwar]

I think that would be more work in the long run to basically recreate our statsd module to parse out "statsd like" fields.

Agree! let's modify the existing one(statsd) for this integration unless there's a better solution suggested.