Add `pipeline` in `@metadata` for logstash output

[nkakouros]

Describe the enhancement: The @metadata field is documented as containing:

"@metadata": { 
      "beat": "auditbeat", 
      "version": "7.6.1" 
    }

But, in Filebeat, if an input has a pipeline specified, it can also contain a pipeline key (source). This is mentioned to show that there is already limited support for what I am describing below.

In every Beat, you can configure a pipeline in the Elasticsearch output. But, there is no way to do so if you are using the Logstash output. For instance, if you want to apply some processing on all packetbeat logs using an ingest node (e.g. to change the timestamp), you can't use sth like:

output {
  if [agent][type] =~ "packetbeat" and [@metadata][pipeline] {
      elasticsearch {
        ...

since @metadata will not contain the pipeline key. One has to fallback to using more complex and fragile conditionals.

[elasticmachine]

Pinging @elastic/integrations-services (Team:Services)

[botelastic[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[nkakouros]

Still a valid request.

[botelastic[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[botelastic[bot]]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[legoguy1000]

definitely valid!!!!

[botelastic[bot]]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[legoguy1000]

still valid