Remove event.original removal processors from ingest pipelines

[Alphayeeeet]

It seems that most of the checked integrations have a removal processor for event.original (except if it contains the preserve tag) in their pipelines. In the other way, the .fleet-final-pipeline also has this processor and runs after all integration pipelines (incl. the @custom pipelines) have finished. If the integration has this removal, the @custom pipeline cannot parse the original event anymore as it is not present in the event anymore.

I would suggest removing the removal processors from all integration ingest pipelines and forward the removal functionality into the fleet-final-pipeline where it belongs. In that case @custom pipelines may parse the original event and can extract additional data or correct malformed parsing in the generic integrations (e.g. catalina.out in the Apache Tomcat integration):

[andrewkroh]

I think this is a good idea. I will occasional install a logs@custom pipeline that keeps the event.original when there is a pipeline failure so that I can debug it.

PUT _ingest/pipeline/logs@custom
{
  "processors": [
    {
      "append": {
        "field": "tags",
        "value": [
          "preserve_original_event"
        ],
        "allow_duplicates": false,
        "tag": "ctx.error?.message != null",
        "ignore_failure": true
      }
    }
  ]
}

And in order for this to work, I currently have to manually drop that remove processor from the managed pipelines.

[Alphayeeeet]

If that change would be approved, I maybe start working on this after my current PR has been merged.