Custom Logs Integration not processing logs through pipeline

[infosecwatchman]

Hello,

I'm trying to set up a number of custom log integration each with a pipeline to process the logs, and I successfully created appropriate pipelines to parse the logs, and have tested them via the simulate API. However, when the integrations are running, the pipelines are not running, or are not correctly processing the logs. I used the rollover API as well for the datastream, but no luck there either. Each of the referenced pipelines in the custom pipeline are essentially just grok patterns and date format processors.

[infosecwatchman]

I ended up using the "Custom Configurations" section of each of the integrations, as described in this post. Though, I would like to see way to configure via the custom pipeline, as that seems like a more stable and consistent approach.

[botelastic[bot]]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!