[journald] Elastic Agent System Integration

[pierrehilbert]

Issue

Meta issue: https://github.com/elastic/beats/issues/37086 To have journald GA, we should switch the Elastic Agent System Integration to use journald to read log data streams. We will need to decide how to migrate existing log inputs to journald since:

• The auth datastream https://github.com/elastic/integrations/blob/main/packages/system/data_stream/auth/agent/stream/log.yml.hbs
• The syslog datastream https://github.com/elastic/integrations/blob/main/packages/system/data_stream/syslog/agent/stream/log.yml.hbs

In addition, we will need to define if we want to make this switch conditional on the Linux distribution specifically, or if a conditional being added only for Linux is enough.

Acceptance Criteria

• [ ] Auth datastream is available and use journald
• [ ] Syslog datastream is available and use journald

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[cmacknz]

I suspect we will have data duplication when we switch from filestream reading syslog to journald, because we will lose the last offset we read from the system log.

I haven't looked into if it is possible to avoid this, but I wanted to note it as something to follow up on as we do this migration.

[pierrehilbert]

That's a good point! I mentioned a test for the duplication here but this is not covering the same reason. Should we plan a path similar to what has been done for the Log Input to Filestream migration and try keep the current offset? I can't see an easy as this is not the same file this time....