Support of multiple journald blocks in the configuration.

Wazuh version	Component	Install type	Install method	Platform
4.9.0	Logcollectoy	Manager/Agent	Packages/Sources	OS version

Description

This issue is part of Stage 2B of our ongoing project to enhance Wazuh's Logcollector with the ability to monitor journald logs. The objective is to enable Logcollector to parse and apply settings from multiple journal configuration blocks within the ossec.conf file, aimed at collecting journald logs with detailed filtering options.

Requirements

Multiple Configuration Blocks Support: ossec.conf must support parsing multiple <localfile> blocks for journald logs.
Mandatory Attributes: Each block must specify location and log_format set to journald, identifying them as sources of journald logs.
Flexible Filtering: Blocks may include <filter> tags for selective log collection based on PCRE2 regex matching against log fields.
Logical Operations Between Blocks: Filters within the same block follow AND logic, while multiple blocks are evaluated in OR logic regarding log collection.
Support for Existing Functions: Blocks should also respect ignore, restrict, and only-future-events settings.

Examples of Configuration

<localfile>
  <location>journald</location>
  <log_format>journald</log_format>
  <filter field="_KERNEL_DEVICE">.+</filter>
</localfile>

<localfile>
  <location>journald</location>
  <log_format>journald</log_format>
  <filter field="_SYSTEMD_UNIT">^cron.service$</filter>
  <filter field="PRIORITY" ignore_if_missing="yes">[0-3]</filter>
</localfile>

<localfile>
  <location>journald</location>
  <log_format>journald</log_format>
  <filter field="MESSAGE_ID">^8d45620c1a4348dbb17410da57c60c66$</filter>
  <only-future-events>no</only-future-events>
</localfile>

These configurations demonstrate the flexibility of the proposed enhancement, allowing for precise control over which journald logs are collected based on various criteria.

Implementation Details

Enhance the configuration parser in Logcollector to recognize and handle multiple <localfile> blocks tailored for journald.
Implement filtering logic that applies PCRE2 regex for log field matching and respects the ignore_if_missing attribute.
Ensure logical operations within and between blocks function as intended.
Integrate with existing Logcollector functionalities like only-future-events.

Testing Criteria

Verify Logcollector correctly parses and applies settings from multiple journald-specific <localfile> blocks.
Test filtering accuracy, including the handling of the ignore_if_missing attribute.
Ensure AND logic within blocks and OR logic between blocks work as specified.
Confirm seamless integration with only-future-events.

Notes

This enhancement significantly broadens Logcollector's capabilities for journald log collection, offering users greater control and precision in their log monitoring setup.

Today's progress on the issue includes two main aspects:

In-depth Research on Logcollector's Operation and Configuration Loading Mechanisms:
- Conducted comprehensive research to understand how Logcollector currently operates, with a focus on its configuration loading process. This involved reviewing existing documentation, analyzing the source code, and exploring how Logcollector interacts with different types of log sources. The goal was to identify best practices and potential challenges in integrating the new journald log collection functionality.
Initiation of Configuration Loading Function Implementation:
- Began the development of the new configuration loading function that will enable Logcollector to parse and apply settings from multiple journald log source blocks specified in the ossec.conf file. This work is focused on ensuring the new function can accurately interpret the extended XML schema, which includes the mandatory attributes (location and log_format set to journald) and the optional <filter> tags with their complex criteria (including regular expressions and the ignore_missing attribute).

These steps are foundational for integrating journald log collection into Wazuh's Logcollector, ensuring it can handle the specified configurations effectively. Tomorrow, I plan to continue with the implementation of the configuration loading function, specifically focusing on the filtering logic and its integration with the existing Logcollector features.

Today's progress on integrating journald logs into Logcollector includes two significant advancements:

Successful Merge of Journald Configuration Blocks:
- Achieved a seamless integration of two journald configuration blocks. This development ensures that Logcollector can now effectively consolidate settings from multiple blocks, enhancing its configurability and flexibility in handling journald logs.
Enhancements to Configuration Block Handling:
- Currently developing a robust method that enables Logcollector to gracefully bypass a journald configuration block if its associated regex fails to compile. This not only prevents potential disruptions in log collection but also optimizes resource usage by reallocating memory from the failed block to another configuration block. This approach underscores our commitment to improving Logcollector's efficiency and reliability.

These steps represent crucial progress in our efforts to refine Logcollector's functionality and offer more versatile log management capabilities.

Today's update marks a significant milestone in enhancing Logcollector's resilience and adaptability:

Completion and Manual Testing of Robust Error Handling Method: Successfully finalized the development of a method that allows Logcollector to elegantly skip over a journald configuration block when its associated regular expression cannot be compiled. This functionality has been thoroughly tested manually to ensure it works as intended, effectively preventing potential disruptions and optimizing Logcollector's performance and reliability in handling diverse journald log configurations. This advancement ensures a more stable and efficient log collection process, even in the face of configuration errors.

wazuh / wazuh