Modify Indexer template to add new field (wazuh.cluster.name)

[Dwordcito]

Description

This issue is aimed at enhancing the functionality of the Wazuh indexer by modifying its template. Specifically, we will be adding a new field under wazuh.cluster.node to capture additional information that can be valuable for cluster management.

Changes Required

The following changes will be implemented:

1. Modify Indexer Template: The Wazuh indexer template will be modified to include a new field under wazuh.cluster.node.

Definition of Done

• The indexer template is successfully modified to include the new field under wazuh.cluster.node.
• The updated template is tested for compatibility with the existing Wazuh cluster functionality.

[AlexRuiz7]

we will be adding a new field under wazuh.cluster.node to capture additional information that can be valuable for cluster management.

What kind of information @Dwordcito ? Is that field a simple field or a nested one?

[Dwordcito]

After discussion in IM. More definitions:

wazuh.cluster.name is a string.

{
  "wazuh": {
    "cluster": {
      "name": "value"
    }
  }
}

[AlexRuiz7]

Understood. The mappings have been updated to match a custom wazuh field with the following properties:

{
  "wazuh": {
    "cluster": {
      "name": {
        "type": "keyword"
      },
      "node": {
        "type": "keyword"
      }
    }
  }
}

[AlexRuiz7]

I've discovered how to disable the dynamic generation of mappings using the ECS mapping generation tool.

If you try to index a document that contains fields not present in the mappings and the "mappings.dynamic" setting is set to "strict", Elasticsearch or Opensearch will reject the document and throw an error. This is because the "strict" setting instructs the engine to reject any document that contains fields not explicitly defined in the mapping.

This is useful when you want to ensure that all documents adhere to a specific schema and prevent the automatic creation of fields. If you want to allow dynamic field creation, you can set "mappings.dynamic" to "true". If you want to ignore new fields, you can set it to "false".

However, after testing, documents are rejected because of the base.labels field, as this field of type object is meant to be used for the addition of custom values. We currently do not use that fields, so we'll remove it and test how the new mappings work in a real environment.

Vagrantfile

```ruby Vagrant.configure("2") do |config| config.vm.define "aio" do |aio| aio.vm.box = "ubuntu/jammy64" aio.vm.synced_folder ".", "/vagrant" aio.vm.network "private_network", ip: "192.168.56.10", name: "vboxnet0" aio.vm.hostname = "aio" aio.vm.provider "virtualbox" do |vb| vb.memory = "4096" vb.cpus = "4" end aio.vm.provision "shell", inline: <<-SHELL # Disable firewall systemctl stop firewalld systemctl disable firewalld curl -sO https://packages-dev.wazuh.com/4.8/wazuh-install.sh && sudo bash ./wazuh-install.sh -a SHELL end config.vm.define "agent" do |agent| agent.vm.box = "ubuntu/jammy64" agent.vm.synced_folder ".", "/vagrant" agent.vm.network "private_network", ip: "192.168.56.11", name: "vboxnet0" agent.vm.hostname = "agent" agent.vm.provider "virtualbox" do |vb| vb.memory = "1024" vb.cpus = "1" end agent.vm.provision "shell", inline: <<-SHELL systemctl stop firewalld systemctl disable firewalld # wget https://packages-dev.wazuh.com/pre-release/apt/pool/main/w/wazuh-agent/wazuh-agent_4.8.0-1_amd64.deb && sudo WAZUH_MANAGER='192.168.56.10' WAZUH_AGENT_GROUP='default' WAZUH_AGENT_NAME='Wazuh-Agent' dpkg -i ./wazuh-agent_4.8.0-1_amd64.deb wget https://packages-dev.wazuh.com/pre-release/apt/pool/main/w/wazuh-agent/wazuh-agent_4.8.0-1_amd64.deb && sudo WAZUH_MANAGER='192.168.56.10' WAZUH_AGENT_GROUP='default' WAZUH_AGENT_NAME='aio-agent' dpkg -i ./wazuh-agent_4.8.0-1_amd64.deb systemctl daemon-reload systemctl enable wazuh-agent systemctl start wazuh-agent SHELL end end ```

First, the template has to be updated. I used the Dev Tools from wazuh-dashboard to do that:

Updated index template

```console POST _index_template/wazuh-vulnerability-detector { "index_patterns": [ "wazuh-states-vulnerabilities" ], "priority": 1, "template": { "settings": { "index": { "codec": "best_compression", "mapping": { "total_fields": { "limit": 1000 } }, "number_of_replicas": "0", "number_of_shards": "1", "query.default_field": [ "base.tags", "agent.id", "ecs.version", "host.os.family", "host.os.full.text", "host.os.version", "package.name", "package.version", "vulnerability.id", "vulnerability.description.text", "vulnerability.severity", "wazuh.cluster.name" ], "refresh_interval": "2s" } }, "mappings": { "date_detection": false, "dynamic": "strict", "properties": { "@timestamp": { "type": "date" }, "agent": { "properties": { "build": { "properties": { "original": { "ignore_above": 1024, "type": "keyword" } } }, "ephemeral_id": { "ignore_above": 1024, "type": "keyword" }, "id": { "ignore_above": 1024, "type": "keyword" }, "name": { "ignore_above": 1024, "type": "keyword" }, "type": { "ignore_above": 1024, "type": "keyword" }, "version": { "ignore_above": 1024, "type": "keyword" } } }, "ecs": { "properties": { "version": { "ignore_above": 1024, "type": "keyword" } } }, "host": { "properties": { "os": { "properties": { "family": { "ignore_above": 1024, "type": "keyword" }, "full": { "fields": { "text": { "type": "text" } }, "ignore_above": 1024, "type": "keyword" }, "kernel": { "ignore_above": 1024, "type": "keyword" }, "name": { "fields": { "text": { "type": "text" } }, "ignore_above": 1024, "type": "keyword" }, "platform": { "ignore_above": 1024, "type": "keyword" }, "type": { "ignore_above": 1024, "type": "keyword" }, "version": { "ignore_above": 1024, "type": "keyword" } } } } }, "message": { "type": "text" }, "package": { "properties": { "architecture": { "ignore_above": 1024, "type": "keyword" }, "build_version": { "ignore_above": 1024, "type": "keyword" }, "checksum": { "ignore_above": 1024, "type": "keyword" }, "description": { "ignore_above": 1024, "type": "keyword" }, "install_scope": { "ignore_above": 1024, "type": "keyword" }, "installed": { "type": "date" }, "license": { "ignore_above": 1024, "type": "keyword" }, "name": { "ignore_above": 1024, "type": "keyword" }, "path": { "ignore_above": 1024, "type": "keyword" }, "reference": { "ignore_above": 1024, "type": "keyword" }, "size": { "type": "long" }, "type": { "ignore_above": 1024, "type": "keyword" }, "version": { "ignore_above": 1024, "type": "keyword" } } }, "tags": { "ignore_above": 1024, "type": "keyword" }, "vulnerability": { "properties": { "category": { "ignore_above": 1024, "type": "keyword" }, "classification": { "ignore_above": 1024, "type": "keyword" }, "description": { "fields": { "text": { "type": "text" } }, "ignore_above": 1024, "type": "keyword" }, "enumeration": { "ignore_above": 1024, "type": "keyword" }, "id": { "ignore_above": 1024, "type": "keyword" }, "reference": { "ignore_above": 1024, "type": "keyword" }, "report_id": { "ignore_above": 1024, "type": "keyword" }, "scanner": { "properties": { "vendor": { "ignore_above": 1024, "type": "keyword" } } }, "score": { "properties": { "base": { "type": "float" }, "environmental": { "type": "float" }, "temporal": { "type": "float" }, "version": { "ignore_above": 1024, "type": "keyword" } } }, "severity": { "ignore_above": 1024, "type": "keyword" } } }, "wazuh": { "properties": { "cluster": { "properties": { "name": { "ignore_above": 1024, "type": "keyword" }, "node": { "ignore_above": 1024, "type": "keyword" } } } } } } } } } ```

Also in the UI, go to Indexer/dashboard management < App Settings < Miscellaneous and run a Health-check to create the index pattern.

Then I generated and uploaded events using our tool.

./event_generator.py
How many events do you want to generate? 100
Do you want to inject the generated data into your indexer? (y/n) y
Enter the IP of your Indexer: 192.168.56.10
Enter the port of your Indexer: 9200
Enter the index name: wazuh-states-vulnerabilities
Username: admin
Password: uUmaWuLr*CdGO+0HI5B7+u.9mMqhw4i?

After that, I checked the UI section at Threat Intelligence < Vulnerability detection. Results are below:

[AlexRuiz7]

I also made an E2E test as follows. The wazuh-server was restarted after editing the ossec.conf and agent.conf files.

ossec.conf

```xml yes yes 60m yes https://192.168.56.10:9200 admin uUmaWuLr*CdGO+0HI5B7+u.9mMqhw4i? /etc/filebeat/certs/root-ca.pem /etc/filebeat/certs/wazuh-server.pem /etc/filebeat/certs/wazuh-server-key.pem ```

shared agent.conf

```xml no 5m yes yes yes yes yes yes yes yes ```

filebeat test output

```console root@aio:/home/vagrant# filebeat test output elasticsearch: https://localhost:9200... parse url... OK connection... parse host... OK dns lookup... OK addresses: 127.0.0.1 dial up... OK TLS... security: server's certificate chain verification is enabled handshake... OK TLS version: TLSv1.2 dial up... OK talk to server... OK version: 7.10.2 ```

Manager & agent logs


No events were generated:

[AlexRuiz7]

I created a newer version of the wazuh-install.sh script with the changes from v4.8.0-alpha2, installed an AIO deployment in Vagrant and carried on the tests:

git clone https://github.com/wazuh/wazuh-packages.git
cd wazuh-packages/
git checkout 4.8.0
cd unattended_installer/builder.sh -i -d pre-release

Before: using the previous template with dynamic mappings enabled, 222 documents were indexer after 5 minutes.

After: using the new template with wazuh.cluster.node and mappings.dynamic: strict, 222 documents after 5 minutes.

Conclusion: tests passed 🟢