Command manager timeseries index model

Description

In order to schedule the management of the commands by the Command Manager plugin, we need to add a @timestamp field to the index data model. This field is part of the ECS.

This field will be initialized by the plugin. In other words, the value of the field depicts the exact moment on which the command is indexed. We'll also include a field to store the result of delivery_timestamp = timestamp + timeout.

Functional requirements

The commands index template has mappings for the @timestamp field.
The command manager initializes the commands with a timestamp generated during runtime.
The command manager job scheduler uses the timestamp to sort the commands by time.

Implementation restrictions

Regenerate the index template using our tools under wazuh-indexer/ecs.

Plan

[x] Add @timestamp to the index data model (wazuh-indexer/ecs + generator)
[x] Update the index data model (wazuh-indexer/plugins/setup)
[x] Generate and add the timestamp to the command object before indexing.
[ ] Update the scheduled job to prioritize the commands by its timestamp (+ timeout)

We decided to add the @timestamp, and delivery_timestamp fields at the document "top level" (same level than the agent, or command groups), the pertinent modifications were made on both repositories, and now are under testing.

Wazuh-indexer updates

The ECS definitions of the command index were modified, creating a base.yml file which stores the delivery_timestamp field definition, and applying the corresponding modifications on the subset.yml file. The event_generator was updated and the index template was successfully generated

Command index pattern

```JSON { "index_patterns": [ ".commands*" ], "mappings": { "date_detection": false, "dynamic": "strict", "properties": { "@timestamp": { "type": "date" }, "agent": { "properties": { "groups": { "ignore_above": 1024, "type": "keyword" } } }, "command": { "properties": { "action": { "properties": { "args": { "ignore_above": 1024, "type": "keyword" }, "name": { "ignore_above": 1024, "type": "keyword" }, "version": { "ignore_above": 1024, "type": "keyword" } } }, "order_id": { "ignore_above": 1024, "type": "keyword" }, "request_id": { "ignore_above": 1024, "type": "keyword" }, "result": { "properties": { "code": { "type": "short" }, "data": { "ignore_above": 1024, "type": "keyword" }, "message": { "ignore_above": 1024, "type": "keyword" } } }, "source": { "ignore_above": 1024, "type": "keyword" }, "status": { "ignore_above": 1024, "type": "keyword" }, "target": { "properties": { "id": { "ignore_above": 1024, "type": "keyword" }, "type": { "ignore_above": 1024, "type": "keyword" } } }, "timeout": { "type": "short" }, "user": { "ignore_above": 1024, "type": "keyword" } } }, "delivery_timestamp": { "type": "date" } } }, "order": 1, "settings": { "index": { "hidden": true, "number_of_replicas": "0", "number_of_shards": "1", "query.default_field": [ "command.source", "command.target.type", "command.status", "command.action.name" ], "refresh_interval": "5s" } } } ```

Wazuh-indexer-plugins updates

Added timestamp and deliveryTimestamp attributes to the command-manager Document model, the attributes are mapped to the @timestamp and delivery_timestamp fields corresponding to each case.

Updated command index template on both plugins' resources.

Command created with a POST request to the command-manager API

{
    "took": 32,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": ".commands",
                "_id": "fZuAdJMBy-t6tcAPACYT",
                "_score": 1.0,
                "_source": {
                    "agent": {
                        "groups": [
                            "groups000"
                        ]
                    },
                    "command": {
                        "source": "Engine",
                        "user": "TestUser",
                        "target": {
                            "type": "agent",
                            "id": "TestTarget"
                        },
                        "action": {
                            "name": "restart",
                            "args": [
                                "/test/path/fake/args"
                            ],
                            "version": "v4"
                        },
                        "timeout": 30,
                        "status": "PENDING",
                        "order_id": "fJuAdJMBy-t6tcAPACYT",
                        "request_id": "e5uAdJMBy-t6tcAPACYS"
                    },
                    "@timestamp": 1732826365971,
                    "delivery_timestamp": 1732826395971
                }
            }
        ]
    }
}

wazuh / wazuh-indexer