Verify basic cases of engine's events and API behavior

[roronoasins]

Target version	Related issue	Related PR/dev branch
5.0	https://github.com/wazuh/wazuh-qa/issues/3533	https://github.com/wazuh/wazuh/issues/11334

Description

Since the team is reworking the engine, we need to cover this new engine rework. This issue will test the new engine to ensure all is correct.

Proposed test cases

• [x] We want to test a correct input returns a valid result, an incorrect input returns an error and an unexpected input doesn't crash the program. The module to test are:
  • [x] Events
  • [x] API
  • [x] Catalog
  • [x] KVDB

[roronoasins]

2022/10/25

Today I was not able to connect an agent with the engine to start the testing using the agent

2022/10/25 16:38:52 wazuh-authd: INFO: Agent key generated for 'engine-agent' (requested by any)

2022/10/25 16:39:11 wazuh-agentd: INFO: Trying to connect to server (192.168.0.101:1514/tcp).
2022/10/25 16:40:21 wazuh-agentd: INFO: Closing connection to server (192.168.0.101:1514/tcp).
2022/10/25 16:40:21 wazuh-agentd: INFO: Trying to connect to server (192.168.0.101:1514/tcp).
2022/10/25 16:41:31 wazuh-agentd: INFO: Closing connection to server (192.168.0.101:1514/tcp).

[roronoasins]

Check events' inputs

Via agent localfile

The engine receive the incoming events in alerts-ECS.json

First, to be able to send the events to the engine, we need to create a localfile section in the agent's side using its ossec.conf.

<localfile>
    <log_format>syslog</log_format>
    <location>/var/log/engine.log</location>
  </localfile>

Send valid json format log :green_circle:

1. Write the log ``` echo "{"wazuh":{"queue":49,"origin":"/var/log/engine.log"},"agent":{"id":"001","name":"engine-agent","registeredIP":"any"},"event":{"original":"{\"data\": {\"affected_items\": [], \"total_affected_items\": 0, \"total_failed_items\": 0, \"failed_items\": []}, \"message\": \"No CISCAT results were returned\", \"error\": 0}"},"_json":{"data":{"affected_items":[],"total_affected_items":0,"total_failed_items":0,"failed_items":[]},"message":"No CISCAT results were returned","error":0}}" >> /var/log/engine.log ``` 2. Verify that the log is within the `/var/ossec/logs/alerts-ECS.json` ``` root@engine:/home/vagrant# tail -f /var/ossec/logs/alerts/alerts-ECS.json | grep 001 {"wazuh":{"queue":49,"origin":"wazuh-logcollector"},"agent":{"id":"001","name":"engine-agent","registeredIP":"any"},"event":{"original":"ossec: File rotated (inode changed): '/var/log/engine.log'."}} {"wazuh":{"queue":49,"origin":"/var/log/engine.log"},"agent":{"id":"001","name":"engine-agent","registeredIP":"any"},"event":{"original":"{\"data\": {\"affected_items\": [], \"total_affected_items\": 0, \"total_failed_items\": 0, \"failed_items\": []}, \"message\": \"No CISCAT results were returned\", \"error\": 0}"},"_json":{"data":{"affected_items":[],"total_affected_items":0,"total_failed_items":0,"failed_items":[]},"message":"No CISCAT results were returned","error":0}} ```

Send a log that contains unicode characters :green_circle:

1. Write the log ``` echo "μλくよロㅋ㉉�" >> /var/log/engine.log ``` 2. Verify that the log is within the `/var/ossec/logs/alerts-ECS.json` ``` root@engine:/home/vagrant# tail -f /var/ossec/logs/alerts/alerts-ECS.json | grep 001 {"wazuh":{"queue":49,"origin":"wazuh-logcollector"},"agent":{"id":"001","name":"engine-agent","registeredIP":"any"},"event":{"original":"ossec: File rotated (inode changed): '/var/log/engine.log'."}} {"wazuh":{"queue":49,"origin":"/var/log/engine.log"},"agent":{"id":"001","name":"engine-agent","registeredIP":"any"},"event":{"original":"\u03BC\u03BB\u304F\u3088\u30ED\u314B\u3249\uFFFD"}} ```

Send a log with expected syslog format :green_circle:

1. Write the log ``` echo "Jun 13 09:16:43 NoName mikrotik: 192.168.0.1 Jun 10 15:51:05 NoName login failure for user admin from 192.168.0.204 via winbox" >> /var/log/engine.log ``` 2. Verify that the log is within the `/var/ossec/logs/alerts-ECS.json` ``` root@engine:/home/vagrant# tail -f /var/ossec/logs/alerts/alerts-ECS.json | grep 001 {"wazuh":{"queue":49,"origin":"/var/log/engine.log"},"agent":{"id":"001","name":"engine-agent","registeredIP":"any"},"event":{"original":"Jun 13 09:16:43 NoName mikrotik: 192.168.0.1 Jun 10 15:51:05 NoName login failure for user admin from 192.168.0.204 via winbox","kind":"event"},"message":"192.168.0.1 Jun 10 15:51:05 NoName login failure for user admin from 192.168.0.204 via winbox","_":" ","host":{"hostname":"NoName"},"process":{"name":"mikrotik"},"timestamp":"Jun 13 09:16:43","related":{"hosts":["NoName"]}} ```

Send a log with non-desired log format :green_circle:

1. Write the log > The log contains another json in the middle of a field, where `"fai{"data"` appears > A log file is being used, as well as a json could be used. ``` echo "{"data": {"affected_items": [], "total_affected_items": 0, "total_failed_items": 0, "fai{"data": {"affected_items": [], "total_affected_items": 0, "total_failed_items": 0, "failed_items": []}, "message": "No CISCAT results were returned", "error": 0}led_items": []}, "message": "No CISCAT results were returned", "error": 0}" >> /var/log/engine.log ``` 2. Verify that the log is within the `/var/ossec/logs/alerts-ECS.json` ``` root@engine:/home/vagrant# tail -f /var/ossec/logs/alerts/alerts-ECS.json | grep 001 {"wazuh":{"queue":49,"origin":"/var/log/engine.log"},"agent":{"id":"001","name":"engine-agent","registeredIP":"any"},"event":{"original":"{\"data\": {\"affected_items\": [], \"total_affected_items\": 0, \"total_failed_items\": 0, \"fai{\"data\": {\"affected_items\": [], \"total_affected_items\": 0, \"total_failed_items\": 0, \"failed_items\": []}, \"message\": \"No CISCAT results were returned\", \"error\": 0}led_items\": []}, \"message\": \"No CISCAT results were returned\", \"error\": 0}"}} ```

Send log mixing unicode chars and special chars :green_circle:

1. Write the log ``` echo "Mixing unicode chars μ and special chars @" >> /var/log/engine.log ``` 2. Verify that the log is within the `/var/ossec/logs/alerts-ECS.json` ``` [root@engine-agent2 vagrant]# tail -f /var/ossec/logs/alerts/alerts-ECS.json | grep 001 {"wazuh":{"queue":49,"origin":"/var/log/engine.log"},"agent":{"id":"001","name":"engine-agent2","registeredIP":"any"},"event":{"original":"Mixing unicode chars \u03BC and special chars @"}} ```

Via wazuh-engine test command

Help message

``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test --help Utility to test the ruleset Usage: /var/ossec/engine/wazuh-engine test [OPTIONS] Options: -h,--help Print this help message and exit -k,--kvdb_path TEXT:DIR [/var/ossec/etc/kvdb/] Path to KVDB folder. -f,--file_storage TEXT:DIR [/var/ossec/engine/store] Path to folder where assets are located. --environment TEXT [environment/wazuh/0] Environment name. -q,--protocol_queue CHAR [1] Protocol queue number of the event. -l,--protocol_location TEXT [/dev/stdin] Protocol location. --log_level INT [3] Log level. 0 = Debug, 1 = Info, 2 = Warning, 3 = Error -d,--debug [0] Enable debug mode [0-2]. Flag can appear multiple times. No flag[0]: No debug, d[1]: Asset history, dd[2]: 1 + Full tracing. -t,--trace TEXT [ALL] Needs: --debug Assets to be traced, separated by commas. Only effective if debug=2. ``` > The engine collects logs with the following format: > > ``` > msg_size(little_endian)queue:location:msg > ```

Verify that original field contains the specified log :green_circle:

To be able to check this, we have to run the engine test feature ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main test ``` And then, we are able to go.

Check json format log :green_circle:

``` Enter log in single line (Crtl+C to exit): {"data": {"affected_items": [], "total_affected_items": 0, "total_failed_items": 0, "failed_items": []}, "message": "No CISCAT results were returned", "error": 0} OUTPUT: { "wazuh": { "queue": 49, "origin": "/dev/stdin" }, "event": { "original": "{\"data\": {\"affected_items\": [], \"total_affected_items\": 0, \"total_failed_items\": 0, \"failed_items\": []}, \"message\": \"No CISCAT results were returned\", \"error\": 0}" }, "_json": { "data": { "affected_items": [], "total_affected_items": 0, "total_failed_items": 0, "failed_items": [] }, "message": "No CISCAT results were returned", "error": 0 } } ```

Check log that contains unicode characters :green_circle:

``` Enter log in single line (Crtl+C to exit): μλくよロㅋ㉉� OUTPUT: { "wazuh": { "queue": 49, "origin": "/dev/stdin" }, "event": { "original": "\u03BC\u03BB\u304F\u3088\u30ED\u314B\u3249\uFFFD" } } ```

Check log with correct format :green_circle:

``` Enter log in single line (Crtl+C to exit): Jun 13 09:16:43 NoName mikrotik: 192.168.0.1 Jun 10 15:51:05 NoName login failure for user admin from 192.168.0.204 via winbox OUTPUT: { "wazuh": { "queue": 49, "origin": "/dev/stdin" }, "event": { "original": "Jun 13 09:16:43 NoName mikrotik: 192.168.0.1 Jun 10 15:51:05 NoName login failure for user admin from 192.168.0.204 via winbox", "kind": "event" }, "message": "192.168.0.1 Jun 10 15:51:05 NoName login failure for user admin from 192.168.0.204 via winbox", "_": " ", "host": { "hostname": "NoName" }, "process": { "name": "mikrotik" }, "timestamp": "Jun 13 09:16:43", "related": { "hosts": [ "NoName" ] } } ```

Check log with wrong format :green_circle:

The event will not break because the log is sent and every character is formatted(or scapped) so it will always be sent as expected. At least the `original` field will have the log that was sent. ``` OUTPUT: { "wazuh": { "queue": 127, "origin": "location" }, "event": { "original": "{\"data\": {\"affected_items\": [], \"total_affected_items\": 0, \"total_failed_items\": 0, \"fai{\"data\": {\"affected_items\": [], \"total_affected_items\": 0, \"total_failed_items\": 0, \"failed_items\": []}, \"message\": \"No CISCAT results were returned\", \"error\": 0}led_items\": []}, \"message\": \"No CISCAT results were returned\", \"error\": 0}" } } ```

Check log mixing unicode chars and special chars :green_circle:

``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test Enter log in single line (Crtl+C to exit): Mixing unicode chars μ and special chars @ OUTPUT: { "wazuh": { "queue": 49, "origin": "/dev/stdin" }, "event": { "original": "Mixing unicode chars \u03BC and special chars @" } } ```

Verify that location field contains the specified string :red_circle:

The engine collects logs with the following format: ``` msg_size(little_endian)queue:location:msg ```

location value contains a colon :red_circle:

1. Start the test feature with the location `location:asd` ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main test -l "location:asd" ``` 2. Send any log ``` Enter log in single line (Crtl+C to exit): a sample log OUTPUT: { "wazuh": { "queue": 49, "origin": "location" }, "event": { "original": "asd:a sample log" } } ``` The engine split the message by the colon(:) and it breaks the message. With more than one colon, it is splitted and the remaining string is concatenated with the log. ``` Enter log in single line (Crtl+C to exit): sample log OUTPUT: { "wazuh": { "queue": 49, "origin": "a" }, "event": { "original": "sample:location::sample log" } } ``` > The location and the log have a extra colon in their union.

using a large location value :green_circle:

The location parameter accepts text with 20k characters but then it just stop working because bash limit. invocation: `./build/main test -l "location_text"` ``` bash: ./build/main: Argument list too long ```

Using a mix of special and unicode characters :green_circle:

``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -l "mixμ@" Enter log in single line (Crtl+C to exit): sample log OUTPUT: { "wazuh": { "queue": 49, "origin": "mix\u03BC@" }, "event": { "original": "sample log" } } ```

Verify that queue field contains the corresponding (decimal) ascii value from the specified string :yellow_circle:

The engine collect logs with the following format: ``` msg_size(little_endian)queue:location:msg ```

queue parameter is not descriptive enough :yellow_circle:

When using the `queue` parameter, something could be messy. When we see the `--help` output: ``` -q,--protocol_queue CHAR [1] Protocol queue number of the event. ``` If we didn't know about this queue before, we could not understand after some tries what does it expect. It uses a dec value from the ascii table but only until the 127 value, so we can't use `128`(or higher): ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main test -l "location" -q "128" Could not convert: --protocol_queue = 128 Run with --help for more information. ``` This could be improved so the user can understand this better. We are able to set the the queue with negative dec values: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -q -128 Enter log in single line (Crtl+C to exit): sample OUTPUT: { "wazuh": { "queue": -128, "origin": "/dev/stdin" }, "event": { "original": "sample" } } ``` > Is there a reason we just support -128 to 127 range?

[roronoasins]

2022/10/27

• Set up windows env and test events
• Could not send bytes to the socket using go script. This case is missing, will research again by sending it with Python(couldn't do it before, only TCP socket was receiving while unixgram not)
• Could not find info about engine's API
• Research cmd parameters(and try some of them, need more info)

[roronoasins]

2022/10/28

• Test cmd parameters: test, catalog, and env. Need an input file example to check the kvdb subcommand
• Write down the results of the testing and some improvement suggestion(documentation)
• Every time I power off the engine env, when starting again no agents can reconnect or connect for the first time
• Verify new vagrant file version works as expected
• Could not test the API and ruleset because info or some guidance is required

[roronoasins]

The events and cmd parameters were done but some cases are missing:

• Sending ascii chars via bytes (sending to the socket)
• env command for cmd parameters (waiting for testing custom ruleset so the custom env management could be tested)

These cases will be done along with the other checks required within this issue https://github.com/wazuh/wazuh-qa/issues/3537

[roronoasins]

Check API behavior

catalog command

help message 🟡

Some details about the catalog and its item-types

The items we can store and manage in the catalog are: `{decoder, rule, environment, output, filter, environment, schema}`. And the expected format to specify them is `item-type/item-id/version`. If we want to check the ids and versions for certain loaded items, we can do it like this: 1. We check the item-type list ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog list environment - environment/wazuh ``` 2. Check the versions for the id we want to ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog list environment/wazuh - environment/wazuh/0 ``` 3. Check the desired item-type/item-id/version content ``` /var/ossec/engine/wazuh-engine catalog get environment/wazuh/0 ``` > We can replace `environment` for any `item-type` value.

``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog --help Operates the engine catalog Usage: ./build/main catalog [OPTIONS] SUBCOMMAND Options: -h,--help Print this help message and exit -a,--api_socket TEXT [/var/ossec/queue/sockets/engine-api] engine api address -j,--json Excludes: --yaml Use Input/Output json format -y,--yaml Excludes: --json [Used by default] Use Input/Output yaml format Subcommands: list list item-type[/item-id]: List all items of the collection. get get item-type/item-id/version: Get an item. update update item-type/item-id/version << item_file: Update an item. create create item-type << item_file: Create and add item to collection. delete delete item-type[/item-id[/version]]: Delete an item or collection. validate validate item-type/item-id/version << item_file: Validate an item. load load item-type path: Tries to create and add all items found in the path to the collection. ``` > A clarification about `item-type` values expected could be usefull, so the users can have detailed info without run the tool and getting errors. > We could also add a reference to the engine's wiki based on the command. For example, if we use the help option with the whole engine, a reference to the index or very first page of wiki. If we use the help option with a command, a reference to the command's entry. > The `--api_socket` has not been tested because the default socket has been used(from the mvp environment). It should be tested in the next testing. > The format type json was not tested because the actual info and assets use yaml format.

cmd catalog's parameter: list :yellow_circle:

> The case where the tool list a decoder when it does not exist, is covered within the `delete` case

List loaded decoders :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog list decoder - decoder/syscollector-dbsync-network-iface-inserted - decoder/queue-rootcheck - decoder/syscollector-network-end - decoder/syscollector-port - decoder/syscollector-dbsync-processes-inserted - decoder/syscollector-program-del - decoder/fim-scan - decoder/syscollector-program - decoder/syscollector-dbsync-packages-inserted - decoder/syscollector-dbsync-network-address - decoder/queue-localfile - decoder/syscollector-program-save - decoder/syscollector-process-del - decoder/syscollector-dbsync-ports - decoder/dbsync - decoder/syscollector-dbsync-network-protocol-inserted - decoder/syscollector-dbsync-osinfo - decoder/syscollector-dbsync-hotfixes-inserted - decoder/syscollector-dbsync-packages - decoder/queue-sca - decoder/syscollector-dbsync-ports-inserted - decoder/syslog - decoder/syscollector-process-save - decoder/syscollector-base - decoder/syscollector-dbsync-hotfixes - decoder/syscollector-netinfo - decoder/syscollector-dbsync-network-protocol - decoder/queue-fim - decoder/syscollector-dbsync-network-iface - decoder/syscollector-hotfix - decoder/syscollector-dbsync-network-address-inserted - decoder/queue-syslog - decoder/syscollector-dbsync-base - decoder/syscollector-port-del - decoder/syscollector-dbsync-hwinfo-inserted - decoder/queue-dbsync - decoder/syscollector-network - decoder/syscollector-network-ip - decoder/fim - decoder/queue-syscollector - decoder/fim-event - decoder/syscollector-dbsync-processes - decoder/json - decoder/syscollector-hardware - decoder/rootcheck - decoder/syscollector-dbsync-osinfo-inserted - decoder/sca - decoder/syscollector-process - decoder/syscollector-dbsync-hwinfo - decoder/syscollector-osinfo - decoder/syscollector-port-save ```

List loaded schemas :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog list schema - schema/wazuh-logpar-types - schema/wazuh-environment - schema/wazuh-asset ```

List loaded environments :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog list environment - environment/wazuh ```

List loaded outputs :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog list output - output/file-output ```

List loaded rule 🟡

> There are no rules by default > ``` > root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog list rule > Error: [Catalog] Could not get content [rule] from store, [FileDriver] File [/var/ossec/engine/store/rule] does not exist

List loaded filter 🟡

> There are no filter by default > ``` > root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog list filter > Error: [Catalog] Could not get content [filter] from store, [FileDriver] File [/var/ossec/engine/store/filter] does not exist > ```

cmd catalog's parameter: get :green_circle:

Obtain some decoders by its id and version :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog get decoder/fim/0 name: decoder/fim/0 sources: - decoder/queue-fim/0 check: - event.original: +s_starts/{ parse: logpar: - event.original: <~json/json> ``` ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog get decoder/json/0 name: decoder/json/0 sources: - decoder/queue-syslog/0 - decoder/queue-localfile/0 check: - event.original: +s_starts/{ parse: logpar: - event.original: <~json/json> ```

Obtain some decoders with non-existing version :yellow_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog get decoder/json/22 Error: [Catalog] Could not get content [decoder/json/22] from store, [FileDriver] File [/var/ossec/engine/store/decoder/json/22] does not exist ``` ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog get decoder/json/𐤌 Error: [Catalog] Could not get content [decoder/json/𐤌] from store, [FileDriver] File [/var/ossec/engine/store/decoder/json/𐤌] does not exist ``` ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog get decoder/json/\\0 Error: [Catalog] Could not get content [decoder/json/\0] from store, [FileDriver] File [/var/ossec/engine/store/decoder/json/\0] does not exist ``` > The logging could be improved, so the user can identify the error instead of talking about the files.

cmd catalog's parameter: update :red_circle:

Update an existing decoder version :green_circle:

Get the current decoder ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog get decoder/json/1 name: decoder/json/1 sources: - decoder/queue-syslog/0 - decoder/queue-localfile/0 check: - event.original: +s_starts/{} parse: logpar: - event.original: <~json/json> ``` Update it by using a copy of the first one but adding `asd` string within the `check` in a custom yml file located in `ruleset/decoders/custom/json2.yml ` ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog update decoder/json/1 < ruleset/decoders/custom/json2.yml OK ``` Check that the update has been performed ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog get decoder/json/1 name: decoder/json/1 sources: - decoder/queue-syslog/0 - decoder/queue-localfile/0 check: - event.original: +s_starts/{asd} parse: logpar: - event.original: <~json/json> ```

Update a non-existent item-type :red_circle:

``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog update newtype/json/1 < /home/vagrant/engine/wazuh/src/engine/ruleset/decoders/json.yml Error: Invalid collection type [newtype] for [error_type] ``` > The log received looks not correct, returning `error_type` and not giving info about expected collection types or a note telling the user to run the `--help` to get the supported collection values.

Update a uncorrect id or version for a given item-type :yellow_circle:

``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog update decoder/asd/1 < /home/vagrant/engine/wazuh/src/engine/ruleset/decoders/json.yml Error: [Catalog] Invalid content name [decoder/json/0] for decoder [decoder/asd/1] ``` ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog update decoder/json/1 < /home/vagrant/engine/wazuh/src/engine/ruleset/decoders/json.yml Error: [Catalog] Invalid content name [decoder/json/0] for decoder [decoder/json/1] ``` > The given log could be improved so the user can identify that the item defined in the decoder file does not match with the one they're trying to update.

cmd catalog's parameter: create :yellow_circle:

Create a custom decoder using a yml file :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog create decoder < ruleset/decoders/custom/json2.yml OK root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog get decoder/json/1 name: decoder/json/1 sources: - decoder/queue-syslog/0 - decoder/queue-localfile/0 check: - event.original: +s_starts/{} parse: logpar: - event.original: <~json/json> ```

Create a custom decoder that already exists using a yml file :yellow_circle:

``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog create decoder < /home/vagrant/engine/wazuh/src/engine/ruleset/decoders/json.yml Error: [Catalog] Could not post content [decoder/json/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/json/0] already exists ``` > The log is just talking about files, it could be improved so the user can read that the `item` already exists.

Create an item using a type that is not expected :yellow_circle:

``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog create newtype < /home/vagrant/engine/wazuh/src/engine/ruleset/decoders/json.yml Error: Invalid collection type [newtype] ``` > The log could be improved, also the possible values can be printed or at least reference the help function so the user can get more info.

cmd catalog's parameter: delete :yellow_circle:

Delete an existing decoder :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog delete decoder OK root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog list decoder Error: [Catalog] Could not get content [decoder] from store, [FileDriver] File [/var/ossec/engine/store/decoder] does not exist ```

Try to delete empty item type :yellow_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog delete decoder Error: [Catalog] Could not delete content [decoder] from store, [FileDriver::erase] File [/var/ossec/engine/store/decoder] does not exist ``` > We could improve this log, and tell the user that there are no decoders instead of X file does not exist. > We could add a log telling the user how to check the list of the loaded `item-type` items within the catalog, in case the user do not know the current items that its environment has.

Missing required parameter :yellow_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog delete name is required Run with --help for more information. ``` > We could improve this log, instead of just logging `name` is required. It could be more like the `--help` message format `item-type[/item-id[/version]]`

Delete non-valid types, ids and versions :yellow_circle:

``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog delete decoder/json/3 Error: [Catalog] Could not delete content [decoder/json/3] from store, [FileDriver::erase] File [/var/ossec/engine/store/decoder/json/3] does not exist root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog delete decoder/jsonn Error: [Catalog] Could not delete content [decoder/jsonn] from store, [FileDriver::erase] File [/var/ossec/engine/store/decoder/jsonn] does not exist root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog delete newtype Error: Invalid collection type [newtype] ``` > As the previous cases, we could improve these logs.

cmd catalog's parameter: validate :yellow_circle:

Validate a decoder with valid syntax 🟡

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog validate decoder/json/1 < ruleset/decoders/custom/json2.yml OK ``` > Is there a way to have a trace of the checks that it performs to know if the validate went good? If not, could we add it?

Validate a decoder with no name section :yellow_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog validate decoder/json/1 < ruleset/decoders/custom/json2.yml Error: [Catalog] Could not validate content, [Catalog] Engine validation failed for [decoder/json/1], exception: [Asset::Asset(jsonDefinition, type)] Asset definition missing string name ```

Validate a decoder with no sources section :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog validate decoder/json/1 < ruleset/decoders/custom/json2.yml OK ```

Validate a decoder with no check section :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog validate decoder/json/1 < ruleset/decoders/custom/json2.yml OK ```

Validate a decoder with no parse section :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog validate decoder/json/1 < ruleset/decoders/custom/json2.yml OK ```

cmd catalog's parameter: load :yellow_circle:

Could not load existing decoders :yellow_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog load decoder ./ruleset/decoders Error: [Catalog] Could not post content [decoder/syscollector-dbsync-osinfo/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-osinfo/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-hotfix/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-hotfix/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-ports-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-ports-inserted/0] already exists Error: [Catalog] Could not post content [decoder/sca/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/sca/0] already exists Error: [Catalog] Could not post content [decoder/queue-rootcheck/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-rootcheck/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-ports/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-ports/0] already exists Error: [Catalog] Could not post content [decoder/queue-dbsync/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-dbsync/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-address/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-address/0] already exists Error: [Catalog] Could not post content [decoder/queue-localfile/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-localfile/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-iface/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-iface/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-hotfixes/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-hotfixes/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-osinfo/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-osinfo/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-processes/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-processes/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-osinfo-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-osinfo-inserted/0] already exists Error: [Catalog] Could not post content [decoder/queue-fim/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-fim/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-protocol/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-protocol/0] already exists Error: [Catalog] Could not post content [decoder/fim/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/fim/0] already exists Error: [Catalog] Could not post content [decoder/json/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/json/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-iface-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-iface-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-hardware/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-hardware/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-process-save/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-process-save/0] already exists Error: [Catalog] Could not post content [decoder/fim-scan/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/fim-scan/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-hwinfo/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-hwinfo/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-address-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-address-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-netinfo/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-netinfo/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-program-save/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-program-save/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-base/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-base/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-network-ip/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-network-ip/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-network/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-network/0] already exists Error: [Catalog] Could not post content [decoder/dbsync/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/dbsync/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-program/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-program/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-base/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-base/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-packages/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-packages/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-hwinfo-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-hwinfo-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-processes-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-processes-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-process/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-process/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-process-del/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-process-del/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-packages-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-packages-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-port-del/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-port-del/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-network-end/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-network-end/0] already exists Error: [Catalog] Could not post content [decoder/queue-syscollector/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-syscollector/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-port/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-port/0] already exists Error: [Catalog] Could not post content [decoder/rootcheck/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/rootcheck/0] already exists Error: [Catalog] Could not post content [decoder/queue-sca/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-sca/0] already exists Error: [Catalog] Could not post content [decoder/queue-syslog/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-syslog/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-port-save/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-port-save/0] already exists Error: [Catalog] Could not post content [decoder/syslog/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syslog/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-protocol-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-protocol-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-hotfixes-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-hotfixes-inserted/0] already exists Error: [Catalog] Could not post content [decoder/fim-event/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/fim-event/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-program-del/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-program-del/0] already exists Error: [Catalog] Could not post content [decoder/json/1] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/json/1] already exists ``` > Same suggestion about the logging.

Load some decoders that already exists and other that do not :yellow_circle:

``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog load decoder /home/vagrant/engine/wazuh/src/engine/ruleset/decoders Error: [Catalog] Could not post content [decoder/syscollector-dbsync-osinfo/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-osinfo/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-hotfix/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-hotfix/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-ports-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-ports-inserted/0] already exists Error: [Catalog] Could not post content [decoder/sca/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/sca/0] already exists Error: [Catalog] Could not post content [decoder/queue-rootcheck/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-rootcheck/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-ports/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-ports/0] already exists Error: [Catalog] Could not post content [decoder/queue-dbsync/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-dbsync/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-address/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-address/0] already exists Error: [Catalog] Could not post content [decoder/queue-localfile/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-localfile/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-iface/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-iface/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-hotfixes/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-hotfixes/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-osinfo/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-osinfo/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-processes/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-processes/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-osinfo-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-osinfo-inserted/0] already exists Error: [Catalog] Could not post content [decoder/queue-fim/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-fim/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-protocol/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-protocol/0] already exists OK Error: [Catalog] Could not post content [decoder/json/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/json/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-iface-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-iface-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-hardware/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-hardware/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-process-save/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-process-save/0] already exists OK Error: [Catalog] Could not post content [decoder/syscollector-dbsync-hwinfo/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-hwinfo/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-address-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-address-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-netinfo/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-netinfo/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-program-save/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-program-save/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-base/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-base/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-network-ip/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-network-ip/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-network/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-network/0] already exists Error: [Catalog] Could not post content [decoder/dbsync/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/dbsync/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-program/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-program/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-base/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-base/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-packages/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-packages/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-hwinfo-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-hwinfo-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-processes-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-processes-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-process/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-process/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-process-del/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-process-del/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-packages-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-packages-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-port-del/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-port-del/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-network-end/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-network-end/0] already exists Error: [Catalog] Could not post content [decoder/queue-syscollector/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-syscollector/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-port/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-port/0] already exists Error: [Catalog] Could not post content [decoder/rootcheck/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/rootcheck/0] already exists Error: [Catalog] Could not post content [decoder/queue-sca/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-sca/0] already exists Error: [Catalog] Could not post content [decoder/queue-syslog/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-syslog/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-port-save/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-port-save/0] already exists Error: [Catalog] Could not post content [decoder/syslog/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syslog/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-protocol-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-protocol-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-hotfixes-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-hotfixes-inserted/0] already exists OK Error: [Catalog] Could not post content [decoder/syscollector-program-del/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-program-del/0] already exists ``` > Same logging suggestion.

Path with special and unicode characters :green_circle:

``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine catalog load decoder /home/vagrant/engine/wazuh/src/engine/ruleset/decodersμइ path: Directory does not exist: /home/vagrant/engine/wazuh/src/engine/ruleset/decodersμइ Run with --help for more information. ```

Load decoders :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog load decoder ./ruleset/decoders OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK ```

Load decoder version :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog load decoder ./ruleset/decoders/custom/ OK root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main catalog get decoder/json/1 name: decoder/json/1 sources: - decoder/queue-syslog/0 - decoder/queue-localfile/0 check: - event.original: +s_starts/{} parse: logpar: - event.original: <~json/json> ```

kvdb command

help message

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main kvdb --help Operates the key-value databases Usage: ./build/main kvdb [OPTIONS] Options: -h,--help Print this help message and exit -p,--path TEXT:DIR [/var/ossec/etc/kvdb/] Path to KVDB folder. -n,--name TEXT REQUIRED KVDB name to be added. -i,--input_file TEXT:FILE REQUIRED Path to file containing the KVDB data. -t,--input_type TEXT:{json} REQUIRED Type of the input file. Allowed values: json ``` > The `-p,--path` has not been tested because the default kvdb has been used(from the mvp environment). It should be tested in the next testing. > As a previous suggestion, could be useful having the kvdb wiki reference in the `kvdb --help` run. So the user can know how to build a kvdb input file and what is `KVDB`.

cmd kvdb's parameter: -i/--input-file :yellow_circle:

> Could the `-i` renamed to `-f` as the `--input-type` is `-t`?

Create a db using a json input file :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main kvdb -i test/kvdb_input_files/malicious_ip/malicious-ip.json -n testing_kvdb --input_type json 10:58:51.955059 kvdbManager.cpp:73 DBG[5225 ] adding DB with name [testing_kvdb] to available Databases 10:58:52.148139 kvdb.cpp:281 DBG[5225 ] Successfull insert [192.168.1.10,""] into DB [testing_kvdb] CF [default] 10:58:52.168986 cmdKvdb.cpp:114 INF[5225 ] KVDB [testing_kvdb] created successfully root@engine:/home/vagrant/engine/wazuh/src/engine# ls -l /var/ossec/etc/kvdb/testing_kvdb/ total 152 -rw-r--r-- 1 root root 36 Nov 7 10:58 000005.log -rw-r--r-- 1 root root 16 Nov 7 10:58 CURRENT -rw-r--r-- 1 root root 36 Nov 7 10:58 IDENTITY -rw-r--r-- 1 root root 0 Nov 7 10:58 LOCK -rw-r--r-- 1 root root 20957 Nov 7 10:58 LOG -rw-r--r-- 1 root root 59 Nov 7 10:58 MANIFEST-000004 -rw-r--r-- 1 root root 6540 Nov 7 10:58 OPTIONS-000007 ```

Create db with a wrong formatted json :yellow_circle:

``` oasosao{"agent":{"id":"021"},"event":{"original":{"ID":123456,"iface":{"name":"iface_name"}}}} ``` ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main kvdb --name test -i input.json -t json 10:42:34.551563 cmdKvdb.cpp:77 ERR[3865 ] Error parsing JSON: exception: [Json(jsonString)] Unable to build json document because: Invalid value. at 0 ``` ``` {"agent":{"id":"021"},"event":{"o riginal":{"ID":123456,"iface":{"name":"iface_name"}}}} ``` ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main kvdb --name test -i input.json -t json 10:44:00.649499 cmdKvdb.cpp:77 ERR[3870 ] Error parsing JSON: exception: [Json(jsonString)] Unable to build json document because: Invalid encoding in string. at 33 ``` > Same logging suggestion

Create db using a non-existent input file :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main kvdb --name test1 -i nofile -t json --input_file: File does not exist: nofile Run with --help for more information. ```

cmd catalog's parameter: -t/--input-type :yellow_circle:

Create a db using a non valid input file format :yellow_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main kvdb --name test -i input.yaml -t yaml --input_type: yaml not in {json} Run with --help for more information. ``` > We could improve the log that seems to be a conditional fragment from a source code.

[roronoasins]

Conclusion

During the testing process we could test the following cases:

• Events' inputs

  • Using the agent localfile feature

  • Sending valid json format log :green_circle:

  • Sending a log that contains unicode characters :green_circle:

  • Sending a log with expected syslog format :green_circle:

  • Sending a log with non-desired log format :green_circle:

  • Sending a mix of unicode and special characters log :green_circle:

  • Using wazuh-engine test command

  • Verify that original field contains the specified log :green_circle:

    • Check json format log :green_circle:
    • Check log that contains unicode characters :green_circle:
    • Check log with correct format :green_circle:
    • Check log with wrong format :green_circle:
    • Check log mixing unicode chars and special chars :green_circle:

  • Verify that location field contains the specified string :red_circle:

    • location value contains a colon :red_circle:
    • using a large location value :green_circle:
    • Using a mix of special and unicode characters :green_circle:

  • Verify that queue field contains the corresponding (decimal) ascii value from the specified string :yellow_circle:

• API

  • catalog command
  • list parameter :yellow_circle:
    • List loaded decoders :green_circle:
    • List loaded schemas :green_circle:
    • List loaded environments :green_circle:
    • List loaded outputs :green_circle:
    • List loaded rule 🔴
    • List loaded filter 🔴
  • get parameter
    • Obtain some decoders by its id and version :green_circle:
    • Obtain some decoders with non-existing version :yellow_circle:
  • update parameter
    • Update an existing decoder version :green_circle:
    • Update a non-existent item-type :red_circle:
    • Update a uncorrect id or version for a given item-type :yellow_circle:
  • create parameter
    • Create a custom decoder using a yml file :green_circle:
    • Create a custom decoder that already exists using a yml file :yellow_circle:
    • Create an item using a type that is not expected :yellow_circle:
  • delete parameter
    • Delete an existing decoder :green_circle:
    • Try to delete empty item type :yellow_circle:
    • Missing required parameter :yellow_circle:
    • Delete non-valid types, ids and versions :yellow_circle:
  • validate parameter
    • Validate a decoder with valid syntax 🟡
    • Validate a decoder with no name section :yellow_circle:
    • Validate a decoder with no sources section :green_circle:
    • Validate a decoder with no check section :green_circle:
    • Validate a decoder with no parse section :green_circle:
  • load parameter
    • Could not load existing decoders :yellow_circle:
    • Load some decoders that already exists and other that do not :yellow_circle:
    • Path with special and unicode characters :green_circle:
    • Load decoders :green_circle:
    • Load decoder version :green_circle:
  • kvdb command
  • -i/--input-file parameter
    • Create a db using a json input file :green_circle:
    • Create db with a wrong formatted json :yellow_circle:
    • Create db using a non-existent input file :green_circle:
  • -t/--input-type
    • Create a db using a non valid input file format :green_circle:

Improvements

• Error logs that are not that descriptive for a user:

  • Referencing the store files

    Error: [Catalog] Could not get content [rule] from store, [FileDriver] File [/var/ossec/engine/store/rule] does not exist

    These logs appear for these cases:

    • List an empty item-type
    • Get a non-existing version for a certain item
    • Create an item where it already exists
    • Update a uncorrect id or version for a given item-type
    • Invoke delete without an item-type at least. The log refers to name when it can confuse an user because it is the error log when there is something happening, for example, with the name of a decoder(defined in its yaml file). And here the user may try to delete all the possible combinations by just running the delete option. The --help advice is good, but the name reference may confuse the user
    • Delete non-valid item ids and versions
    • Validate a decoder with no name section
    • Load an already existing item-type

  • Uncompleted logging

    Error: Invalid collection type [newtype]

    or

    Could not convert: --protocol_queue = 128
    Run with --help for more information.

    These logs appear for these cases:

    • Create a not supported collection type
    • Use a not expected queue value
    • Delete an empty item-type
    • Delete non-valid item-types

  • JSON wrong format

    10:44:00.649499 cmdKvdb.cpp:77   ERR[3870  ] Error parsing JSON: exception: [Json(jsonString)] Unable to build json document because: Invalid encoding in string. at 33

    We could add some logging to the stack trace so the last logging message is more user-friendly.

    These logs appear for these cases:

    • Creating a database using a non-well-formatted json input file

  • Unsupported type

    --input_type: yaml not in {json}

    This log can be improved so the log do not seem to be a conditional fragment from a source code.

    These logs appear for these cases:

    • Create a db using a non valid input file format

• Rename the kvdb's -i option: The format used for --input-type is -t, so we could use the same format.

• Check that validation is correct: Is there a way to have a trace of the checks that it performs to know if the validate went good? If not, could we add it?

• The catalog list command must have default rules and filter.

• Helper of catalog command:

  A clarification about item-type values expected could be usefull, so the users can have detailed info without run the tool and getting errors.

  We could also add a reference to the engine's wiki based on the command. For example, if we use the help option with the whole engine, a reference to the index or very first page of wiki. If we use the help option with a command, a reference to the command's entry.

Bugs

• Location being splitted

  When the location contains at least a colon, the substring that take place after that colon appears as part of the original log. There is also an aditional colon where this substring and the log are combined.

  • Input:

    root@engine:/home/vagrant/engine/wazuh/src/engine# ./build/main test -l "location:asd"

  • Current output:

    Enter log in single line (Crtl+C to exit):
    
    a sample log
    
    OUTPUT:
    
    {
        "wazuh": {
            "queue": 49,
            "origin": "location"
        },
        "event": {
            "original": "asd:a sample log"
        }
    }

  • Expected output:

    Enter log in single line (Crtl+C to exit):
    
    a sample log
    
    OUTPUT:
    
    {
        "wazuh": {
            "queue": 49,
            "origin": "location:asd"
        },
        "event": {
            "original": "a sample log"
        }
    }

• Uncorrect logs

  • Current output

    Error: Invalid collection type [newtype] for [error_type]

  • Expected output Fix the string error_type and print the supported item-types. It could contains a reference to the help option instead (or both).

  These logs appear for these cases:

  • Update a non-existent item-type