Test recursive item load using the engine's API

[roronoasins]

Target version	Related issue	Related PR/dev branch
5.0	https://github.com/wazuh/wazuh-qa/issues/3620	https://github.com/wazuh/wazuh/issues/15312

Description

The dev team did some changes to the load feature, so you can recursive load any item-type instead of load every path you need, being more friendly the item additions.

Proposed test cases

• [x] Recursively load each item-type
  • [x] decoder
  • [x] rule
  • [x] output
  • [x] environment
  • [x] schema

The filter item-type won't be tested becauses there are no samples and it is not a priority

Extra cases

• [x] Load with empty path
• [x] Load a non-existent item-type

Expected results

Being able to load decoders located in directories that have also more than one level.

[roronoasins]

Ruleset reorganization

Catalog recursive load

Help message

``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog load --help load item-type path: Tries to create and add all items found in the path to the collection. Usage: /var/ossec/engine/wazuh-engine catalog load [OPTIONS] name path Positionals: name TEXT REQUIRED Name identifying the collection to add items: item-type path TEXT:DIR REQUIRED Path to the directory containing the item files. Options: -h,--help Print this help message and exit -r,--recursive Recursive loading of the directory. ```

Recursive load: decoder :yellow_circle:

Load decoders recursively :green_circle:

Perform the load using the -r option

When we have decoders and more directories that also contains decoders, we can recursively load them For example, having these directories ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/decoders/ json.yml queue-localfile.yml queue-syslog.yml syslog.yml wazuh-agent root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/decoders/wazuh-agent/ dbsync fim rootcheck sca syscollector root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/decoders/wazuh-agent/syscollector/ queue-syscollector.yml syscollector-dbsync-network-iface.yml syscollector-dbsync-processes.yml syscollector-port.yml syscollector-base.yml syscollector-dbsync-network-protocol-inserted.yml syscollector-hardware.yml syscollector-process-del.yml syscollector-dbsync-base.yml syscollector-dbsync-network-protocol.yml syscollector-hotfix.yml syscollector-process-save.yml syscollector-dbsync-hotfixes-inserted.yml syscollector-dbsync-osinfo-inserted.yml syscollector-netinfo.yml syscollector-process.yml syscollector-dbsync-hotfixes.yml syscollector-dbsync-osinfo.yml syscollector-network-end.yml syscollector-program-del.yml syscollector-dbsync-hwinfo-inserted.yml syscollector-dbsync-packages-inserted.yml syscollector-network-ip.yml syscollector-program-save.yml syscollector-dbsync-hwinfo.yml syscollector-dbsync-packages.yml syscollector-network.yml syscollector-program.yml syscollector-dbsync-network-address-inserted.yml syscollector-dbsync-ports-inserted.yml syscollector-osinfo.yml syscollector-dbsync-network-address.yml syscollector-dbsync-ports.yml syscollector-port-del.yml syscollector-dbsync-network-iface-inserted.yml syscollector-dbsync-processes-inserted.yml syscollector-port-save.yml ``` Now we load them ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog load -r decoder ruleset/decoders/ OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK ``` And we check if they have been loaded as expected ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog list decoder - decoder/syscollector-dbsync-network-iface-inserted - decoder/queue-rootcheck - decoder/syscollector-network-end - decoder/syscollector-port - decoder/syscollector-dbsync-processes-inserted - decoder/syscollector-program-del - decoder/fim-scan - decoder/syscollector-program - decoder/syscollector-dbsync-packages-inserted - decoder/syscollector-dbsync-network-address - decoder/queue-localfile - decoder/syscollector-program-save - decoder/syscollector-process-del - decoder/syscollector-dbsync-ports - decoder/dbsync - decoder/syscollector-dbsync-network-protocol-inserted - decoder/syscollector-dbsync-osinfo - decoder/syscollector-dbsync-hotfixes-inserted - decoder/syscollector-dbsync-packages - decoder/queue-sca - decoder/syscollector-dbsync-ports-inserted - decoder/syslog - decoder/syscollector-process-save - decoder/syscollector-base - decoder/syscollector-dbsync-hotfixes - decoder/syscollector-netinfo - decoder/syscollector-dbsync-network-protocol - decoder/queue-fim - decoder/syscollector-dbsync-network-iface - decoder/syscollector-hotfix - decoder/syscollector-dbsync-network-address-inserted - decoder/queue-syslog - decoder/syscollector-dbsync-base - decoder/syscollector-port-del - decoder/syscollector-dbsync-hwinfo-inserted - decoder/queue-dbsync - decoder/syscollector-network - decoder/syscollector-network-ip - decoder/fim - decoder/queue-syscollector - decoder/fim-event - decoder/syscollector-dbsync-processes - decoder/json - decoder/myjson - decoder/syscollector-hardware - decoder/rootcheck - decoder/syscollector-dbsync-osinfo-inserted - decoder/syscollector-process - decoder/syscollector-dbsync-hwinfo - decoder/syscollector-osinfo - decoder/syscollector-port-save root@engine:/home/vagrant/engine/wazuh/src/engine# ```

Perform the load using the --recursive option

When we have decoders and more directories that also contains decoders, we can recursively load them For example, having these directories ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/decoders/ json.yml queue-localfile.yml queue-syslog.yml syslog.yml wazuh-agent root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/decoders/wazuh-agent/ dbsync fim rootcheck sca syscollector ``` Now we load them ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog load --recursive decoder ruleset/decoders/ OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK root@engine:/home/vagrant/engine/wazuh/src/engine# ``` And then, verify if they have been loaded as expected ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog list decoder - decoder/syscollector-dbsync-network-iface-inserted - decoder/queue-rootcheck - decoder/syscollector-network-end - decoder/syscollector-port - decoder/syscollector-dbsync-processes-inserted - decoder/syscollector-program-del - decoder/fim-scan - decoder/syscollector-program - decoder/syscollector-dbsync-packages-inserted - decoder/syscollector-dbsync-network-address - decoder/queue-localfile - decoder/syscollector-program-save - decoder/syscollector-process-del - decoder/syscollector-dbsync-ports - decoder/dbsync - decoder/syscollector-dbsync-network-protocol-inserted - decoder/syscollector-dbsync-osinfo - decoder/syscollector-dbsync-hotfixes-inserted - decoder/syscollector-dbsync-packages - decoder/queue-sca - decoder/syscollector-dbsync-ports-inserted - decoder/syslog - decoder/syscollector-process-save - decoder/syscollector-base - decoder/syscollector-dbsync-hotfixes - decoder/syscollector-netinfo - decoder/syscollector-dbsync-network-protocol - decoder/queue-fim - decoder/syscollector-dbsync-network-iface - decoder/syscollector-hotfix - decoder/syscollector-dbsync-network-address-inserted - decoder/queue-syslog - decoder/syscollector-dbsync-base - decoder/syscollector-port-del - decoder/syscollector-dbsync-hwinfo-inserted - decoder/queue-dbsync - decoder/syscollector-network - decoder/syscollector-network-ip - decoder/fim - decoder/queue-syscollector - decoder/fim-event - decoder/syscollector-dbsync-processes - decoder/json - decoder/myjson - decoder/syscollector-hardware - decoder/rootcheck - decoder/syscollector-dbsync-osinfo-inserted - decoder/syscollector-process - decoder/syscollector-dbsync-hwinfo - decoder/syscollector-osinfo - decoder/syscollector-port-save root@engine:/home/vagrant/engine/wazuh/src/engine# ```

Load existent decoders :yellow_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog load -r decoder ruleset/decoders/ Error: [Catalog] Could not post content [decoder/queue-localfile/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-localfile/0] already exists Error: [Catalog] Could not post content [decoder/queue-dbsync/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-dbsync/0] already exists Error: [Catalog] Could not post content [decoder/dbsync/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/dbsync/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-osinfo/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-osinfo/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-hotfix/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-hotfix/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-ports-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-ports-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-ports/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-ports/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-address/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-address/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-iface/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-iface/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-hotfixes/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-hotfixes/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-osinfo/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-osinfo/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-processes/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-processes/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-osinfo-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-osinfo-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-protocol/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-protocol/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-iface-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-iface-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-hardware/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-hardware/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-process-save/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-process-save/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-hwinfo/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-hwinfo/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-address-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-address-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-netinfo/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-netinfo/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-program-save/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-program-save/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-base/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-base/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-network-ip/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-network-ip/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-network/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-network/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-program/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-program/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-base/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-base/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-packages/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-packages/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-hwinfo-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-hwinfo-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-processes-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-processes-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-process/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-process/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-process-del/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-process-del/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-packages-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-packages-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-port-del/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-port-del/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-network-end/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-network-end/0] already exists Error: [Catalog] Could not post content [decoder/queue-syscollector/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-syscollector/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-port/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-port/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-port-save/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-port-save/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-network-protocol-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-network-protocol-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-dbsync-hotfixes-inserted/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-dbsync-hotfixes-inserted/0] already exists Error: [Catalog] Could not post content [decoder/syscollector-program-del/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syscollector-program-del/0] already exists Error: [Catalog] Could not post content [decoder/queue-fim/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-fim/0] already exists Error: [Catalog] Could not post content [decoder/fim/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/fim/0] already exists Error: [Catalog] Could not post content [decoder/fim-scan/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/fim-scan/0] already exists Error: [Catalog] Could not post content [decoder/fim-event/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/fim-event/0] already exists Error: [Catalog] Could not post content [decoder/queue-rootcheck/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-rootcheck/0] already exists Error: [Catalog] Could not post content [decoder/rootcheck/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/rootcheck/0] already exists Error: [Catalog] Could not post content [decoder/sca/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/sca/0] already exists Error: [Catalog] Could not post content [decoder/queue-sca/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-sca/0] already exists Error: [Catalog] Could not post content [decoder/json/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/json/0] already exists Error: [Catalog] Could not post content [decoder/queue-syslog/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-syslog/0] already exists Error: [Catalog] Could not post content [decoder/syslog/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/syslog/0] already exists ``` > This has been reported in a previous testing, the logs could be more user-friendly

Recursive load: rule :green_circle:

Recursive load the same rule :green_circle:

When we have rules and more directories that also contains rules, we can recursively load them For example ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/rules/ custom valid-acc.yml root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/rules/custom/3/ 3.3 valid-acc.yml root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/rules/custom/3/3.3/ 3.3.1 valid-acc.yml root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/rules/custom/3/3.3/3.3.1/ valid-acc.yml ``` We can load them now by using the `-r` option ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog load -r rule ruleset/rules/ OK Error: [Catalog] Could not post content [rule/valid-accounts-local-accounts/1] in store, [FileDriver::add] File [/var/ossec/engine/store/rule/valid-accounts-local-accounts/1] already exists Error: [Catalog] Could not post content [rule/valid-accounts-local-accounts/1] in store, [FileDriver::add] File [/var/ossec/engine/store/rule/valid-accounts-local-accounts/1] already exists Error: [Catalog] Could not post content [rule/valid-accounts-local-accounts/1] in store, [FileDriver::add] File [/var/ossec/engine/store/rule/valid-accounts-local-accounts/1] already exists Error: [Catalog] Could not post content [rule/valid-accounts-local-accounts/1] in store, [FileDriver::add] File [/var/ossec/engine/store/rule/valid-accounts-local-accounts/1] already exists ``` > When we try to load the same rule placed in different dir levels we got that error.

dummy rule

```yaml name: rule/valid-accounts-local-accounts/1 metadata: - description: Adversaries may obtain and abuse credentials - detection: - DS0028 - threat: - tactic: TA0001 - technique: T1078.003 check: - event.module: logcollector - user.name: +ef_exists normalize: - check: - user.name: +ef_exists map: - indicator.description: A local account created a new session - indicator.type: user-account - indicator.reference: http://documentation.wazuh.com/ - indicator.provider: wazuh - indicator.confidence: High - indicator.matched.atomic: $.user.name - indicator.matched.field: user.name - indicator.matched.type: indicator_match_rule ```

Recursive load different rules :green_circle:

When we have rules and more directories that also contains rules, we can recursively load them For example ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/rules/ custom valid-acc.yml root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/rules/custom/3/ 3.3 valid-acc.yml root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/rules/custom/3/3.3/ 3.3.1 valid-acc.yml root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/rules/custom/3/3.3/3.3.1/ valid-acc.yml ``` Then, we can load them by using the `-r` option ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog load -r rule ruleset/rules/ OK OK OK OK OK root@engine:/home/vagrant/engine/wazuh/src/engine# ```

dummy rules

dummy rule 1

```yaml name: rule/valid-accounts-local-accounts/1 metadata: - description: Adversaries may obtain and abuse credentials - detection: - DS0028 - threat: - tactic: TA0001 - technique: T1078.003 check: - event.module: logcollector - user.name: +ef_exists normalize: - check: - user.name: +ef_exists map: - indicator.description: A local account created a new session - indicator.type: user-account - indicator.reference: http://documentation.wazuh.com/ - indicator.provider: wazuh - indicator.confidence: High - indicator.matched.atomic: $.user.name - indicator.matched.field: user.name - indicator.matched.type: indicator_match_rule ```

dummy rule 2

```yaml name: rule/valid-accounts-local-accounts/2 metadata: - description: Adversaries may obtain and abuse credentials - detection: - DS0028 - threat: - tactic: TA0001 - technique: T1078.003 check: - event.module: logcollector - user.name: +ef_exists normalize: - check: - user.name: +ef_exists map: - indicator.description: A local account created a new session - indicator.type: user-account - indicator.reference: http://documentation.wazuh.com/ - indicator.provider: wazuh - indicator.confidence: High - indicator.matched.atomic: $.user.name - indicator.matched.field: user.name - indicator.matched.type: indicator_match_rule ```

dummy rule 3

```yaml name: rule/valid-accounts-local-accounts/3 metadata: - description: Adversaries may obtain and abuse credentials - detection: - DS0028 - threat: - tactic: TA0001 - technique: T1078.003 check: - event.module: logcollector - user.name: +ef_exists normalize: - check: - user.name: +ef_exists map: - indicator.description: A local account created a new session - indicator.type: user-account - indicator.reference: http://documentation.wazuh.com/ - indicator.provider: wazuh - indicator.confidence: High - indicator.matched.atomic: $.user.name - indicator.matched.field: user.name - indicator.matched.type: indicator_match_rule ```

dummy rule 4

```yaml name: rule/valid-accounts-local-accounts/4 metadata: - description: Adversaries may obtain and abuse credentials - detection: - DS0028 - threat: - tactic: TA0001 - technique: T1078.003 check: - event.module: logcollector - user.name: +ef_exists normalize: - check: - user.name: +ef_exists map: - indicator.description: A local account created a new session - indicator.type: user-account - indicator.reference: http://documentation.wazuh.com/ - indicator.provider: wazuh - indicator.confidence: High - indicator.matched.atomic: $.user.name - indicator.matched.field: user.name - indicator.matched.type: indicator_match_rule ```

dummy rule 5

```yaml name: rule/valid-accounts-local-accounts/5 metadata: - description: Adversaries may obtain and abuse credentials - detection: - DS0028 - threat: - tactic: TA0001 - technique: T1078.003 check: - event.module: logcollector - user.name: +ef_exists normalize: - check: - user.name: +ef_exists map: - indicator.description: A local account created a new session - indicator.type: user-account - indicator.reference: http://documentation.wazuh.com/ - indicator.provider: wazuh - indicator.confidence: High - indicator.matched.atomic: $.user.name - indicator.matched.field: user.name - indicator.matched.type: indicator_match_rule ```

> The engine detects the rule as a new one (does not exists yet) if it hasn't the same name but the content is equal, do we want this?

Recursive load: output :green_circle:

Some custom output files have been placed in this directory tree (just rewriting the `name` field) ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/outputs/ custom file-output.yml root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/outputs/custom/3/ 3.1 3.2 3.3 root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/outputs/custom/3/3.3/ 3.3.1/ file-output.yml root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/outputs/custom/3/3.3/3.3.1/ file-output.yml ``` Recursive load ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog load -r output ruleset/outputs/ OK OK OK OK OK OK ```

Recursive load: environment :green_circle:

Some custom environments have been placed in this directory tree (just rewriting the `name` field) ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/environments/ custom wazuh-environment.yml root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/environments/custom/3/ 3.1 3.2 3.3 root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/environments/custom/3/3.3/ 3.3.1/ wazuh-environment.yml root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/environments/custom/3/3.3/3.3.1/ wazuh-environment.yml ``` Recursive load ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog load -r environment ruleset/environments/ OK OK OK OK OK OK ```

Recursive load: schema :yellow_circle:

We can split the current schema dir into dir levels. ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/schemas/ custom wazuh-asset.json wazuh-environments.json root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/schemas/custom/ 1 root@engine:/home/vagrant/engine/wazuh/src/engine# ls ruleset/schemas/custom/1 1.1 1.2 1.3 ``` Then, we load them ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog load -r schema ruleset/schemas/ Error: [Catalog] Content name not found OK Error: [Catalog] Content name not found Error: [Catalog] Content name not found Error: [Catalog] Could not post content [schema/wazuh-asset/0] in store, [FileDriver::add] File [/var/ossec/engine/store/schema/wazuh-asset/0] already exists ``` > The recursive load works as expected but there are only two schemas within the current `ruleset/schemas` dir and the `wazuh-environments.json` schema has no `name` field, so it fails. Besides, there are more files that are no schemas like `ecs_types.json`, so they won't load. > This has been discussed with the dev team and they will take care of this if it it necessary in the futur.

Recursive load: Load an empty path :green_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog load -r atype asd path: Directory does not exist: asd Run with --help for more information. root@engine:/home/vagrant/engine/wazuh/src/engine# ``` ``` root@engine:/home/vagrant/engine/wazuh/src/engine# ls asd ls: cannot access 'asd': No such file or directory ```

Recursive load: non-existent item-type :yellow_circle:

``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog load -r atype /tmp Invalid collection name: atype root@engine:/home/vagrant/engine/wazuh/src/engine# ``` > This log could be improved with `--help` reference and/or expected collection names

[roronoasins]

Conclusion

During the testing process we could test the following cases:

• Catalog load
  • -r option
  • decoder item-type
    • Load decoders recursively :green_circle:
    • Load existent decoders :yellow_circle:
  • rule item-type
    • Recursive load the same rule :green_circle:
    • Recursive load different rules :green_circle:
  • output item-type
    • Recursive load :green_circle:
  • environment item-type
    • Recursive load :green_circle:
  • schema item-type
    • Recursive load :yellow_circle:
  • Load an empty path :green_circle:
  • Load a non-existent item-type :yellow_circle:
  • --recursive option
  • decoder item-type
    • Load decoders recursively :green_circle:

Improvements

• Error logs that are no so friendly to the users

  • Error: [Catalog] Could not post content [decoder/queue-localfile/0] in store, [FileDriver::add] File [/var/ossec/engine/store/decoder/queue-localfile/0] already exists
  • Error: [Catalog] Content name not found
  • Invalid collection name: atype when no possible values are printed in the log.

  These logs appear for these cases:

  • Recursive load: decoder. Load existent decoders
  • Recursive load: schema
  • Recursive load: non-existent item-type

• Schemas directory contains files that are not ignored when you try to recursive load the directory.

  • Recursive load: schema

• Rules' versioning

  When we add different rules' versions, we are able to add multiple rules that are equals but with another name/version, is it desired? We may do not want that in a different version the rule is the same.

  This has been discussed with the dev team and it is expected.

• Repeated item error logs

  When you try to load an already existent item-type, this kind of log appears:

  Error: [Catalog] Could not post content [rule/valid-accounts-local-accounts/1] in store, [FileDriver::add] File [/var/ossec/engine/store/rule/valid-accounts-local-accounts/1] already exists 

  It should be more user-friendly and maybe contains the file path where the item-type file is located.