wazuh / wazuh-qa

Wazuh - Quality Assurance
GNU General Public License v2.0
65 stars 32 forks source link

Verify engine's API behavior #3530

Closed damarisg closed 1 year ago

damarisg commented 2 years ago
Target version Related issue Related PR/dev branch
5.0 https://github.com/wazuh/wazuh-qa/issues/3533 https://github.com/wazuh/wazuh/issues/11334

Description

Since the team is reworking the engine, we need to cover this new engine rework. This issue will test the new engine to ensure all is correct. The first two commands with their subcommand were tested in https://github.com/wazuh/wazuh-qa/issues/3475, now some of them like env are remaining

Proposed test cases

Considerations

roronoasins commented 2 years ago

2022/11/02

After a meeting with the dev team about API usage. We need to send the requests to the /var/ossec/queue/sockets/engine-api socket (unixgram) using some JSON format https://github.com/wazuh/wazuh/issues/5934#issuecomment-688380312.

As a first attempt to send these to the socket using Python:

INFO: Sending encoded event: b'\x88\x00\x00\x001:location:{"version": 1, "origin": {"name": "worker1","module": "api"},"command": "env","parameters": {"action": "get","name": "test"}}'

It seems like it is received:

root@engine:/home/vagrant/engine/wazuh/src/engine# nc -uUlk /var/ossec/queue/sockets/engine-api
�1:location:{"version": 1, "origin": {"name": "worker1","module": "api"},"command": "env","parameters": {"action": "get","name": "test"}}

But there is no API logging within the /tmp/engine.log. So something is missing or the unixgram socket is not receiving the message as expected.

roronoasins commented 2 years ago

2022/11/03

I worked trying to make this out, but I was not able to. This issue will be blocked until we have some way to test this

roronoasins commented 2 years ago

Check API behavior

graph command

Help message ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine graph --help Generates a dot description of an environment Usage: /var/ossec/engine/wazuh-engine graph [OPTIONS] Options: -h,--help Print this help message and exit -k,--kvdb_path TEXT:DIR [/var/ossec/etc/kvdb/] Path to KVDB folder. -f,--file_storage TEXT:DIR [/var/ossec/engine/store] Path to folder where assets are located. --environment TEXT [environment/wazuh/0] Environment name. -o,--output_dir TEXT [.] Directory to save graph files ``` > The `-k,--kvdb_path` and `-f,--file_storage` were not tested as this testing was performed in a mvp environment
Generate a graph file :green_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine graph --environment environment/wazuh/0 -k /var/ossec/etc/kvdb/ -f /var/ossec/engine/store/ -o /tmp/ 16:21:10.086790 hlp.cpp:73 DBG[5311 ] Invalid parser type [geo_point] for field [client.geo.location] 16:21:10.087124 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [container.labels] 16:21:10.087127 hlp.cpp:73 DBG[5311 ] Invalid parser type [geo_point] for field [destination.geo.location] 16:21:10.087135 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [dns.answers] 16:21:10.087137 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [email.attachments] 16:21:10.087145 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [faas.trigger] 16:21:10.087151 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [file.elf.exports] 16:21:10.087152 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [file.elf.imports] 16:21:10.087152 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [file.elf.sections] 16:21:10.087153 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [file.elf.segments] 16:21:10.087161 hlp.cpp:73 DBG[5311 ] Invalid parser type [geo_point] for field [host.geo.location] 16:21:10.087166 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [labels] 16:21:10.087167 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [log.syslog] 16:21:10.087168 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [log.syslog.structured_data] 16:21:10.087169 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [network.inner] 16:21:10.087170 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [observer.egress] 16:21:10.087171 hlp.cpp:73 DBG[5311 ] Invalid parser type [geo_point] for field [observer.geo.location] 16:21:10.087172 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [observer.ingress] 16:21:10.087185 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [process.elf.exports] 16:21:10.087186 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [process.elf.imports] 16:21:10.087186 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [process.elf.sections] 16:21:10.087188 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [process.elf.segments] 16:21:10.087205 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [process.entry_leader.tty] 16:21:10.087211 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [process.group_leader.tty] 16:21:10.087218 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [process.parent.elf.exports] 16:21:10.087221 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [process.parent.elf.imports] 16:21:10.087222 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [process.parent.elf.sections] 16:21:10.087223 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [process.parent.elf.segments] 16:21:10.087232 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [process.parent.tty] 16:21:10.087240 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [process.session_leader.tty] 16:21:10.087243 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [process.tty] 16:21:10.087247 hlp.cpp:73 DBG[5311 ] Invalid parser type [geo_point] for field [server.geo.location] 16:21:10.087260 hlp.cpp:73 DBG[5311 ] Invalid parser type [geo_point] for field [source.geo.location] 16:21:10.087265 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [threat.enrichments] 16:21:10.087265 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [threat.enrichments.indicator] 16:21:10.087269 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [threat.enrichments.indicator.file.elf.exports] 16:21:10.087271 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [threat.enrichments.indicator.file.elf.imports] 16:21:10.087271 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [threat.enrichments.indicator.file.elf.sections] 16:21:10.087272 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [threat.enrichments.indicator.file.elf.segments] 16:21:10.087284 hlp.cpp:73 DBG[5311 ] Invalid parser type [geo_point] for field [threat.enrichments.indicator.geo.location] 16:21:10.087307 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [threat.indicator.file.elf.exports] 16:21:10.087330 hlp.cpp:73 DBG[5311 ] Invalid parser type [object] for field [threat.indicator.file.elf.imports] 16:21:10.087330 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [threat.indicator.file.elf.sections] 16:21:10.087331 hlp.cpp:73 DBG[5311 ] Invalid parser type [nested] for field [threat.indicator.file.elf.segments] 16:21:10.087342 hlp.cpp:73 DBG[5311 ] Invalid parser type [geo_point] for field [threat.indicator.geo.location] Environment graph saved on /tmp/env_graph.dot Environment expression graph saved on /tmp/env_expr_graph.dot ``` > The invalid parser msg is expected as it is an mvp environment It generates a valid graphviz file:
/tmp/env_graph.dot file ``` digraph G { compound=true; fontname="Helvetica,Arial,sans-serif"; fontsize=12; node [fontname="Helvetica,Arial,sans-serif", fontsize=10]; edge [fontname="Helvetica,Arial,sans-serif", fontsize=8]; environment [label="environment/wazuh/0", shape=Mdiamond]; subgraph cluster_decoders { label="decoders"; style=filled; color=lightgrey; node [style=filled,color=white]; decoderqueuefim0 [label="decoder/queue-fim/0"]; decodersyscollectordbsynchotfixes0 [label="decoder/syscollector-dbsync-hotfixes/0"]; decodersyscollectordbsyncnetworkaddressinserted0 [label="decoder/syscollector-dbsync-network-address-inserted/0"]; decodersyscollectordbsyncpackagesinserted0 [label="decoder/syscollector-dbsync-packages-inserted/0"]; decodersyscollectordbsyncnetworkifaceinserted0 [label="decoder/syscollector-dbsync-network-iface-inserted/0"]; decodersyscollectordbsyncnetworkprotocolinserted0 [label="decoder/syscollector-dbsync-network-protocol-inserted/0"]; decodersyscollectordbsyncpackages0 [label="decoder/syscollector-dbsync-packages/0"]; decoderapacheaccess0 [label="decoder/apache-access/0"]; decodersyscollectorhardware0 [label="decoder/syscollector-hardware/0"]; decodersyscollectorhotfix0 [label="decoder/syscollector-hotfix/0"]; decodersyscollectordbsynchwinfo0 [label="decoder/syscollector-dbsync-hwinfo/0"]; decodersyscollectornetinfo0 [label="decoder/syscollector-netinfo/0"]; decoderdbsync0 [label="decoder/dbsync/0"]; decodersyslog0 [label="decoder/syslog/0"]; decodersyscollectorprocessdel0 [label="decoder/syscollector-process-del/0"]; decodersyscollectorprogram0 [label="decoder/syscollector-program/0"]; decodersyscollectordbsyncports0 [label="decoder/syscollector-dbsync-ports/0"]; decodersyscollectorprogramdel0 [label="decoder/syscollector-program-del/0"]; decodersyscollectorport0 [label="decoder/syscollector-port/0"]; decodersyscollectordbsynchwinfoinserted0 [label="decoder/syscollector-dbsync-hwinfo-inserted/0"]; decoderqueuesca0 [label="decoder/queue-sca/0"]; decodersyscollectordbsyncportsinserted0 [label="decoder/syscollector-dbsync-ports-inserted/0"]; decoderrootcheck0 [label="decoder/rootcheck/0"]; decodersyscollectorportsave0 [label="decoder/syscollector-port-save/0"]; decodersyscollectorportdel0 [label="decoder/syscollector-port-del/0"]; decodersyscollectorprogramsave0 [label="decoder/syscollector-program-save/0"]; decodersInput [label="decodersInput"]; decodersyscollectordbsyncnetworkiface0 [label="decoder/syscollector-dbsync-network-iface/0"]; decoderqueuesyslog0 [label="decoder/queue-syslog/0"]; decodersyscollectornetwork0 [label="decoder/syscollector-network/0"]; decodersyscollectorosinfo0 [label="decoder/syscollector-osinfo/0"]; decodersca0 [label="decoder/sca/0"]; decodersyscollectordbsyncnetworkaddress0 [label="decoder/syscollector-dbsync-network-address/0"]; decodersyscollectorprocess0 [label="decoder/syscollector-process/0"]; decodersyscollectornetworkip0 [label="decoder/syscollector-network-ip/0"]; decodersyscollectornetworkend0 [label="decoder/syscollector-network-end/0"]; decoderfimscan0 [label="decoder/fim-scan/0"]; decoderqueuerootcheck0 [label="decoder/queue-rootcheck/0"]; decoderjson0 [label="decoder/json/0"]; decoderqueuelocalfile0 [label="decoder/queue-localfile/0"]; decodersyscollectorprocesssave0 [label="decoder/syscollector-process-save/0"]; decoderqueuedbsync0 [label="decoder/queue-dbsync/0"]; decodersyscollectordbsyncosinfoinserted0 [label="decoder/syscollector-dbsync-osinfo-inserted/0"]; decoderfim0 [label="decoder/fim/0"]; decodersyscollectorbase0 [label="decoder/syscollector-base/0"]; decodersyscollectordbsyncnetworkprotocol0 [label="decoder/syscollector-dbsync-network-protocol/0"]; decodersyscollectordbsyncbase0 [label="decoder/syscollector-dbsync-base/0"]; decoderqueuesyscollector0 [label="decoder/queue-syscollector/0"]; decoderfimevent0 [label="decoder/fim-event/0"]; decodersyscollectordbsyncosinfo0 [label="decoder/syscollector-dbsync-osinfo/0"]; decodersyscollectordbsyncprocesses0 [label="decoder/syscollector-dbsync-processes/0"]; decodersyscollectordbsynchotfixesinserted0 [label="decoder/syscollector-dbsync-hotfixes-inserted/0"]; decodersyscollectordbsyncprocessesinserted0 [label="decoder/syscollector-dbsync-processes-inserted/0"]; decoderqueuefim0 -> decoderfim0; decoderqueuelocalfile0 -> decodersyslog0; decoderqueuelocalfile0 -> decoderapacheaccess0; decoderqueuelocalfile0 -> decoderjson0; decodersyscollectorprogram0 -> decodersyscollectorprogramdel0; decodersyscollectorprogram0 -> decodersyscollectorprogramsave0; decodersyscollectorbase0 -> decodersyscollectorprogram0; decodersyscollectorbase0 -> decodersyscollectorport0; decodersyscollectorbase0 -> decodersyscollectorosinfo0; decodersyscollectorbase0 -> decodersyscollectornetinfo0; decodersyscollectorbase0 -> decodersyscollectorhotfix0; decodersyscollectorbase0 -> decodersyscollectorhardware0; decodersyscollectorbase0 -> decodersyscollectorprocess0; decodersyscollectorbase0 -> decodersyscollectordbsyncbase0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncpackages0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncnetworkifaceinserted0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncnetworkiface0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsynchwinfo0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncnetworkaddress0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncnetworkprotocol0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncportsinserted0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncnetworkprotocolinserted0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncosinfo0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncosinfoinserted0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncports0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsynchotfixesinserted0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncprocesses0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncprocessesinserted0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncpackagesinserted0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsynchotfixes0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsynchwinfoinserted0; decodersyscollectordbsyncbase0 -> decodersyscollectordbsyncnetworkaddressinserted0; decoderqueuesyslog0 -> decodersyslog0; decoderqueuesyslog0 -> decoderjson0; decoderqueuesca0 -> decodersca0; decoderqueuedbsync0 -> decoderdbsync0; decodersyscollectorport0 -> decodersyscollectorportsave0; decodersyscollectorport0 -> decodersyscollectorportdel0; decodersyscollectornetwork0 -> decodersyscollectornetworkip0; decodersyscollectornetinfo0 -> decodersyscollectornetwork0; decodersyscollectornetinfo0 -> decodersyscollectornetworkend0; decoderqueuerootcheck0 -> decoderrootcheck0; decodersyscollectorprocess0 -> decodersyscollectorprocessdel0; decodersyscollectorprocess0 -> decodersyscollectorprocesssave0; decoderfim0 -> decoderfimevent0; decoderfim0 -> decoderfimscan0; decoderqueuesyscollector0 -> decodersyscollectorbase0; decodersInput -> decoderqueuesca0; decodersInput -> decoderqueuesyslog0; decodersInput -> decoderqueuedbsync0; decodersInput -> decoderqueuerootcheck0; decodersInput -> decoderqueuelocalfile0; decodersInput -> decoderqueuesyscollector0; decodersInput -> decoderqueuefim0; } environment -> decodersInput; subgraph cluster_outputs { label="outputs"; style=filled; color=lightgrey; node [style=filled,color=white]; outputfileoutput0 [label="output/file-output/0"]; outputsInput [label="outputsInput"]; outputsInput -> outputfileoutput0; } environment -> outputsInput; } ```
Generate a graph file using a non-valid environment :yellow_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine graph --environment my-env 14:44:31.406082 hlp.cpp:73 DBG[4089 ] Invalid parser type [geo_point] for field [client.geo.location] 14:44:31.406414 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [container.labels] 14:44:31.406417 hlp.cpp:73 DBG[4089 ] Invalid parser type [geo_point] for field [destination.geo.location] 14:44:31.406425 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [dns.answers] 14:44:31.406428 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [email.attachments] 14:44:31.406435 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [faas.trigger] 14:44:31.406440 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [file.elf.exports] 14:44:31.406441 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [file.elf.imports] 14:44:31.406441 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [file.elf.sections] 14:44:31.406443 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [file.elf.segments] 14:44:31.406451 hlp.cpp:73 DBG[4089 ] Invalid parser type [geo_point] for field [host.geo.location] 14:44:31.406456 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [labels] 14:44:31.406456 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [log.syslog] 14:44:31.406457 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [log.syslog.structured_data] 14:44:31.406459 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [network.inner] 14:44:31.406460 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [observer.egress] 14:44:31.406461 hlp.cpp:73 DBG[4089 ] Invalid parser type [geo_point] for field [observer.geo.location] 14:44:31.406462 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [observer.ingress] 14:44:31.406469 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [process.elf.exports] 14:44:31.406470 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [process.elf.imports] 14:44:31.406470 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [process.elf.sections] 14:44:31.406472 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [process.elf.segments] 14:44:31.406484 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [process.entry_leader.tty] 14:44:31.406488 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [process.group_leader.tty] 14:44:31.406492 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [process.parent.elf.exports] 14:44:31.406495 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [process.parent.elf.imports] 14:44:31.406495 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [process.parent.elf.sections] 14:44:31.406496 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [process.parent.elf.segments] 14:44:31.406502 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [process.parent.tty] 14:44:31.406509 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [process.session_leader.tty] 14:44:31.406512 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [process.tty] 14:44:31.406516 hlp.cpp:73 DBG[4089 ] Invalid parser type [geo_point] for field [server.geo.location] 14:44:31.406527 hlp.cpp:73 DBG[4089 ] Invalid parser type [geo_point] for field [source.geo.location] 14:44:31.406532 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [threat.enrichments] 14:44:31.406532 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [threat.enrichments.indicator] 14:44:31.406536 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [threat.enrichments.indicator.file.elf.exports] 14:44:31.406537 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [threat.enrichments.indicator.file.elf.imports] 14:44:31.406537 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [threat.enrichments.indicator.file.elf.sections] 14:44:31.406539 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [threat.enrichments.indicator.file.elf.segments] 14:44:31.406550 hlp.cpp:73 DBG[4089 ] Invalid parser type [geo_point] for field [threat.enrichments.indicator.geo.location] 14:44:31.406567 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [threat.indicator.file.elf.exports] 14:44:31.406585 hlp.cpp:73 DBG[4089 ] Invalid parser type [object] for field [threat.indicator.file.elf.imports] 14:44:31.406585 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [threat.indicator.file.elf.sections] 14:44:31.406587 hlp.cpp:73 DBG[4089 ] Invalid parser type [nested] for field [threat.indicator.file.elf.segments] 14:44:31.406595 hlp.cpp:73 DBG[4089 ] Invalid parser type [geo_point] for field [threat.indicator.geo.location] 14:44:31.406714 cmdGraph.cpp:77 ERR[4089 ] Exception while building environment: [exception: [Environment] Error retreiving environment [my-env] from store: [FileDriver] File [/var/ossec/engine/store/my-env] does not exist ] ``` > The logging could be improved, so it is more user-friendly. It can print the format expected, `environment/env-id/version`. And/or the help message reference.
Generate a graph file specifying an output path with a directory level that does not exist :red_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine graph -o /tmp/another_level 14:37:09.353423 hlp.cpp:73 DBG[4043 ] Invalid parser type [geo_point] for field [client.geo.location] 14:37:09.353745 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [container.labels] 14:37:09.353748 hlp.cpp:73 DBG[4043 ] Invalid parser type [geo_point] for field [destination.geo.location] 14:37:09.353756 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [dns.answers] 14:37:09.353758 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [email.attachments] 14:37:09.353764 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [faas.trigger] 14:37:09.353770 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [file.elf.exports] 14:37:09.353771 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [file.elf.imports] 14:37:09.353771 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [file.elf.sections] 14:37:09.353772 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [file.elf.segments] 14:37:09.353780 hlp.cpp:73 DBG[4043 ] Invalid parser type [geo_point] for field [host.geo.location] 14:37:09.353785 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [labels] 14:37:09.353785 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [log.syslog] 14:37:09.353786 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [log.syslog.structured_data] 14:37:09.353787 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [network.inner] 14:37:09.353788 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [observer.egress] 14:37:09.353789 hlp.cpp:73 DBG[4043 ] Invalid parser type [geo_point] for field [observer.geo.location] 14:37:09.353790 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [observer.ingress] 14:37:09.353797 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [process.elf.exports] 14:37:09.353798 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [process.elf.imports] 14:37:09.353798 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [process.elf.sections] 14:37:09.353800 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [process.elf.segments] 14:37:09.353811 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [process.entry_leader.tty] 14:37:09.353815 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [process.group_leader.tty] 14:37:09.353819 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [process.parent.elf.exports] 14:37:09.353821 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [process.parent.elf.imports] 14:37:09.353821 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [process.parent.elf.sections] 14:37:09.353822 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [process.parent.elf.segments] 14:37:09.353828 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [process.parent.tty] 14:37:09.353835 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [process.session_leader.tty] 14:37:09.353838 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [process.tty] 14:37:09.353843 hlp.cpp:73 DBG[4043 ] Invalid parser type [geo_point] for field [server.geo.location] 14:37:09.353852 hlp.cpp:73 DBG[4043 ] Invalid parser type [geo_point] for field [source.geo.location] 14:37:09.353856 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [threat.enrichments] 14:37:09.353856 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [threat.enrichments.indicator] 14:37:09.353860 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [threat.enrichments.indicator.file.elf.exports] 14:37:09.353861 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [threat.enrichments.indicator.file.elf.imports] 14:37:09.353862 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [threat.enrichments.indicator.file.elf.sections] 14:37:09.353863 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [threat.enrichments.indicator.file.elf.segments] 14:37:09.353873 hlp.cpp:73 DBG[4043 ] Invalid parser type [geo_point] for field [threat.enrichments.indicator.geo.location] 14:37:09.353889 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [threat.indicator.file.elf.exports] 14:37:09.353907 hlp.cpp:73 DBG[4043 ] Invalid parser type [object] for field [threat.indicator.file.elf.imports] 14:37:09.353907 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [threat.indicator.file.elf.sections] 14:37:09.353908 hlp.cpp:73 DBG[4043 ] Invalid parser type [nested] for field [threat.indicator.file.elf.segments] 14:37:09.353916 hlp.cpp:73 DBG[4043 ] Invalid parser type [geo_point] for field [threat.indicator.geo.location] Environment graph saved on /tmp/another_level/env_graph.dot Environment expression graph saved on /tmp/another_level/env_expr_graph.dot ``` Even when it says that it has been saved, the middle-directory was not created ``` root@engine:/home/vagrant# ls /tmp/ engine.log systemd-private-4c6170f23c2447eb8be2c5d4199e134b-haveged.service-gMuwag systemd-private-4c6170f23c2447eb8be2c5d4199e134b-systemd-logind.service-qDysug ssh-drlYpQQuiw systemd-private-4c6170f23c2447eb8be2c5d4199e134b-ModemManager.service-B7Y4Ki systemd-private-4c6170f23c2447eb8be2c5d4199e134b-systemd-resolved.service-F19Gmf root@engine:/home/vagrant# ```
Generate a graph file specifying an output path with a directory that does not exist :red_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine graph -o /dende 14:39:45.534220 hlp.cpp:73 DBG[4050 ] Invalid parser type [geo_point] for field [client.geo.location] 14:39:45.534573 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [container.labels] 14:39:45.534575 hlp.cpp:73 DBG[4050 ] Invalid parser type [geo_point] for field [destination.geo.location] 14:39:45.534588 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [dns.answers] 14:39:45.534591 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [email.attachments] 14:39:45.534601 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [faas.trigger] 14:39:45.534608 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [file.elf.exports] 14:39:45.534610 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [file.elf.imports] 14:39:45.534610 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [file.elf.sections] 14:39:45.534612 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [file.elf.segments] 14:39:45.534625 hlp.cpp:73 DBG[4050 ] Invalid parser type [geo_point] for field [host.geo.location] 14:39:45.534633 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [labels] 14:39:45.534634 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [log.syslog] 14:39:45.534635 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [log.syslog.structured_data] 14:39:45.534637 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [network.inner] 14:39:45.534639 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [observer.egress] 14:39:45.534640 hlp.cpp:73 DBG[4050 ] Invalid parser type [geo_point] for field [observer.geo.location] 14:39:45.534641 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [observer.ingress] 14:39:45.534649 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [process.elf.exports] 14:39:45.534650 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [process.elf.imports] 14:39:45.534650 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [process.elf.sections] 14:39:45.534652 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [process.elf.segments] 14:39:45.534672 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [process.entry_leader.tty] 14:39:45.534678 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [process.group_leader.tty] 14:39:45.534684 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [process.parent.elf.exports] 14:39:45.534687 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [process.parent.elf.imports] 14:39:45.534687 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [process.parent.elf.sections] 14:39:45.534689 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [process.parent.elf.segments] 14:39:45.534698 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [process.parent.tty] 14:39:45.534706 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [process.session_leader.tty] 14:39:45.534709 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [process.tty] 14:39:45.534713 hlp.cpp:73 DBG[4050 ] Invalid parser type [geo_point] for field [server.geo.location] 14:39:45.534724 hlp.cpp:73 DBG[4050 ] Invalid parser type [geo_point] for field [source.geo.location] 14:39:45.534729 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [threat.enrichments] 14:39:45.534729 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [threat.enrichments.indicator] 14:39:45.534734 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [threat.enrichments.indicator.file.elf.exports] 14:39:45.534735 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [threat.enrichments.indicator.file.elf.imports] 14:39:45.534735 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [threat.enrichments.indicator.file.elf.sections] 14:39:45.534736 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [threat.enrichments.indicator.file.elf.segments] 14:39:45.534747 hlp.cpp:73 DBG[4050 ] Invalid parser type [geo_point] for field [threat.enrichments.indicator.geo.location] 14:39:45.534764 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [threat.indicator.file.elf.exports] 14:39:45.534789 hlp.cpp:73 DBG[4050 ] Invalid parser type [object] for field [threat.indicator.file.elf.imports] 14:39:45.534790 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [threat.indicator.file.elf.sections] 14:39:45.534791 hlp.cpp:73 DBG[4050 ] Invalid parser type [nested] for field [threat.indicator.file.elf.segments] 14:39:45.534804 hlp.cpp:73 DBG[4050 ] Invalid parser type [geo_point] for field [threat.indicator.geo.location] Environment graph saved on /dende/env_graph.dot Environment expression graph saved on /dende/env_expr_graph.dot ``` Even when it says taht it has been saved, the directory was not created ``` root@engine:/home/vagrant# ls / bin boot dev etc home lib lib32 lib64 libx32 lost+found media mnt opt proc root run sbin snap srv sys tmp usr var root@engine:/home/vagrant# ls /dende ls: cannot access '/dende': No such file or directory ``` > Has been tested using the path ending with `/`, just in case it is sensitive to this case as in previous testing.

env command

Help message ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine env --help Operates the running environments Usage: /var/ossec/engine/wazuh-engine env [OPTIONS] SUBCOMMAND Options: -h,--help Print this help message and exit -a,--api_socket TEXT [/var/ossec/queue/sockets/engine-api] engine api address Subcommands: get get: Get active environments. set set [environment]: Set active environments. delete delete [environment]: Delete an environment. ``` > The `-a,--api_socket` parameter was not tested as this testing was performed in a mvp environment
get subcommand 🟡
Get current env :green_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine env get Active environment: environment/wazuh/0 ```
Get current env when there is no env set 🟡 ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine env get No active environments found root@engine:/home/vagrant/engine/wazuh/src/engine# ```
set subcommand :yellow_circle:
Set without any name :yellow_circle: ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine env set environment is required Run with --help for more information. ``` > The message could be improved, adding the proper way to run the command.
Set with a env name that does not exist :yellow_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine env set environment/wazuh/2 Error building environment [environment/wazuh/2]: exception: [Environment] Error retreiving environment [environment/wazuh/2] from store: [FileDriver] File [/var/ossec/engine/store/environment/wazuh/2] does not exist root@engine:/home/vagrant# ``` > The logging could be improved so it is more user-friendly, instead of talking about the `store` and files.
Set with a not supported environment format :yellow_circle: ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine env set env Invalid environment name envroot@engine:/home/vagrant/engine/wazuh/src/engine# ``` > The behavior is expected, but the message format requires a newline, it has two spaces and it misses talking about the format. The expected format could be added and/or a help message reference.
Set an existent environment :yellow_circle: We create a sample environment defined in `ruleset/environments/custom-environment.yml`. Then, it is set. ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine catalog create environment < ruleset/environments/custom-environment.yml OK root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine env set environment/wazuh/1 Environment created and startedroot@engine:/home/vagrant/engine/wazuh/src/engine# root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine env get Active environment: environment/wazuh/1 ``` > A newline is missing when setting an environment
Set the environment that is already active :yellow_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine env set environment/wazuh/0 Environment already existsroot@engine:/home/vagrant# ``` > A newline is missing. Also, it may confuse the user saying that the env already `exists`. Because the tool always says `active` instead.
delete subcommand :yellow_circle:
Invoke delete without an env name :yellow_circle: ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine env delete environment is required Run with --help for more information. ``` > We could improve this message, adding the proper way to run the subcommand, and a help command suggestion Also, the help message may confuse the users. It uses the notation as default or optional values used in other commands ``` set set [environment]: Set active environments. delete delete [environment]: Delete an environment. ``` The user can think that environment is the default value and that it can be optional(for the delete subcommand, for example)
Delete the current env :yellow_circle: ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine env delete environment/wazuh/1 Environment deletedroot@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine env get No active environments found root@engine:/home/vagrant/engine/wazuh/src/engine# ``` > The behavior is expected, but the message format requires a newline and could be improved/extended.
Delete the current environment when it has not been set yet. :yellow_circle: ``` root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine env get No active environments found root@engine:/home/vagrant/engine/wazuh/src/engine# /var/ossec/engine/wazuh-engine env delete environment/wazuh/1 Environment does not existroot@engine:/home/vagrant/engine/wazuh/src/engine# ``` > The behavior is expected, but the message format requires a newline and could be improved/extended.
roronoasins commented 2 years ago

2022/11/07

Completed test command testing. Gathered all logs and cases, and tomorrow I will post the markdown message.

roronoasins commented 1 year ago

Check API behavior

test command

Help message ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test --help Utility to test the ruleset Usage: /var/ossec/engine/wazuh-engine test [OPTIONS] Options: -h,--help Print this help message and exit -k,--kvdb_path TEXT:DIR [/var/ossec/etc/kvdb/] Path to KVDB folder. -f,--file_storage TEXT:DIR [/var/ossec/engine/store] Path to folder where assets are located. --environment TEXT [environment/wazuh/0] Environment name. -q,--protocol_queue CHAR [1] Protocol queue number of the event. -l,--protocol_location TEXT [/dev/stdin] Protocol location. --log_level INT [3] Log level. 0 = Debug, 1 = Info, 2 = Warning, 3 = Error -d,--debug [0] Enable debug mode [0-2]. Flag can appear multiple times. No flag[0]: No debug, d[1]: Asset history, dd[2]: 1 + Full tracing. -t,--trace TEXT [ALL] Needs: --debug Assets to be traced, separated by commas. Only effective if debug=2. ```
--kvdb-path option :red_circle:
Dir that is not kvdb path :yellow_circle: The testing tool runs normally, but is this desired? When a rule uses KVDB it won't work. > It may log a warning so the user can know that the specified path does not contain a kvdb at all. So they can now that some rules that use the kvdb won't work.
testsubcommand does not accept kvdb paths without an ending / :red_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -k /var/ossec/etc/kvdb -dd --environment environment/wazuh/1 16:17:50.613326 kvdb.cpp:88 ERR[7529 ] Couldn't create DB [malicious-ip] file, error: [Invalid argument: /var/ossec/etc/kvdbmalicious-ip/CURRENT: does not exist (create_if_missing is false)] 16:17:50.613985 kvdb.cpp:88 ERR[7529 ] Couldn't create DB [malicious-ip] file, error: [Invalid argument: /var/ossec/etc/kvdbmalicious-ip/CURRENT: does not exist (create_if_missing is false)] 16:17:50.614085 kvdbManager.cpp:143 ERR[7529 ] Error initializing db [malicious-ip]. 16:17:50.614949 cmdTest.cpp:152 ERR[7529 ] Exception while building environment: [exception: Failed to build asset: rule/source-malicious-ip/0 exception: [Asset::Asset(jsonDefinition, type)] failed to build stage check exception: [builders::operationBuilder()] Exception building helper [kvdb_match] exception: [malicious-ip] DB isn't available for usage ] ```
testsubcommand accepts kvdb paths with an ending / :red_circle: The testing feature works as expected. ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -k /var/ossec/etc/kvdb/ -dd --environment environment/wazuh/1 Enter log in single line (Crtl+C to exit): ``` > It is marked as red because ending the path with an `\` it does not work.
Running test with a non-existent kvdb path :yellow_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -k /apath -dd --environment environment/wazuh/1 --kvdb_path: Directory does not exist: /apath Run with --help for more information. root@engine:/home/vagrant# running test with a non-valid kvdb path root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -k /tmp/ -dd --environment environment/wazuh/1 16:16:12.310526 kvdb.cpp:88 ERR[7510 ] Couldn't create DB [malicious-ip] file, error: [Invalid argument: /tmp/malicious-ip/CURRENT: does not exist (create_if_missing is false)] 16:16:12.311237 kvdb.cpp:88 ERR[7510 ] Couldn't create DB [malicious-ip] file, error: [Invalid argument: /tmp/malicious-ip/CURRENT: does not exist (create_if_missing is false)] 16:16:12.311352 kvdbManager.cpp:143 ERR[7510 ] Error initializing db [malicious-ip]. 16:16:12.312183 cmdTest.cpp:152 ERR[7510 ] Exception while building environment: [exception: Failed to build asset: rule/source-malicious-ip/0 exception: [Asset::Asset(jsonDefinition, type)] failed to build stage check exception: [builders::operationBuilder()] Exception building helper [kvdb_match] exception: [malicious-ip] DB isn't available for usage ] root@engine:/home/vagrant# ``` > The loggin could me improved so the user can know the reason because it is failing. Maybe `available` is not the perfect term so the users can understand it quickly.
-f,--file_storage option :yellow_circle:
Use a non-existent path :green_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test --trace decoder/apache-access/0 --environment environment/wazuh/1 -dd -f /nopath --file_storage: Directory does not exist: /nopath Run with --help for more information. root@engine:/home/vagrant# ```
Use a path where no assets are located :yellow_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test --trace decoder/apache-access/0 --environment environment/wazuh/1 -dd -f /tmp 15:55:35.921218 cmdTest.cpp:74 ERR[3482 ] Could not retreive configuration file [schema/wazuh-logpar-types/0] needed by the HLP module, error: [FileDriver] File [/tmp/schema/wazuh-logpar-types/0] does not exist root@engine:/home/vagrant# ``` > The logging could be improved, so the user can understand where is the problem
--environment option :yellow_circle:
Running test with a non-existent environment :yellow_circle: ``` non-existent environment root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -k /var/ossec/etc/kvdb/ -dd --environment environment/wazuh/3 16:21:01.995793 cmdTest.cpp:113 ERR[7557 ] Error while getting environment definition: [[FileDriver] File [/var/ossec/engine/store/environment/wazuh/3] does not exist] root@engine:/home/vagrant# ``` > The logging could be improved, so the user can understand where is the problem
Run test with different environments: one with a loaded rule and other without it :yellow_circle: In the first one we can see how with the environment that has no rules, they does not appear. ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -k /var/ossec/etc/kvdb/ -dd --environment environment/wazuh/0 Enter log in single line (Crtl+C to exit): 192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] "GET /someicon.ico HTTP/1.1" 404 209 DECODERS: decoder/queue-sca/0 -> failure [decoder/queue-sca/0] condition.value[/wazuh/queue==112] -> Failure [decoder/queue-sca/0] [condition]:failure decoder/queue-syslog/0 -> failure [decoder/queue-syslog/0] condition.value[/wazuh/queue==50] -> Failure [decoder/queue-syslog/0] [condition]:failure decoder/queue-dbsync/0 -> failure [decoder/queue-dbsync/0] condition.value[/wazuh/queue==53] -> Failure [decoder/queue-dbsync/0] [condition]:failure decoder/queue-rootcheck/0 -> failure [decoder/queue-rootcheck/0] condition.value[/wazuh/queue==57] -> Failure [decoder/queue-rootcheck/0] [condition]:failure decoder/queue-localfile/0 -> success [decoder/queue-localfile/0] condition.value[/wazuh/queue==49] -> Success [decoder/queue-localfile/0] [condition]:success decoder/syslog/0 -> failure [decoder/syslog/0] [/event/original: []:<~/ignore/ >] -> Failure: Parser trace: Parser["event.start"] failure [decoder/syslog/0] [/event/original: :<~/ignore/ >] -> Failure: Parser trace: Parser["event.start"] failure [decoder/syslog/0] [/event/original: []: ] -> Failure: Parser trace: Parser["event.start"] failure [decoder/syslog/0] [/event/original: : ] -> Failure: Parser trace: Parser["event.start"] failure [decoder/syslog/0] [condition]:failure decoder/apache-access/0 -> success [decoder/apache-access/0] [helper.ef_exists[/event/original]] -> Success [decoder/apache-access/0] /event/original: - - [] " HTTP/" -> Success [decoder/apache-access/0] [condition]:success [decoder/apache-access/0] map.value[/event/kind="event"] -> Success [decoder/apache-access/0] map.value[/event/dataset="apache.access"] -> Success [decoder/apache-access/0] [helper.a_append[/event/category, web]] -> Success [decoder/apache-access/0] map.value[/event/module="apache"] -> Success [decoder/apache-access/0] map.value[/service/type="apache"] -> Success [decoder/apache-access/0] map.value[/event/outcome="success"] -> Success [decoder/apache-access/0] [helper.i_gt[/http/response/status_code, 399]] -> Success [decoder/apache-access/0] map.value[/event/outcome="failure"] -> Success RULES: OUTPUT: { "wazuh": { "queue": 49, "origin": "/dev/stdin" }, "event": { "original": "192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] \"GET /someicon.ico HTTP/1.1\" 404 209", "created": "1992-07-10T10:07:13.000Z", "kind": "event", "dataset": "apache.access", "category": [ "web" ], "module": "apache", "outcome": "failure" }, "http": { "response": { "status_code": 404, "body": { "bytes": 209 } }, "version": "1.1", "request": { "method": "GET" } }, "url": { "original": "/someicon.ico" }, "source": { "ip": "192.168.1.10" }, "service": { "type": "apache" } } ``` And using the environment with the rule: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -k /var/ossec/etc/kvdb/ -dd --environment environment/wazuh/1 Enter log in single line (Crtl+C to exit): 192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] "GET /someicon.ico HTTP/1.1" 404 209 DECODERS: decoder/queue-sca/0 -> failure [decoder/queue-sca/0] condition.value[/wazuh/queue==112] -> Failure [decoder/queue-sca/0] [condition]:failure decoder/queue-syslog/0 -> failure [decoder/queue-syslog/0] condition.value[/wazuh/queue==50] -> Failure [decoder/queue-syslog/0] [condition]:failure decoder/queue-dbsync/0 -> failure [decoder/queue-dbsync/0] condition.value[/wazuh/queue==53] -> Failure [decoder/queue-dbsync/0] [condition]:failure decoder/queue-rootcheck/0 -> failure [decoder/queue-rootcheck/0] condition.value[/wazuh/queue==57] -> Failure [decoder/queue-rootcheck/0] [condition]:failure decoder/queue-localfile/0 -> success [decoder/queue-localfile/0] condition.value[/wazuh/queue==49] -> Success [decoder/queue-localfile/0] [condition]:success decoder/syslog/0 -> failure [decoder/syslog/0] [/event/original: []:<~/ignore/ >] -> Failure: Parser trace: Parser["event.start"] failure [decoder/syslog/0] [/event/original: :<~/ignore/ >] -> Failure: Parser trace: Parser["event.start"] failure [decoder/syslog/0] [/event/original: []: ] -> Failure: Parser trace: Parser["event.start"] failure [decoder/syslog/0] [/event/original: : ] -> Failure: Parser trace: Parser["event.start"] failure [decoder/syslog/0] [condition]:failure decoder/apache-access/0 -> success [decoder/apache-access/0] [helper.ef_exists[/event/original]] -> Success [decoder/apache-access/0] /event/original: - - [] " HTTP/" -> Success [decoder/apache-access/0] [condition]:success [decoder/apache-access/0] map.value[/event/kind="event"] -> Success [decoder/apache-access/0] map.value[/event/dataset="apache.access"] -> Success [decoder/apache-access/0] [helper.a_append[/event/category, web]] -> Success [decoder/apache-access/0] map.value[/event/module="apache"] -> Success [decoder/apache-access/0] map.value[/service/type="apache"] -> Success [decoder/apache-access/0] map.value[/event/outcome="success"] -> Success [decoder/apache-access/0] [helper.i_gt[/http/response/status_code, 399]] -> Success [decoder/apache-access/0] map.value[/event/outcome="failure"] -> Success RULES: rule/source-malicious-ip/0 -> success [rule/source-malicious-ip/0] [helper.ef_exists[/source/ip]] -> Success [rule/source-malicious-ip/0] [helper./source/ip[kvdb_match, malicious-ip]] -> Success [rule/source-malicious-ip/0] [condition]:success [rule/source-malicious-ip/0] map.value[/rule/id="source-malicious-ip"] -> Success [rule/source-malicious-ip/0] map.value[/rule/author="Wazuh"] -> Success [rule/source-malicious-ip/0] map.value[/rule/description="Detected malicious IP activity from source"] -> Success [rule/source-malicious-ip/0] map.reference[/~0threat/indicator/ip=/source/ip] -> Success OUTPUT: { "wazuh": { "queue": 49, "origin": "/dev/stdin" }, "event": { "original": "192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] \"GET /someicon.ico HTTP/1.1\" 404 209", "created": "1992-07-10T10:07:13.000Z", "kind": "event", "dataset": "apache.access", "category": [ "web" ], "module": "apache", "outcome": "failure" }, "http": { "response": { "status_code": 404, "body": { "bytes": 209 } }, "version": "1.1", "request": { "method": "GET" } }, "url": { "original": "/someicon.ico" }, "source": { "ip": "192.168.1.10" }, "service": { "type": "apache" }, "rule": { "id": "source-malicious-ip", "author": "Wazuh", "description": "Detected malicious IP activity from source" }, "~threat": { "indicator": { "ip": "192.168.1.10" } } } Enter log in single line (Crtl+C to exit): ^C root@engine:/home/vagrant ``` > In case that no rules are triggered, it could tell that no rules has been triggered.
source-malicious-ip rule ```yaml name: rule/source-malicious-ip/0 check: - source.ip: +ef_exists - source.ip: +kvdb_match/malicious-ip normalize: - map: - rule.id: source-malicious-ip - rule.author: Wazuh - rule.description: Detected malicious IP activity from source - ~threat.indicator.ip: $source.ip ```
Setting a custom protocol queue option :yellow_circle: The queue option accepts -128 to 127 ascii decimal value characters as reported in this issue: https://github.com/wazuh/wazuh-qa/issues/3475#issuecomment-1292339045 ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -k /var/ossec/etc/kvdb/ -dd --environment environment/wazuh/1 -q 128 Could not convert: --protocol_queue = 128 Run with --help for more information. root@engine:/home/vagrant# ``` > This logging could be improved so the users can understand which values can they use
Setting a custom protocol location option :red_circle: The location option has the same bug reported in https://github.com/wazuh/wazuh-qa/issues/3475#issuecomment-1292339045 ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -k /var/ossec/etc/kvdb/ --environment environment/wazuh/1 -l location:extra Enter log in single line (Crtl+C to exit): sample log OUTPUT: { "wazuh": { "queue": 49, "origin": "location" }, "event": { "original": "extra:sample log" } } Enter log in single line (Crtl+C to exit): ``` > The `original` field should have only the log
Verifying debug mode :green_circle:
Help message ``` -d,--debug [0] Enable debug mode [0-2]. Flag can appear multiple times. No flag[0]: No debug, d[1]: Asset history, dd[2]: 1 + Full tracing. ```
No debug :green_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -k /var/ossec/etc/kvdb/ --environment environment/wazuh/1 Enter log in single line (Crtl+C to exit): 192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] "GET /someicon.ico HTTP/1.1" 404 209 OUTPUT: { "wazuh": { "queue": 49, "origin": "/dev/stdin" }, "event": { "original": "192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] \"GET /someicon.ico HTTP/1.1\" 404 209", "created": "1992-07-10T10:07:13.000Z", "kind": "event", "dataset": "apache.access", "category": [ "web" ], "module": "apache", "outcome": "failure" }, "http": { "response": { "status_code": 404, "body": { "bytes": 209 } }, "version": "1.1", "request": { "method": "GET" } }, "url": { "original": "/someicon.ico" }, "source": { "ip": "192.168.1.10" }, "service": { "type": "apache" }, "rule": { "id": "source-malicious-ip", "author": "Wazuh", "description": "Detected malicious IP activity from source" }, "~threat": { "indicator": { "ip": "192.168.1.10" } } } ```
Verifying asset history :green_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -k /var/ossec/etc/kvdb/ --environment environment/wazuh/1 -d Enter log in single line (Crtl+C to exit): 192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] "GET /someicon.ico HTTP/1.1" 404 209 DECODERS: decoder/queue-sca/0 -> failure decoder/queue-syslog/0 -> failure decoder/queue-dbsync/0 -> failure decoder/queue-rootcheck/0 -> failure decoder/queue-localfile/0 -> success decoder/syslog/0 -> failure decoder/apache-access/0 -> success RULES: rule/source-malicious-ip/0 -> success OUTPUT: { "wazuh": { "queue": 49, "origin": "/dev/stdin" }, "event": { "original": "192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] \"GET /someicon.ico HTTP/1.1\" 404 209", "created": "1992-07-10T10:07:13.000Z", "kind": "event", "dataset": "apache.access", "category": [ "web" ], "module": "apache", "outcome": "failure" }, "http": { "response": { "status_code": 404, "body": { "bytes": 209 } }, "version": "1.1", "request": { "method": "GET" } }, "url": { "original": "/someicon.ico" }, "source": { "ip": "192.168.1.10" }, "service": { "type": "apache" }, "rule": { "id": "source-malicious-ip", "author": "Wazuh", "description": "Detected malicious IP activity from source" }, "~threat": { "indicator": { "ip": "192.168.1.10" } } } ```
Verifying asset history plus full tracing :green_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test -k /var/ossec/etc/kvdb/ --environment environment/wazuh/1 -dd Enter log in single line (Crtl+C to exit): 192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] "GET /someicon.ico HTTP/1.1" 404 209 DECODERS: decoder/queue-sca/0 -> failure [decoder/queue-sca/0] condition.value[/wazuh/queue==112] -> Failure [decoder/queue-sca/0] [condition]:failure decoder/queue-syslog/0 -> failure [decoder/queue-syslog/0] condition.value[/wazuh/queue==50] -> Failure [decoder/queue-syslog/0] [condition]:failure decoder/queue-dbsync/0 -> failure [decoder/queue-dbsync/0] condition.value[/wazuh/queue==53] -> Failure [decoder/queue-dbsync/0] [condition]:failure decoder/queue-rootcheck/0 -> failure [decoder/queue-rootcheck/0] condition.value[/wazuh/queue==57] -> Failure [decoder/queue-rootcheck/0] [condition]:failure decoder/queue-localfile/0 -> success [decoder/queue-localfile/0] condition.value[/wazuh/queue==49] -> Success [decoder/queue-localfile/0] [condition]:success decoder/syslog/0 -> failure [decoder/syslog/0] [/event/original: []:<~/ignore/ >] -> Failure: Parser trace: Parser["event.start"] failure [decoder/syslog/0] [/event/original: :<~/ignore/ >] -> Failure: Parser trace: Parser["event.start"] failure [decoder/syslog/0] [/event/original: []: ] -> Failure: Parser trace: Parser["event.start"] failure [decoder/syslog/0] [/event/original: : ] -> Failure: Parser trace: Parser["event.start"] failure [decoder/syslog/0] [condition]:failure decoder/apache-access/0 -> success [decoder/apache-access/0] [helper.ef_exists[/event/original]] -> Success [decoder/apache-access/0] /event/original: - - [] " HTTP/" -> Success [decoder/apache-access/0] [condition]:success [decoder/apache-access/0] map.value[/event/kind="event"] -> Success [decoder/apache-access/0] map.value[/event/dataset="apache.access"] -> Success [decoder/apache-access/0] [helper.a_append[/event/category, web]] -> Success [decoder/apache-access/0] map.value[/event/module="apache"] -> Success [decoder/apache-access/0] map.value[/service/type="apache"] -> Success [decoder/apache-access/0] map.value[/event/outcome="success"] -> Success [decoder/apache-access/0] [helper.i_gt[/http/response/status_code, 399]] -> Success [decoder/apache-access/0] map.value[/event/outcome="failure"] -> Success RULES: rule/source-malicious-ip/0 -> success [rule/source-malicious-ip/0] [helper.ef_exists[/source/ip]] -> Success [rule/source-malicious-ip/0] [helper./source/ip[kvdb_match, malicious-ip]] -> Success [rule/source-malicious-ip/0] [condition]:success [rule/source-malicious-ip/0] map.value[/rule/id="source-malicious-ip"] -> Success [rule/source-malicious-ip/0] map.value[/rule/author="Wazuh"] -> Success [rule/source-malicious-ip/0] map.value[/rule/description="Detected malicious IP activity from source"] -> Success [rule/source-malicious-ip/0] map.reference[/~0threat/indicator/ip=/source/ip] -> Success OUTPUT: { "wazuh": { "queue": 49, "origin": "/dev/stdin" }, "event": { "original": "192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] \"GET /someicon.ico HTTP/1.1\" 404 209", "created": "1992-07-10T10:07:13.000Z", "kind": "event", "dataset": "apache.access", "category": [ "web" ], "module": "apache", "outcome": "failure" }, "http": { "response": { "status_code": 404, "body": { "bytes": 209 } }, "version": "1.1", "request": { "method": "GET" } }, "url": { "original": "/someicon.ico" }, "source": { "ip": "192.168.1.10" }, "service": { "type": "apache" }, "rule": { "id": "source-malicious-ip", "author": "Wazuh", "description": "Detected malicious IP activity from source" }, "~threat": { "indicator": { "ip": "192.168.1.10" } } } ```
Verify the trace for a selection of assets :green_circle:
assets: rule/source-malicious-ip/0 :green_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test --trace rule/source-malicious-ip/0 --environment environment/wazuh/1 -dd Enter log in single line (Crtl+C to exit): 192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] "GET /someicon.ico HTTP/1.1" 404 209 DECODERS: decoder/queue-sca/0 -> failure decoder/queue-syslog/0 -> failure decoder/queue-dbsync/0 -> failure decoder/queue-rootcheck/0 -> failure decoder/queue-localfile/0 -> success decoder/syslog/0 -> failure decoder/apache-access/0 -> success RULES: rule/source-malicious-ip/0 -> success [rule/source-malicious-ip/0] [helper.ef_exists[/source/ip]] -> Success [rule/source-malicious-ip/0] [helper./source/ip[kvdb_match, malicious-ip]] -> Success [rule/source-malicious-ip/0] [condition]:success [rule/source-malicious-ip/0] map.value[/rule/id="source-malicious-ip"] -> Success [rule/source-malicious-ip/0] map.value[/rule/author="Wazuh"] -> Success [rule/source-malicious-ip/0] map.value[/rule/description="Detected malicious IP activity from source"] -> Success [rule/source-malicious-ip/0] map.reference[/~0threat/indicator/ip=/source/ip] -> Success OUTPUT: { "wazuh": { "queue": 49, "origin": "/dev/stdin" }, "event": { "original": "192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] \"GET /someicon.ico HTTP/1.1\" 404 209", "created": "1992-07-10T10:07:13.000Z", "kind": "event", "dataset": "apache.access", "category": [ "web" ], "module": "apache", "outcome": "failure" }, "http": { "response": { "status_code": 404, "body": { "bytes": 209 } }, "version": "1.1", "request": { "method": "GET" } }, "url": { "original": "/someicon.ico" }, "source": { "ip": "192.168.1.10" }, "service": { "type": "apache" }, "rule": { "id": "source-malicious-ip", "author": "Wazuh", "description": "Detected malicious IP activity from source" }, "~threat": { "indicator": { "ip": "192.168.1.10" } } } ```
assets: decoder/apache-access/0 :green_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test --trace decoder/apache-access/0 --environment environment/wazuh/1 -dd Enter log in single line (Crtl+C to exit): 192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] "GET /someicon.ico HTTP/1.1" 404 209 DECODERS: decoder/queue-sca/0 -> failure decoder/queue-syslog/0 -> failure decoder/queue-dbsync/0 -> failure decoder/queue-rootcheck/0 -> failure decoder/queue-localfile/0 -> success decoder/syslog/0 -> failure decoder/apache-access/0 -> success [decoder/apache-access/0] [helper.ef_exists[/event/original]] -> Success [decoder/apache-access/0] /event/original: - - [] " HTTP/" -> Success [decoder/apache-access/0] [condition]:success [decoder/apache-access/0] map.value[/event/kind="event"] -> Success [decoder/apache-access/0] map.value[/event/dataset="apache.access"] -> Success [decoder/apache-access/0] [helper.a_append[/event/category, web]] -> Success [decoder/apache-access/0] map.value[/event/module="apache"] -> Success [decoder/apache-access/0] map.value[/service/type="apache"] -> Success [decoder/apache-access/0] map.value[/event/outcome="success"] -> Success [decoder/apache-access/0] [helper.i_gt[/http/response/status_code, 399]] -> Success [decoder/apache-access/0] map.value[/event/outcome="failure"] -> Success RULES: rule/source-malicious-ip/0 -> success OUTPUT: { "wazuh": { "queue": 49, "origin": "/dev/stdin" }, "event": { "original": "192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] \"GET /someicon.ico HTTP/1.1\" 404 209", "created": "1992-07-10T10:07:13.000Z", "kind": "event", "dataset": "apache.access", "category": [ "web" ], "module": "apache", "outcome": "failure" }, "http": { "response": { "status_code": 404, "body": { "bytes": 209 } }, "version": "1.1", "request": { "method": "GET" } }, "url": { "original": "/someicon.ico" }, "source": { "ip": "192.168.1.10" }, "service": { "type": "apache" }, "rule": { "id": "source-malicious-ip", "author": "Wazuh", "description": "Detected malicious IP activity from source" }, "~threat": { "indicator": { "ip": "192.168.1.10" } } } ```
assets: decoder/apache-access/0,rule/source-malicious-ip/0 :green_circle: ``` root@engine:/home/vagrant# /var/ossec/engine/wazuh-engine test --trace rule/source-malicious-ip/0,decoder/apache-access/0 --environment environment/wazuh/1 -dd Enter log in single line (Crtl+C to exit): 192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] "GET /someicon.ico HTTP/1.1" 404 209 DECODERS: decoder/queue-sca/0 -> failure decoder/queue-syslog/0 -> failure decoder/queue-dbsync/0 -> failure decoder/queue-rootcheck/0 -> failure decoder/queue-localfile/0 -> success decoder/syslog/0 -> failure decoder/apache-access/0 -> success [decoder/apache-access/0] [helper.ef_exists[/event/original]] -> Success [decoder/apache-access/0] /event/original: - - [] " HTTP/" -> Success [decoder/apache-access/0] [condition]:success [decoder/apache-access/0] map.value[/event/kind="event"] -> Success [decoder/apache-access/0] map.value[/event/dataset="apache.access"] -> Success [decoder/apache-access/0] [helper.a_append[/event/category, web]] -> Success [decoder/apache-access/0] map.value[/event/module="apache"] -> Success [decoder/apache-access/0] map.value[/service/type="apache"] -> Success [decoder/apache-access/0] map.value[/event/outcome="success"] -> Success [decoder/apache-access/0] [helper.i_gt[/http/response/status_code, 399]] -> Success [decoder/apache-access/0] map.value[/event/outcome="failure"] -> Success RULES: rule/source-malicious-ip/0 -> success [rule/source-malicious-ip/0] [helper.ef_exists[/source/ip]] -> Success [rule/source-malicious-ip/0] [helper./source/ip[kvdb_match, malicious-ip]] -> Success [rule/source-malicious-ip/0] [condition]:success [rule/source-malicious-ip/0] map.value[/rule/id="source-malicious-ip"] -> Success [rule/source-malicious-ip/0] map.value[/rule/author="Wazuh"] -> Success [rule/source-malicious-ip/0] map.value[/rule/description="Detected malicious IP activity from source"] -> Success [rule/source-malicious-ip/0] map.reference[/~0threat/indicator/ip=/source/ip] -> Success OUTPUT: { "wazuh": { "queue": 49, "origin": "/dev/stdin" }, "event": { "original": "192.168.1.10 - - [10/Jul/1992:10:07:13 +0200] \"GET /someicon.ico HTTP/1.1\" 404 209", "created": "1992-07-10T10:07:13.000Z", "kind": "event", "dataset": "apache.access", "category": [ "web" ], "module": "apache", "outcome": "failure" }, "http": { "response": { "status_code": 404, "body": { "bytes": 209 } }, "version": "1.1", "request": { "method": "GET" } }, "url": { "original": "/someicon.ico" }, "source": { "ip": "192.168.1.10" }, "service": { "type": "apache" }, "rule": { "id": "source-malicious-ip", "author": "Wazuh", "description": "Detected malicious IP activity from source" }, "~threat": { "indicator": { "ip": "192.168.1.10" } } } ```
roronoasins commented 1 year ago

Conclusion

During the testing process we could test the following cases:

Improvements

Bugs