Open denizciftci-sec opened 1 year ago
Hello @denizciftci-sec
Could you tell me which of the two deployments you are using (single_node or multi_node) and what steps did you take for it?
I recommend that to deploy an environment with docker you follow the steps in our documentation: https://documentation.wazuh.com/current/deployment-options/docker/wazuh-container.html
Hi @vcerenu ,
many thanks for the reply. We went for the single mode deployment and followed the guide except the certification generation part. The generate-indexer-certs.yml is clearly not working so I downloaded the bash script(wazuh-certs-tool.sh) and created/edited config.yml then I generated all certificates manually/successfully and moved them in wazuh_indexer_ssl_certs directory.
Certificate generation error: [root@t-ifs-wazuh-srv01 single-node]# docker-compose -f generate-indexer-certs.yml run --rm generator WARN[0000] Found orphan containers ([single-node-wazuh.dashboard-1 single-node-wazuh.manager-1 single-node-wazuh.indexer-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. Cert tool does not exist in any bucket ERROR: certificates were not created
[root@t-ifs-wazuh-srv01 single-node]# docker --version Docker version 20.10.21, build baeda1f [root@t-ifs-wazuh-srv01 single-node]# docker-compose --version Docker Compose version v2.12.2
docker ps output:
root@wazuh:/# [root@t-ifs-wazuh-srv01 single-node]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 8ca7bcee5754 wazuh/wazuh-dashboard:4.3.10 "/entrypoint.sh" 11 minutes ago Up 6 minutes 443/tcp, 0.0.0.0:443->5601/tcp single-node-wazuh.dashboard-1 53d99bb9a1b9 wazuh/wazuh-manager:4.3.10 "/init" 11 minutes ago Up 8 seconds 0.0.0.0:1514-1515->1514-1515/tcp, 0.0.0.0:514->514/udp, 0.0.0.0:55000->55000/tcp, 1516/tcp single-node-wazuh.manager-1 e8a14083200f wazuh/wazuh-indexer:4.3.10 "/entrypoint.sh open…" 11 minutes ago Up 6 minutes 0.0.0.0:9200->9200/tcp single-node-wazuh.indexer-1
I am not sure is this relevant but I can paste some indications that we have seen so far:
root@wazuh:/# service wazuh-manager status wazuh-clusterd not running... wazuh-modulesd not running... wazuh-monitord not running... wazuh-logcollector not running... wazuh-remoted not running... wazuh-syscheckd not running... wazuh-analysisd not running... wazuh-maild not running... wazuh-execd not running... wazuh-db not running... wazuh-authd not running... wazuh-agentlessd not running... wazuh-integratord not running... wazuh-dbd not running... wazuh-csyslogd not running... wazuh-apid not running...
root@wazuh:/# /var/ossec/bin/wazuh-apid -f wazuh-apid: Orphan child process 404 was terminated. wazuh-apid: Orphan child process 407 was terminated. wazuh-apid: Orphan child process 333 was terminated. Starting API in foreground wazuh-apid: Orphan child process 407 was terminated. wazuh-apid: Orphan child process 410 was terminated. wazuh-apid: Orphan child process 332 was terminated. Starting API in foreground
Got this error in wazuh-manager; root@wazuh:/# cat /var/ossec/logs/ossec.log | grep -iE "ERROR|CRITICAL" 2022/11/22 13:26:46 wazuh-db: ERROR: at run_worker(): at recv(): Connection reset by peer (104)
The part of docker logs output of wazuh-manager (tried to capture events related to apid)
The path /etc/filebeat is already mounted Identified Wazuh configuration files to mount... '/wazuh-config-mount/etc/ossec.conf' -> '/var/ossec/etc/ossec.conf' [cont-init.d] 0-wazuh-init: exited 0. [cont-init.d] 1-config-filebeat: executing... [cont-init.d] 1-config-filebeat: exited 0. [cont-init.d] 2-manager: executing... Starting Wazuh v4.3.10... wazuh-apid: Process 404 not used by Wazuh, removing... wazuh-apid: Non existent process 475, removing from /var/ossec/var/run... wazuh-apid: Non existent process 478, removing from /var/ossec/var/run... wazuh-apid: Non existent process 475, removing from /var/ossec/var/run... wazuh-apid: Non existent process 478, removing from /var/ossec/var/run... wazuh-apid: Non existent process 475, removing from /var/ossec/var/run... wazuh-apid: Non existent process 478, removing from /var/ossec/var/run... Started wazuh-apid... Started wazuh-csyslogd... Started wazuh-dbd... Started wazuh-integratord... Started wazuh-agentlessd... wazuh-authd: Process 444 not used by Wazuh, removing... Started wazuh-authd... Started wazuh-db... Started wazuh-execd... Started wazuh-analysisd... Started wazuh-syscheckd... Started wazuh-remoted... Started wazuh-logcollector... Started wazuh-monitord... wazuh-modulesd: Process 762 not used by Wazuh, removing... Started wazuh-modulesd... Completed.
Hi @denizciftci-sec
I see that when you tried to create you had containers up, that means that you should already have the certificates or you have created directories with the names of the certificates.
I recommend that you delete all the files and directories inside the wazuh_indexer_ssl_certs
directory, delete all the wazuh stack containers that are running and generate the certificates again with the command docker-compose -f generate-indexer-certs.yml run --rm generator
.
It checks if the internet connection is open for the container that is created for the creation of certificates, which checks that the wazuh-certs-tool.sh file is in our repository.
Also, I ask you, are you running on linux? because certificate creation doesn't work on MacOS.
Hi @vcerenu,
many thanks for the reply. There is an internet connection for sure - where we were able to pull the images successfully from the docker repository. Specifically, we are using proxy for docker process via > /etc/systemd/system/docker.service.d/http-proxy.conf
I got the following error when I execute it;
[root@t-ifs-wazuh-srv01 single-node]# docker-compose -f generate-indexer-certs.yml run --rm generator WARN[0000] Found orphan containers ([single-node-wazuh.dashboard-1 single-node-wazuh.manager-1 single-node-wazuh.indexer-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. Cert tool does not exist in any bucket ERROR: certificates were not created
About the containers that I told you, it is referred to this warning:
WARN[0000] Found orphan containers ([single-node-wazuh.dashboard-1 single-node-wazuh.manager-1 single-node-
You must download all the containers that are running, so that it is not taking the certificate files. Also, the last error does not seem to reach the repository, I recommend that you try if from that PC you reach the following paths:
https://packages.wazuh.com/4.3/ https://packages-dev.wazuh.com/4.3/
Within those two buckets you access the file that the container has to use to create the certificates, otherwise in the wazuh-docker repository you can check what the container does to create the certificates in the file indexer-certs-creator/config/ entrypoint.sh.
Hi @vcerenu,
I deleted all the containers, volumes and files/folders inside wazuh_indexer_ssl_certs as you recommended. Initially, I defined the proxy in generate-indexer-certs.yml, but still having the same error (We should use proxy in our infrastructure). There is no firewalld or iptables service is running.
[root@t-ifs-wazuh-srv01 single-node]# vi generate-indexer-certs.yml
Wazuh App Copyright (C) 2021 Wazuh Inc. (License GPLv2) version: '3'
services: generator: image: wazuh/wazuh-certs-generator:0.0.1 hostname: wazuh-certs-generator volumes:
[root@t-ifs-wazuh-srv01 single-node]# docker-compose -f generate-indexer-certs.yml run --rm generator Cert tool does not exist in any bucket
I guess, it connects the relevant paths with 200/OK.
[root@t-ifs-wazuh-srv01 single-node]# curl -X HEAD -i https://packages.wazuh.com/4.3/ HTTP/1.1 200 Connection established
HTTP/1.1 403 Forbidden Content-Type: application/xml Connection: keep-alive Date: Wed, 23 Nov 2022 08:37:58 GMT Server: AmazonS3 X-Cache: Error from cloudfront Via: 1.1 2afacc6ad96dbba3f0b477cd95f16458.cloudfront.net (CloudFront) X-Amz-Cf-Pop: FRA2-C2 X-Amz-Cf-Id: jxKn2k0R8R6bF3lqYUui24BEs4uM28X-W-lCe4x6eJxGPYmPSApzPw==
[root@t-ifs-wazuh-srv01 single-node]# curl -X HEAD -i https://packages-dev.wazuh.com/4.3/ HTTP/1.1 200 Connection established
HTTP/1.1 404 Not Found Content-Type: application/xml Connection: keep-alive Date: Wed, 23 Nov 2022 08:38:31 GMT Server: AmazonS3 X-Cache: Error from cloudfront Via: 1.1 dc0aad619823d3400ef947433d0af8fa.cloudfront.net (CloudFront) X-Amz-Cf-Pop: FRA60-P3 X-Amz-Cf-Id: u3LuXpcwJ5cPTnd8zcCtkm--x5L543hOgwo1FeRaBA8NdLRYL5H0_w==
Hello @denizciftci-sec
You can check directly with the commands with which we check in the image if the tool exists
curl --silent -I https://packages.wazuh.com/4.3/wazuh-certs-tool.sh | grep -E "^HTTP" | awk '{print $2}'
curl --silent -I https://packages-dev.wazuh.com/4.3/wazuh-certs-tool.sh | grep -E "^HTTP" | awk '{print $2}'
Either of these two commands should return 200
, which indicates that you are reaching the tool. It may be that you get another code through the proxy and this does not allow it to complete, so you can perform the test that I tell you, see what response code the first command gives you on the packages.wazuh.com
address and with that response modify the entrypoint indexer-certs-creator/config/entrypoint.sh
, then on the path indexer-certs-creator/
you can generate the image with the modified code with the following command:
docker build -t wazuh/wazuh-certs-generator:0.0.1 .
With this you generate the modified image so that it takes the code that you changed and you can launch the certificate generation command again.
Let me know how it went when you finished these tasks.
hi @vcerenu, many thanks for the rapid reply. I was able to solve it via > following the official procedure on a test-PC (has no problems with proxy) and able to generate the certificates - and these are generated on the IP address of the main server. When I moved all certificates into wazuh_indexer_ssl_certs > these API errors went away...
Only 1 error we are countering at the moment is (also I saw the identical error in the test PC) Check alerts index pattern >
INFO: Index pattern id in cookie: yes [wazuh-alerts-] INFO: Getting list of valid index patterns... INFO: Valid index patterns found: 1 INFO: Found default index pattern with title [wazuh-alerts-]: yes INFO: Checking the app default pattern exists: id [wazuh-alerts-]... INFO: Default pattern with id [wazuh-alerts-] exists: yes ACTION: Default pattern id [wazuh-alerts-] set as default index pattern INFO: Checking the index pattern id [wazuh-alerts-] exists... INFO: Index pattern id exists [wazuh-alerts-]: yes INFO: Index pattern id in cookie: yes [wazuh-alerts-] INFO: Checking if the index pattern id [wazuh-alerts-] exists... INFO: Index pattern id [wazuh-alerts-] found: yes title [wazuh-alerts-] INFO: Checking if exists a template compatible with the index pattern title [wazuh-alerts-] *INFO: Template found for the selected index-pattern title [wazuh-alerts-]: no* ERROR: No template found for the selected index-pattern title [wazuh-alerts-] INFO: Index pattern id in cookie: [wazuh-alerts-] INFO: Getting index pattern data [wazuh-alerts-]... INFO: Index pattern data found: [yes] INFO: Refreshing index pattern fields: title [wazuh-alerts-], id [wazuh-alerts-]... ACTION: Refreshed index pattern fields: title [wazuh-alerts-], id [wazuh-alerts-] [Alerts index pattern] No template found for the selected index-pattern title [wazuh-alerts-*]
I tried to add the templates manually, but still having the error. Am I missing any steps here?
curl https://raw.githubusercontent.com/wazuh/wazuh/v4.3.10/extensions/elasticsearch/7.x/wazuh-template.json | curl --noproxy '' -X PUT "https://localhost:9200/_template/wazuh" -H 'Content-Type: application/json' -d @- -u wazuh-wui:xx- -k
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 58530 100 58530 0 0 254k 0 --:--:-- --:--:-- --:--:-- 254k
Current Templates:
Index Patterns in GUI:
Hello @denizciftci-sec
This problem with the index pattern
is due to the fact that it is created when, from the Wazuh manager
container, Filebeat
connects with Wazuh Indexer
.
I recommend that you enter the Wazuh manager container and execute the following command
filebeat test output
If you have any errors, you should check the certificates that have been mounted, to see if they are correct.
Hi @vcerenu,
This is what we see in manager;
elasticsearch: https://wazuh.indexer:9200... parse url... OK connection... parse host... OK dns lookup... OK addresses: 172.19.0.3 dial up... OK TLS... security: server's certificate chain verification is enabled handshake... OK TLS version: TLSv1.3 dial up... OK talk to server... ERROR Get "https://wazuh.indexer:9200": Forbidden root@wazuh:/tmp#
Hi @denizciftci-sec
This error when executing the filebeat test output
command indicates that you have no connection between Wazuh manager
and Wazuh indexer
, so it will not be able to pass some things to it, including the missing index pattern.
You should check that the certificates that you are mounting to Wazuh manager
do not have errors.
Hi @vcerenu, The problem was fixed. I was able to generate the certificates successfully by not chaning the IP address of certs.yml file on my test PC. When I moved the generated certifications to production one, it worked! thanks for the support. But still not able to fix the the container service which generated certificates.
Hello,
We have a all-in-one docker-compose setup. Completed the installation steps successfully - when the docker-compose is up, we saw wazuh-manager keeps restarting in 15-20 second. Is there a workaround for this problem?
Under the test connections menu: 1513629884013 https://wazuh.manager/ 55000 Offline
The erros on GUI: [API connection] No API available to connect [Alerts index pattern] No template found for the selected index-pattern title [wazuh-alerts-*]
Wazuh API Connection Details on GUI: INFO: No current API selected INFO: Getting API hosts... INFO: API hosts found: 1 INFO: Checking API host id [1513629884013]... INFO: Could not connect to API id [1513629884013]: 3099 - ERROR3099 - Some Wazuh daemons are not ready yet in node "node01" (wazuh-modulesd->failed) INFO: Removed [navigate] cookie ERROR: No API available to connect
Check alerts index pattern Errors on GUI: INFO: Index pattern id in cookie: yes [wazuh-alerts-] INFO: Getting list of valid index patterns... INFO: Valid index patterns found: 1 INFO: Found default index pattern with title [wazuh-alerts-]: yes INFO: Checking the app default pattern exists: id [wazuh-alerts-]... INFO: Default pattern with id [wazuh-alerts-] exists: yes ACTION: Default pattern id [wazuh-alerts-] set as default index pattern INFO: Checking the index pattern id [wazuh-alerts-] exists... INFO: Index pattern id exists [wazuh-alerts-]: yes INFO: Index pattern id in cookie: yes [wazuh-alerts-] INFO: Checking if the index pattern id [wazuh-alerts-] exists... INFO: Index pattern id [wazuh-alerts-] found: yes title [wazuh-alerts-] INFO: Checking if exists a template compatible with the index pattern title [wazuh-alerts-] INFO: Template found for the selected index-pattern title [wazuh-alerts-]: no ERROR: No template found for the selected index-pattern title [wazuh-alerts-] INFO: Index pattern id in cookie: [wazuh-alerts-] INFO: Getting index pattern data [wazuh-alerts-]... INFO: Index pattern data found: [yes] INFO: Refreshing index pattern fields: title [wazuh-alerts-], id [wazuh-alerts-]... ACTION: Refreshed index pattern fields: title [wazuh-alerts-], id [wazuh-alerts-]