MarcelKemp commented 1 year ago

The following issue aims to run the specified test for the current release candidate, report the results, and open new issues for any encountered errors.

Test information


Test name	Deployment with Puppet
Category	Installation
Deployment option	Deployment with Puppet
Main release issue	https://github.com/wazuh/wazuh/issues/15505
Release candidate #	https://github.com/wazuh/wazuh/issues/15519
Previous issue	https://github.com/wazuh/wazuh/issues/13225, https://github.com/wazuh/wazuh/issues/12877

Environment


Component	Type	OS	Installation
Wazuh dashboard		Ubuntu 20.04	Deployment with Puppet
Wazuh indexer	Multi node	Ubuntu 20.04	Deployment with Puppet
Wazuh server	Multi node	Ubuntu 20.04	Deployment with Puppet
Wazuh agent		Ubuntu 20.04	Deployment with Puppet

Test description

Test distributed deployment with Wazuh manager cluster and Wazuh indexer cluster via Puppet. Test deployment of agent via Puppet.

Test report procedure

All test results must have one of the following statuses:
:green_circle:	All checks passed.
:red_circle:	There is at least one failed result.
:yellow_circle:	There is at least one expected failure or skipped test and no failures.

Any failing test must be properly addressed with a new issue, detailing the error and the possible cause.

An extended report of the test results must be attached as a ZIP or TXT file. Please attach any documents, screenshots, or tables to the issue update with the results. This report can be used by the auditors to dig deeper into any possible failures and details.

Conclusions

All tests have been executed, and the results can be found [here]().


Status	Test	Failure type	Notes
:green_circle:	Set up Puppet
:red_circle:	Wazuh Puppet module	wazuh-puppet & wazuh-documentation	Without fixing the following errors, the deployment is not performed correctly: Filebeat-OSS error, misconfiguration in API (puppet and documentation)

All tests have passed and the fails have been reported or justified. Therefore, I conclude that this issue is finished and OK for this release candidate.

Auditors validation

The definition of done for this one is the validation of the conclusions and the test results from all auditors.

All checks from below must be accepted in order to close this issue.

[x] @wazuh/cicd
[x] @davidjiglesias

MarcelKemp commented 1 year ago

Set up Puppet

:green_circle: 1. Installing Puppet master

Modified the /etc/hosts file as described in the documentation.

:green_circle: 1.1. Installation on Debian/Ubuntu

Installation was OK.

:green_circle: 1.2. Memory Allocation

Configured to use 512 MB. Instructions easy to follow.

:green_circle: 1.3. Configuration

If you copy both configuration blocks from the documentation, you may have problems in the future, however, if you assign the value of your DNS to the server variable ('puppet' in my case, or just don't add it, and it will take it by default), it will work correctly.

This is the content of my /etc/puppetlabs/puppet/puppet.conf file so far:

[server]
vardir = /opt/puppetlabs/server/data/puppetserver
logdir = /var/log/puppetlabs/puppetserver
rundir = /var/run/puppetlabs/puppetserver
pidfile = /var/run/puppetlabs/puppetserver/puppetserver.pid
codedir = /etc/puppetlabs/code

[main]
dns_alt_names = puppet,puppet-server

The Puppet server was started correctly:

# systemctl status puppetserver
● puppetserver.service - puppetserver Service
     Loaded: loaded (/lib/systemd/system/puppetserver.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-12-02 11:55:03 UTC; 7s ago
   Main PID: 6976 (java)
      Tasks: 48 (limit: 4915)
     Memory: 567.6M
     CGroup: /system.slice/puppetserver.service
             └─6976 /usr/bin/java -Xms512m -Xmx512m -Djruby.logger.class=com.puppetlabs.jruby_utils.>

Dec 02 11:54:36 puppet-master systemd[1]: Starting puppetserver Service...
Dec 02 11:54:41 puppet-master puppetserver[6976]: WARNING: abs already refers to: #'clojure.core/abs>
Dec 02 11:55:03 puppet-master systemd[1]: Started puppetserver Service.

:green_circle: 2. Installing Puppet agent

:green_circle: 2.1. Installation on Debian/Ubuntu

Installation was OK.

:green_circle: 2.2 Agent Configuration

The agent configuration header is smaller than the installation headers:

https://documentation-dev.wazuh.com/v4.4.0-alpha1/deployment-options/deploying-with-puppet/setup-puppet/install-puppet-agent.html#agent-configuration

I followed those instructions, since it seems like an important and generic step. This is my /etc/puppetlabs/puppet/puppet.conf file after that:

[main]
server = puppet

And finally, the Puppet agent was started correctly:

# sudo systemctl status puppet
● puppet.service - Puppet agent
     Loaded: loaded (/lib/systemd/system/puppet.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-12-02 12:26:41 UTC; 99ms ago
       Docs: man:puppet-agent(8)
   Main PID: 4837 (puppet)
      Tasks: 1 (limit: 2320)
     Memory: 15.2M
     CGroup: /system.slice/puppet.service
             └─4837 /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/puppet agent --no-dae>

Dec 02 12:26:41 puppet-agent systemd[1]: Started Puppet agent.

:green_circle: 3. Setting up Puppet certificates

3.1:

# puppet agent -t
Info: csr_attributes file loading from /etc/puppetlabs/puppet/csr_attributes.yaml
Info: Creating a new SSL certificate request for puppet-agent.home
Info: Certificate Request fingerprint (SHA256): XX:XX:...:XX
Info: Certificate for puppet-agent.home has not been signed yet
Couldn't fetch certificate from CA server; you might still need to sign this agent's certificate (puppet-agent.home).
Exiting now because the waitforcert setting is set to 0.

3.2:

# puppetserver ca list
Requested Certificates:
puppet-agent.home       (SHA256)  XX:...:XX

3.3:

# puppetserver ca sign --all
Successfully signed certificate request for puppet-agent.home

3.4:

# puppet agent -t
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for puppet-agent.home
Info: Applying configuration version '1669984256'
Notice: Applied catalog in 0.01 seconds

MarcelKemp commented 1 year ago

:red_circle: 4. Wazuh Puppet module

:green_circle: 4.1. Install Wazuh module

Install Wazuh 4.4.0 module on Puppet master node

wget https://github.com/wazuh/wazuh-puppet/tarball/4.4
puppet module install /home/vagrant/4.4

Wazuh module installation was successful.

Configure repo.pp for testing

Replace the following:

packages.wazuh.com by packages-dev.wazuh.com
4.x by pre-release
release => 'stable' by release => 'unstable'

:red_circle: 4.2. Install manager (cluster) via Puppet

I created the file /etc/puppetlabs/code/environments/production/manifests/wazuh-manager.pp with this content in the puppet master:

node "puppet-manager" {
 class { 'wazuh::manager':
    ossec_cluster_name => 'wazuh-cluster',
    ossec_cluster_node_name => 'wazuh-master',
    ossec_cluster_node_type => 'master',
    ossec_cluster_key => '01234567890123456789012345678912',
    ossec_cluster_bind_addr => '10.2.0.12',
    ossec_cluster_nodes => ['10.2.0.12'],
    ossec_cluster_disabled => 'no',
 }
 class { 'wazuh::indexer':
 }
 class { 'wazuh::filebeat_oss':
 }
 class { 'wazuh::dashboard':
 }
}

node "puppet-worker" {
 class { 'wazuh::manager':
    ossec_cluster_name => 'wazuh-cluster',
    ossec_cluster_node_name => 'wazuh-worker',
    ossec_cluster_node_type => 'worker',
    ossec_cluster_key => '01234567890123456789012345678912',
    ossec_cluster_bind_addr => '10.2.0.12',
    ossec_cluster_nodes => ['10.2.0.12'],
    ossec_cluster_disabled => 'no',
 }
 class { 'wazuh::indexer':
 }
}

With this configuration, we would install all the components (manager, indexer, filebeat-oss and dashboard) in the wazuh-master, and then in the wazuh-worker we would have an indexer that would be necessary to manually configure the cluster. And then, a manager that would already be configured and connected as a worker of the master node.

However, the following errors have been encountered which have prevented us from deploying the relevant components:

A first error in filebeat-oss that did not allow us to move forward with the deployment, where the variable wazuh_app_version is not well defined.
After manually fixing the previous error to continue testing, another error has been found in filebeat-oss, due to the lack of a file in its corresponding URL.
- Both bugs are reported in the following issue: https://github.com/wazuh/wazuh-puppet/issues/610

In order to move forward, it has been decided not to deploy filebeat-oss on any puppet agent.

Then, following the testing, an API configuration error was found in the installation of the manager, which did not allow the manager to be installed correctly.
This problem has been reported in the following issues:
- Wazuh-puppet: https://github.com/wazuh/wazuh-puppet/issues/611
- Wazuh-documentation: https://github.com/wazuh/wazuh-documentation/issues/5800

And finally, after manually fixing the last bug, the following managers have been installed correctly:

wazuh-master: Dashboard, Indexer (manually configured to make it multi-node) and manager (master).
wazuh-worker: Indexer (manually configured) and manager (worker).

# /var/ossec/bin/cluster_control -l
NAME          TYPE    VERSION  ADDRESS    
wazuh-master  master  4.4.0    10.2.0.12  
wazuh-worker  worker  4.4.0    10.2.0.19

:green_circle: 5.3. Install agent via Puppet

I created the /etc/puppetlabs/code/environments/production/manifests/wazuh-agent.pp manifest with this content:

node "wazuh-agent1" {
  class { "wazuh::agent":
    wazuh_register_endpoint => "10.2.0.19",
    wazuh_reporting_endpoint => "10.2.0.19"
  }
}

node "wazuh-agent2" {
  class { "wazuh::agent":
    wazuh_register_endpoint => "10.2.0.12",
    wazuh_reporting_endpoint => "10.2.0.12"
  }
}

Where the following agents have been correctly installed:

wazuh-agent1: Connected to master node.
wazuh-agent2: Connected to worker node.

# /var/ossec/bin/agent_control -l

Wazuh agent_control. List of available agents:
   ID: 000, Name: puppet-manager (server), IP: 127.0.0.1, Active/Local
   ID: 001, Name: wazuh-agent1, IP: 10.2.0.19, Connected
   ID: 002, Name: wazuh-agent2, IP: 10.2.0.12, Connected

wazuh / wazuh

Release 4.4.0 - Alpha 1 - E2E UX tests - Deployment with Puppet #15538

Test information

Environment

Test description

Test report procedure

Conclusions

Auditors validation

Set up Puppet

:green_circle: 1. Installing Puppet master

:green_circle: 1.1. Installation on Debian/Ubuntu

:green_circle: 1.2. Memory Allocation

:green_circle: 1.3. Configuration

:green_circle: 2. Installing Puppet agent

:green_circle: 2.1. Installation on Debian/Ubuntu

:green_circle: 2.2 Agent Configuration

:green_circle: 3. Setting up Puppet certificates

:red_circle: 4. Wazuh Puppet module

:green_circle: 4.1. Install Wazuh module

Install Wazuh 4.4.0 module on Puppet master node

Configure repo.pp for testing

:red_circle: 4.2. Install manager (cluster) via Puppet

:green_circle: 5.3. Install agent via Puppet