search

wazuh / wazuh-indexer

Wazuh indexer, the Wazuh search engine
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
11 stars 19 forks source link

Data Persistence Model Redesign MVP validation test - I #478

Open AlexRuiz7 opened 2 days ago

AlexRuiz7 commented 2 days ago

Description

For the delivery of the MVP phase of https://github.com/wazuh/wazuh/issues/22887, we need to check that:

Tasks

For each of the sections, provide instructions and evidences of their testing.

QU3B1M commented 2 days ago

Validation process

  1. The packages can be built. Check the packages are built as result of the GHA Build workflow

    $ GITHUB_TOKEN=<MY_GITHUB_TOKEN> bash ./check_package.sh 11406244332 5.0.0

Fetching artifacts list...
Checking wazuh-indexer_5.0.0-0_amd64.deb package is generated for workflow run 11406244332
Wazuh indexer package built successfully.
[ Artifact ID: 2074853568 ]
    Test script ```bash #!/bin/bash ## SPDX-License-Identifier: Apache-2.0 ## The OpenSearch Contributors require contributions made to # this file be licensed under the Apache-2.0 license or a # compatible open source license. # Check if the necessary arguments are provided if [ "$#" -ne 2 ]; then echo "Usage: $0 <(Optional)PKG_REVISION>" echo echo "Parameters:" echo " RUN_ID The GHA workflow execution ID." echo " PKG_VERSION The version of the wazuh-indexer package." echo " PKG_REVISION (Optional) The revision of the package. Defaults to 'test' if not provided." echo echo "Please ensure you have the GITHUB_TOKEN environment variable set to access the GitHub repository." echo exit 1 fi RUN_ID=$1 PKG_VERSION=$2 PKG_REVISION=${3:-"0"} REPO="wazuh/wazuh-indexer" URL="https://api.github.com/repos/$REPO/actions/artifacts" # Detect OS and architecture if [ -f /etc/os-release ]; then . /etc/os-release OS=$(echo $NAME | tr '[:upper:]' '[:lower:]') else echo "Unsupported OS." exit 1 fi ARCH=$(uname -m) # Determine package type case "$OS" in "ubuntu" | "debian") PKG_FORMAT="deb" [ "$ARCH" == "x86_64" ] && ARCH="amd64" PKG_NAME="wazuh-indexer_${PKG_VERSION}-${PKG_REVISION}_${ARCH}.${PKG_FORMAT}" ;; "centos" | "fedora" | "rhel" | "red hat enterprise linux") PKG_FORMAT="rpm" PKG_NAME="wazuh-indexer-${PKG_VERSION}-${PKG_REVISION}.${ARCH}.${PKG_FORMAT}" ;; *) echo "Unsupported OS: ${OS}." exit 1 ;; esac # Fetch the list of artifacts echo "Fetching artifacts list..." RESPONSE=$(curl -s -L -H "Accept: application/vnd.github+json" -H "Authorization: Bearer $GITHUB_TOKEN" -H "X-GitHub-Api-Version: 2022-11-28" $URL?name=$PKG_NAME) # Check if the curl command was successful if [ $? -ne 0 ]; then echo "Error: Failed to fetch artifacts." exit 1 fi # Check if the artifact from the specified workflow run ID exists echo "Checking ${PKG_NAME} package is generated for workflow run ${RUN_ID}" ARTIFACT=$(echo "$RESPONSE" | jq -e ".artifacts[] | select(.workflow_run.id == $RUN_ID)") if [ -n "$ARTIFACT" ]; then ARTIFACT_ID=$(echo "$ARTIFACT" | jq -r '.id') echo "Wazuh indexer package built successfully." echo "[ Artifact ID: $ARTIFACT_ID ]" else echo "Error: Wazuh indexer package not found." fi ```

  2. The package can be installed Download and install the package generated on the GHA workflow

    $ GITHUB_TOKEN=<MY_GITHUB_TOKEN> bash ./check_installation.sh 2074853568 5.0.0

Downloading wazuh-indexer package from GitHub artifactory...
Package downloaded successfully
Decompressing wazuh-indexer package...
Archive:  ./package.zip
  inflating: wazuh-indexer_5.0.0-0_amd64.deb
Package decompressed
Installing wazuh-indexer package...
(Reading database ... 77544 files and directories currently installed.)
Preparing to unpack wazuh-indexer_5.0.0-0_amd64.deb ...
Running Wazuh Indexer Pre-Removal Script
Running Wazuh Indexer Pre-Installation Script
Unpacking wazuh-indexer (5.0.0-0) over (5.0.0-0) ...
Setting up wazuh-indexer (5.0.0-0) ...
Running Wazuh Indexer Post-Installation Script
### NOT starting on installation, please execute the following statements to configure wazuh-indexer service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable wazuh-indexer.service
### You can start wazuh-indexer service by executing
 sudo systemctl start wazuh-indexer.service
Package installed successfully.
    Testing script ```bash #!/bin/bash # SPDX-License-Identifier: Apache-2.0 # The OpenSearch Contributors require contributions made to # this file be licensed under the Apache-2.0 license or a # compatible open source license. # Usage function to display help usage() { echo "Usage: $0 <(Optional)PKG_REVISION>" echo echo "Parameters:" echo " ARTIFACT_ID The unique ID of the GHA artifact." echo " PKG_VERSION The version of the wazuh-indexer package." echo " PKG_REVISION (Optional) The revision of the package. Defaults to 'test' if not provided." echo echo "Please ensure you have the GITHUB_TOKEN environment variable set to access the GitHub repository." echo exit 1 } # Check if GITHUB_TOKEN env var is set if [ -z "$1" ]; then echo "Error: Environment variable GITHUB_TOKEN is not configured." usage fi # Check if ARTIFACT_ID is provided if [ -z "$1" ]; then echo "Error: ARTIFACT_ID not provided." usage fi # Check if PKG_VERSION is provided if [ -z "$2" ]; then echo "Error: PKG_VERSION not provided." usage fi ARTIFACT_ID=$1 PKG_VERSION=$2 PKG_REVISION=${3:-"0"} REPO="wazuh/wazuh-indexer" URL="https://api.github.com/repos/${REPO}/actions/artifacts/${ARTIFACT_ID}/zip" # Detect OS and architecture if [ -f /etc/os-release ]; then . /etc/os-release OS=$(echo $NAME | tr '[:upper:]' '[:lower:]') else echo "Unsupported OS." exit 1 fi ARCH=$(uname -m) # Determine package type case "$OS" in "ubuntu" | "debian") PKG_FORMAT="deb" [ "$ARCH" == "x86_64" ] && ARCH="amd64" # Construct package name PKG_NAME="wazuh-indexer_${PKG_VERSION}-${PKG_REVISION}_${ARCH}.${PKG_FORMAT}" ;; "centos" | "fedora" | "rhel" | "red hat enterprise linux") PKG_FORMAT="rpm" # Construct package name PKG_NAME="wazuh-indexer-${PKG_VERSION}-${PKG_REVISION}.${ARCH}.${PKG_FORMAT}" ;; *) echo "Unsupported OS." exit 1 ;; esac # Download the package echo "Downloading wazuh-indexer package from GitHub artifactory..." echo "(It could take a couple minutes)" curl -L -H "Accept: application/vnd.github+json" \ -H "Authorization: Bearer $GITHUB_TOKEN" \ -H "X-GitHub-Api-Version: 2022-11-28" \ $URL -o package.zip > /dev/null 2>&1 if [ $? -ne 0 ]; then echo "Error downloading package." exit 1 fi echo "Package downloaded successfully" # Unzip the package echo "Decompressing wazuh-indexer package..." unzip ./package.zip rm package.zip if [ $? -ne 0 ]; then echo "Error unzipping package." exit 1 fi echo "Package decompressed" # Install the package echo "Installing wazuh-indexer package..." case "$PKG_FORMAT" in "deb") sudo dpkg -i $PKG_NAME ;; "rpm") sudo rpm -i $PKG_NAME ;; esac if [ $? -ne 0 ]; then echo "Error installing package." exit 1 fi echo "Package installed successfully." ```

  3. The Wazuh Indexer service starts and runs (start, stop, restart).

    • Configure wazuh-indexer and deploy certificates

      sudo bash ./02_apply_certificates.sh node-1 node-2 192.168.56.10
Creating a backup of the original config file...
Updating configuration...
Configuration updated successfully. Backup created at ./opensearch.yml.bak
Creating certificates directory and extracting certificates...
Moving and setting permissions for certificates...
Certificates configured successfully.
      Test script ```bash #!/bin/bash # SPDX-License-Identifier: Apache-2.0 # The OpenSearch Contributors require contributions made to # this file be licensed under the Apache-2.0 license or a # compatible open source license. # Function to display usage help usage() { echo echo "Usage: $0 <(Optional)CURRENT_NODE_IP> <(Optional)SECOND_NODE_IP>" echo echo "Parameters:" echo " CURRENT_NODE Name of the current node" echo " SECOND_NODE Name of the second node" echo " CURRENT_NODE_IP IP address of the current node (optional, defaults to CURRENT_NODE)" echo " SECOND_NODE_IP IP address of the second node (optional, defaults to SECOND_NODE)" echo exit 1 } # Check if at least two arguments are provided if [ $# -lt 2 ]; then usage fi # Assigning variables CURRENT_NODE=$1 SECOND_NODE=$2 CURRENT_NODE_IP=${3:-$CURRENT_NODE} SECOND_NODE_IP=${4:-$SECOND_NODE} CONFIG_FILE="/etc/wazuh-indexer/opensearch.yml" BACKUP_FILE="./opensearch.yml.bak" # Backup the original config file echo "Creating a backup of the original config file..." cp $CONFIG_FILE $BACKUP_FILE # Replace values in the config file echo "Updating configuration..." sed -i "s/network\.host: \"0\.0\.0\.0\"/network.host: \"${CURRENT_NODE_IP}\"/" $CONFIG_FILE sed -i "s/node\.name: \"node-1\"/node.name: \"${CURRENT_NODE}\"/" $CONFIG_FILE sed -i "s/#discovery\.seed_hosts:/discovery.seed_hosts:\n - \"${CURRENT_NODE_IP}\"\n - \"${SECOND_NODE_IP}\"/" $CONFIG_FILE sed -i "s/cluster\.initial_master_nodes:\n- \"node-1\"/cluster.initial_master_nodes:\n- ${CURRENT_NODE}\n- ${SECOND_NODE}/" $CONFIG_FILE sed -i ':a;N;$!ba;s/plugins\.security\.nodes_dn:\n- "CN=node-1,OU=Wazuh,O=Wazuh,L=California,C=US"/plugins.security.nodes_dn:\n- "CN='"${CURRENT_NODE}"',OU=Wazuh,O=Wazuh,L=California,C=US"\n- "CN='"${SECOND_NODE}"',OU=Wazuh,O=Wazuh,L=California,C=US"/' $CONFIG_FILE if [ $? -eq 0 ]; then echo "Configuration updated successfully. Backup created at ${BACKUP_FILE}" else echo "Error updating configuration." fi # Directory for certificates CERT_DIR="/etc/wazuh-indexer/certs" # Extract certificates echo "Creating certificates directory and extracting certificates..." mkdir -p $CERT_DIR tar -xf ./wazuh-certificates.tar -C $CERT_DIR ./$CURRENT_NODE.pem ./$CURRENT_NODE-key.pem ./admin.pem ./admin-key.pem ./root-ca.pem if [ $? -ne 0 ]; then echo "Error extracting certificates." exit 1 fi # Move and set permissions for certificates echo "Moving and setting permissions for certificates..." mv -n $CERT_DIR/$CURRENT_NODE.pem $CERT_DIR/indexer.pem mv -n $CERT_DIR/$CURRENT_NODE-key.pem $CERT_DIR/indexer-key.pem chmod 500 $CERT_DIR chmod 400 $CERT_DIR/* chown -R wazuh-indexer:wazuh-indexer $CERT_DIR if [ $? -eq 0 ]; then echo "Certificates configured successfully." else echo "Error configuring certificates." fi ```

    • Check wazuh-indexer service

      sudo bash ./03_check_service.sh

Starting wazuh-indexer service...
Synchronizing state of wazuh-indexer.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable wazuh-indexer
wazuh-indexer service is running.
Stopping wazuh-indexer service...
wazuh-indexer service stopped successfully.
Restarting wazuh-indexer service...
wazuh-indexer service is running.
      Test script ```bash #!/bin/bash # SPDX-License-Identifier: Apache-2.0 # The OpenSearch Contributors require contributions made to # this file be licensed under the Apache-2.0 license or a # compatible open source license. # Function to check the status of the wazuh-indexer service check_service_is_running() { systemctl is-active --quiet wazuh-indexer if [ $? -eq 0 ]; then echo "wazuh-indexer service is running." else echo "Error: wazuh-indexer service is not running." >&2 exit 1 fi } # Start wazuh-indexer service echo "Starting wazuh-indexer service..." systemctl daemon-reload systemctl enable wazuh-indexer systemctl start wazuh-indexer # Check if the service is running check_service_is_running # Stop wazuh-indexer service echo "Stopping wazuh-indexer service..." systemctl stop wazuh-indexer # Check if the service is stopped systemctl is-active --quiet wazuh-indexer if [ $? -ne 0 ]; then echo "wazuh-indexer service stopped successfully." else echo "Error: Failed to stop wazuh-indexer service." >&2 exit 1 fi # Restart wazuh-indexer service echo "Restarting wazuh-indexer service..." systemctl restart wazuh-indexer # Check if the service is running after restart check_service_is_running ```

    • Extra manual check

      systemctl status wazuh-indexer
● wazuh-indexer.service - wazuh-indexer
     Loaded: loaded (/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2024-10-19 01:43:50 UTC; 4min 27s ago
       Docs: https://documentation.wazuh.com
   Main PID: 9869 (java)
      Tasks: 44 (limit: 2220)
     Memory: 1.2G
        CPU: 19.813s
     CGroup: /system.slice/wazuh-indexer.service
             └─9869 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Do>