search

wazuh / wazuh-packages

Wazuh - Tools for packages creation
https://wazuh.com
GNU General Public License v2.0
105 stars 96 forks source link

Failed checks due to missing wazuh-indexer package #2749

Closed MarcelKemp closed 9 months ago

MarcelKemp commented 10 months ago

Description

PR #2746 checks are failing due to the following reasons:

Error details

Wazuh agent RPM packages build

Error:

+ /usr/local/bin/rpmbuild --define '_sysconfdir /etc' --define '_topdir /build_wazuh/rpmbuild' --define '_threads 2' --define '_release merge.4.8.0.into.4.8.1' --define '_localstatedir /var/ossec' --define '_debugenabled no' --define '_download_content_enabled ' --target x86_64 -ba /build_wazuh/rpmbuild/SPECS/wazuh-agent-4.8.1.spec
error: Macro %_download_content_enabled has empty body

The error appears with the build of the RPM package via GitHub action:

All in one and Distributed cases

Error (or similar):

No package wazuh-indexer-4.8.1 available.
...
05/01/2024 17:23:32 ERROR: Wazuh indexer installation failed.

All In One - Amazonlinux 2_base installation — Test_unattended

All In One - Centos 7 installation — Test_unattended

All In One - Centos 8 installation — Test_unattended

All In One - Rhel 7 installation — Test_unattended

All In One - Rhel 8 installation — Test_unattended

All In One - Rhel 9 installation — Test_unattended

All In One - Ubuntu bionic installation — Test_unattended

All In One - Ubuntu focal installation — Test_unattended

All In One - Ubuntu jammy installation — Test_unattended

All In One - Ubuntu xenial installation — Test_unattended

Distributed - Certificates — Test_unattended_distributed

Distributed - Wazuh indexer installation — Test_unattended_distributed

Distributed Cases - manager - centos - Wazuh indexer installation — Test_unattended_distributed_cases

Distributed Cases - manager - ubuntu - Wazuh indexer installation — Test_unattended_distributed_cases

Distributed Cases - manager_dashboard - centos - Wazuh indexer installation — Test_unattended_distributed_cases

Distributed Cases - manager_dashboard - ubuntu - Wazuh indexer installation — Test_unattended_distributed_cases

Distributed Cases - manager_indexer - centos - Wazuh indexer installation — Test_unattended_distributed_cases

Distributed Cases - manager_indexer - ubuntu - Wazuh indexer installation — Test_unattended_distributed_cases

DoD

vikman90 commented 10 months ago

Let's assign this issue to @wazuh/qa.

rauldpm commented 10 months ago

About this issue

This issue presents the following main errors:

  1. RPM agent build error in GHA -> @wazuh/core
  2. Wazuh installation assistant tests error -> @wazuh/cicd

From what I can see of the error in the tests, it is because the packages do not exist, which should be the responsibility of CICD as it is related to its tests, on the other hand, in the link to the package builder, is possible to see that it fails when building them due to the lack of the following TD resources

Packages_rpm_indexer_x86_64_4-8-1 Packages_deb_indexer_amd64_4-8-1 Packages_deb_dashboard_amd64_4-8-1 Packages_rpm_dashboard_x86_64_4-8-1

The creation of these Task Definitions is carried out (or should be) during upward merges by the @wazuh/core team, an example of the latest generation of resources (4.7.2) https://github.com/wazuh/internal-devel-requests/issues/558, this process should be included in some procedure such as merges or support for new patches

It is also possible to see an error when uploading a package due to an already-known error https://github.com/wazuh/wazuh-jenkins/issues/4639 whose solution is to repeat the build

An issue should be opened for each team for each error cc @MarcelKemp (this issue could be edited and used for one of them), with an additional issue to create the missing ECS TD resources.

jotacarma90 commented 10 months ago

Update 15/01/2024

Images created:

C:\Users\Vhalgarv\Wazuh\wazuh-packages\wazuhapp\opensearch-dashboards\Docker (4.8.1 -> origin)
λ docker images
REPOSITORY                                                        TAG                                   IMAGE ID       CREATED          SIZE
966237403726.dkr.ecr.us-east-1.amazonaws.com/package-generation   dashboardapp_builder_noarch_4-8-1     2e03b866e528   30 seconds ago   647MB
966237403726.dkr.ecr.us-east-1.amazonaws.com/package-generation   deb_indexer_builder_amd64_4-8-1       6aa13c1a4f91   24 minutes ago   647MB
966237403726.dkr.ecr.us-east-1.amazonaws.com/package-generation   rpm_indexer_builder_x86_64_4-8-1      66c7b089c166   26 minutes ago   600MB
966237403726.dkr.ecr.us-east-1.amazonaws.com/package-generation   deb_dashboard_builder_amd64_4-8-1     ea6464a9c22c   28 minutes ago   972MB
966237403726.dkr.ecr.us-east-1.amazonaws.com/package-generation   rpm_dashboard_builder_x86_64_4-8-1    8a11c3fece53   2 hours ago      1.29GB
966237403726.dkr.ecr.us-east-1.amazonaws.com/package-generation   base_dashboard_builder_amd64_4-8-1    c2acc156e067   2 hours ago      875MB
966237403726.dkr.ecr.us-east-1.amazonaws.com/package-generation   base_dashboard_builder_x86_64_4-8-1   c2acc156e067   2 hours ago      875MB
966237403726.dkr.ecr.us-east-1.amazonaws.com/package-generation   base_indexer_builder_amd64_4-8-1      77b3c89d68f2   2 hours ago      918MB
966237403726.dkr.ecr.us-east-1.amazonaws.com/package-generation   base_indexer_builder_x86_64_4-8-1     77b3c89d68f2   2 hours ago      918MB

Images uploaded: image

Creating task definitions for those images in progress...

Update 16/01/2024

TDs created: image

Logs group created. Solved an error in the images due to Windows endofline text that generate problems executing build.sh scripts. Launched builds in Jenkins to create app, dashboard and indexer packages success:

🟢 https://ci.wazuh.info/job/Packages_builder/180879/ 🟢 https://ci.wazuh.info/job/Packages_builder_tier/3369/

ncvicchi commented 10 months ago

@rauldpm we have solved the task definition section, and we think we can close this issue, but since in your commentary you mention a 2nd problem related to @wazuh/cicd, we are not sure if we can or cannot close it. Could you please confirm?

Should this enough to allow us to close this issue?

rauldpm commented 10 months ago

If the AWS ECR/ECS resources have been created and the packages are available, I guess that yes, the issue can be closed, but are the PR checks still failing?

jotacarma90 commented 10 months ago

Update 18/01/2024

I have recreated the PR branch: https://github.com/wazuh/wazuh-packages/pull/2746 To relaunch the checks, I created the indexer and dashboard packages in staging. I have launched this pipe for it: https://ci.wazuh.info/job/Packages_builder_tier/3370

Once everything is ready, I will relaunch the checks and everything should be fine.

Update 19/01/2024

We need to delay the ETA because the PR checks cannot go through the error Raul commented to upload the packages to staging.

@wazuh/cicd Can you take a look at these builds failing to upload packages (deb indexer and dashboard packages 4.8.1)? https://ci.wazuh.info/job/Packages_builder/181487/ https://ci.wazuh.info/job/Packages_builder/181488/

Issue blocked until the provisioning problems are solved: https://github.com/wazuh/wazuh-jenkins/issues/4639

jnasselle commented 10 months ago

Hi @jotacarma90 ,

Package upload procedure is failing due to timeout (2hs). Example https://ci.wazuh.info/job/Packages_uploader_repository/11147/consoleFull

18:45:57  Timeout set to expire in 2 hr 0 min
.
.
.
.
 454 / 538   84.39% 12m43sSending interrupt signal to process
20:46:04  /home/ec2-user/workspace/Packages_uploader_repository@tmp/durable-886c1cec/script.sh: line 1:  5775 Terminated              ./upload_dev_debian.sh -f staging/apt -r /mnt/efs/packages -s3b packages-dev.wazuh.com -s -i **** -sp ****
20:46:04  script returned exit code 143

As you mention https://github.com/wazuh/wazuh-jenkins/issues/4639 should solve the problem

jotacarma90 commented 9 months ago

Update 23/01/2024

Issue unlocked, packages can be generated. Jenkins builds:

Updated ETA to today, everything should be finished.

Update 24/01/2024

Analyzing more pipeline errors. Found some problems generating the debian manager package, related to vulnerability detector possibly because of some merges related to the VD refactor after the upwards merges. Trying to understand some errors in the pipelines of both PR_packages and PR check github actions. Last builds executed:

jotacarma90 commented 9 months ago

Conclusion

Hello team, after some problems with failed checks, what I have done is to open a PR dummy modifying the same files that were modified in the PR we were testing:

We believe that what was happening is that, since the PR was already merged, the PR checks were not detecting the changes in the branch, and they were still taking images of the moment in which those checks were launched for the first time. In short, now with the PR dummy all the checks are green for 4.8.1.

So this issue can be closed.

MarcelKemp commented 9 months ago

Review

The problems reported in the issue have been fixed, and all checks have been verified to pass correctly :green_circle: So LGTM!

GJ Jose!

MarcelKemp commented 9 months ago

We seem to have fixed version 4.8.1, however 4.8.2 requires the same steps, so I am reopening the issue:

[!note] I have reopened the following issue which seems to be the most related to the problem, so I am closing this one again.