wazuh / wazuh-agent

Wazuh agent, the Wazuh agent for endpoints.
GNU Affero General Public License v3.0
26 stars 18 forks source link

Linux (DEB) Package Creation #143

Closed vikman90 closed 1 week ago

vikman90 commented 1 month ago

Parent Issue:

Description

This issue focuses on creating a DEB package for Debian-based distributions such as Debian and Ubuntu. The package should manage all necessary dependencies and include pre/post-installation scripts to ensure that the Wazuh agent service is automatically enabled and running after installation.

Functional Requirements

Acceptance Criteria

lchico commented 1 month ago

Update

I've begun researching this issue and have started working on a solution.

2024-09-13

I encountered some issues when generating packages using the SPECS files from the repository. After some investigation, I discovered the cause: the files were uploaded with Windows-style line endings (CRLF) instead of Unix-style (LF). This caused the following error during the build process:

dpkg-buildpackage -rsudo -D -us -uc -b -nc
dpkg-buildpackage: warning: using a gain-root-command while being root
dpkg-buildpackage: source package wazuh-agent
dpkg-buildpackage: source version 4.9.0-0
dpkg-buildpackage: source changed by Wazuh, Inc <info@wazuh.com>
 dpkg-source --before-build wazuh-agent-5.0.0
dpkg-buildpackage: host architecture amd64
'pkg-source: error: invalid Format field `3.0 (quilt)
dpkg-buildpackage: error: dpkg-source --before-build wazuh-agent-5.0.0 gave error exit status 25
debuild: fatal error at line 1357:
dpkg-buildpackage -rsudo -D -us -uc -b -nc failed
root@4be8bb734266:/# 

2024-09-16

To avoid potential issues with older OS versions (meaning that when we generate Debian packages using a newer version of Debian, the packages might not work on older Debian versions), I am trying to use the same OS we currently use for generating Debian packages:

lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 7.11 (wheezy)
Release:    7.11
Codename:   wheezy
/bootstrap-vcpkg.sh
Downloading vcpkg-glibc...
curl: option --tlsv1.2: is unknown
curl: try 'curl --help' or 'curl --manual' for more information
 ./bootstrap-vcpkg.sh 
Downloading vcpkg-glibc...
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.
Build curl 8-10.0 ```bash curl-8.10.0# ./configure --with-gnutls --with-secure-transport --with-openssl checking whether to enable maintainer-specific portions of Makefiles... no checking whether make supports nested variables... yes checking whether to enable debug build options... no checking whether to enable compiler optimizer... (assumed) yes checking whether to enable strict compiler warnings... no checking whether to enable compiler warnings as errors... no checking whether to enable curl debug memory tracking... no checking whether to enable hiding of library internal symbols... yes checking whether to enable c-ares for DNS lookups... no checking whether to disable dependency on -lrt... (assumed no) checking whether to enable HTTPSRR support... no checking whether to enable ECH support... no checking for path separator... : ... checking default CA cert bundle/path... configure: want unset ca /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs (capath) checking whether to use built-in CA store of SSL library... no checking CA cert bundle path to embed... no checking for pkg-config... (cached) /usr/bin/pkg-config checking for libpsl options with pkg-config... no checking for psl_builtin in -lpsl... no configure: error: libpsl libs and/or directories were not found where specified! ```

2024-09-17

Based on tier-1, I am trying to generate Debian packages using Debian 10. With this change, I got vcpkg to download the dependencies. However, when the build process started, I encountered problems with certain dependencies and g++ versions. After trying g++-10, it didn't work, so I had to build g++-14 from the source. It looks like I am close to completing the build process. I just have this pending issue to fix:

[ 96%] Building CXX object agent/command_store/CMakeFiles/CommandStore.dir/src/command_store.cpp.o
In file included from /build_wazuh/agent/wazuh-agent-5.0.0/src/agent/command_store/include/command_store.hpp:4,
                 from /build_wazuh/agent/wazuh-agent-5.0.0/src/agent/command_store/src/command_store.cpp:1:
In copy constructor 'constexpr sqlite_manager::Column::Column(const sqlite_manager::Column&)',
    inlined from 'bool command_store::CommandStore::UpdateCommand(const command_store::Command&)' at /build_wazuh/agent/wazuh-agent-5.0.0/src/agent/command_store/src/command_store.cpp:203:5:
/build_wazuh/agent/wazuh-agent-5.0.0/src/agent/sqlite_manager/include/sqlite_manager.hpp:25:11: error: '*(__vector(2) unsigned char*)(&condition + offsetof(sqlite_manager::Column, sqlite_manager::Column::m_notNull))' may be used uninitialized [-Werror=maybe-uninitialized]
   25 |     class Column
      |           ^~~~~~
/build_wazuh/agent/wazuh-agent-5.0.0/src/agent/command_store/src/command_store.cpp: In member function 'bool command_store::CommandStore::UpdateCommand(const command_store::Command&)':
/build_wazuh/agent/wazuh-agent-5.0.0/src/agent/command_store/src/command_store.cpp:192:32: note: 'condition' declared here
  192 |         sqlite_manager::Column condition("id", sqlite_manager::ColumnType::INTEGER, std::to_string(cmd.m_id));
      |                                ^~~~~~~~~
In copy constructor 'constexpr sqlite_manager::Column::Column(const sqlite_manager::Column&)',
    inlined from 'bool command_store::CommandStore::UpdateCommand(const command_store::Command&)' at /build_wazuh/agent/wazuh-agent-5.0.0/src/agent/command_store/src/command_store.cpp:203:5:
/build_wazuh/agent/wazuh-agent-5.0.0/src/agent/sqlite_manager/include/sqlite_manager.hpp:25:11: error: 'condition.sqlite_manager::Column::m_primaryKey' may be used uninitialized [-Werror=maybe-uninitialized]
   25 |     class Column
      |           ^~~~~~
/build_wazuh/agent/wazuh-agent-5.0.0/src/agent/command_store/src/command_store.cpp: In member function 'bool command_store::CommandStore::UpdateCommand(const command_store::Command&)':
/build_wazuh/agent/wazuh-agent-5.0.0/src/agent/command_store/src/command_store.cpp:192:32: note: 'condition' declared here
  192 |         sqlite_manager::Column condition("id", sqlite_manager::ColumnType::INTEGER, std::to_string(cmd.m_id));
      |                                ^~~~~~~~~
cc1plus: all warnings being treated as errors
make[2]: *** [agent/command_store/CMakeFiles/CommandStore.dir/build.make:76: agent/command_store/CMakeFiles/CommandStore.dir/src/command_store.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:396: agent/command_store/CMakeFiles/CommandStore.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

I haven't had time to investigate this further.

2024-09-18

lchico commented 1 month ago

Update

/opt/wazuh-agent/bin/wazuh-agent: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /opt/wazuh-agent/bin/wazuh-agent)
/opt/wazuh-agent/bin/wazuh-agent: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.31' not found (required by /opt/wazuh-agent/bin/wazuh-agent)
/opt/wazuh-agent/bin/wazuh-agent: /lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.15' not found (required by /opt/wazuh-agent/bin/wazuh-agent)

I am not prioritizing this issue for now as there are other things that need to be defined first.

vikman90 commented 1 month ago

Dear @lchico, let me write here a brief of our discussion:

  1. Service Definition: The man 7 file-hierarchy document clearly indicates that the /usr directory is reserved for vendor-provided packages. Therefore, it's more appropriate for us to maintain the service file under /etc/systemd/system/.

  2. User and Group: We haven't defined a change in user/group yet, but it's likely we will do so via Systemd in the future. For now, we should continue using wazuh for both the user and group. This may change later, but we can address it at that point.

  3. Configuration Files: Following the vendor recommendations is the best approach here. I propose we use /etc/wazuh-agent/wazuh-agent.yml for the main configuration file. This aligns with the structure of other files (like those for OpenSearch), and allows us to manage multiple files if needed. Additionally, this would allow for both a manager and an agent to coexist on the same system.

  4. Generated Files Location: For generated files, it's more appropriate to place them in directories like /var/local/wazuh-agent or /var/opt/wazuh-agent, rather than /opt, which is reserved for static files. We'll need to follow system recommendations here as well.

  5. Logs: Currently, agent logs will be sent to stdout and captured by Systemd via Journald. In the future, we may define an option to write logs to files, in which case using a directory like /var/log/wazuh-agent/*.log might be more suitable. This isn't yet fully defined, though.

  6. Dependencies: This requires further discussion. Since we're moving to a single binary, it would be best to statically link as many libraries as possible. If certain libraries are absolutely necessary, we may need to ship them with the agent (similar to how we handle /var/ossec/lib). The most problematic libraries here may be libstdc++ and libgcc_s.

  7. Removing Package: I agree with your approach: remove /etc/opt/wazuh-agent but keep /etc/wazuh.conf. For backups, there's nothing crucial that we need to preserve for now. However, it may make sense to remove the wazuh user as well, since it's only required at runtime.

  8. Purge: The purge process looks good. Removing /etc/opt/wazuh-agent, /etc/wazuh.conf, and the wazuh user/group should suffice.


In conclusion, for all of these items, we need to ensure we follow the system standards for each platform (Debian, RedHat, Windows, macOS), reference the vendor's documentation, and review our requirements. We should adhere to the default standards unless we have a clear reason to deviate, which should be documented thoroughly.

References

lchico commented 1 month ago

First, thank you @vikman90. That sounds very clear to me.

Update

mjcr99 commented 1 month ago

Update

(24/09/2024) Updating Dockerfile to enable compiling with the new repository, covering different dependencies. The GCC version has been downgraded to 13.2.0 since using 14.2 was causing procps port-overlay not to compile. (25/09/2024) Added https://github.com/wazuh/wazuh-agent/pull/159/commits/33047deb1631cf3432125fc43af0d833504e6fe6 to make the procps port-overlay finally compilable. And added more dependencies to the Dockerfile. (26/09/2024) Finally committed https://github.com/wazuh/wazuh-agent/pull/159/commits/c1ff35cdb79c92c8f1fc331e4680c5528575b5ff to upload Dockerfile final version which builds the agent with no problem.

jotacarma90 commented 1 month ago

Update 26/09/2024

Update 27/09/2024

Update 30/09/2024

jotacarma90 commented 1 month ago

Current DEB package

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.

MarcelKemp commented 2 weeks ago

Update

Test branch: test/deb-package-generation

In the branch, I have rebased the rest of the branches and added some steps to the Dockerfile to try to generate a 5.0 package. It still needs modifications to add the service functionality.

(10/21) - Compilation problems due to some vcpkg dependencies, which are not needed for Debian (librpm), but cannot be filtered out in vcpkg in a simple way. Also adapting the code to install, get and run services, and uninstall correctly.

(10/22) - Fixed vcpkg dependencies issue. Found new problem with wazuh-agent compilation, related to flags and logcollector development. Trying to generate a complete package.

(10/23) - Package generated successfully.