Review Wazuh embedded Python interpreter related files

[fdalmaup]

Wazuh version	Install type	Action performed	Platform
-	Manager	Install	-

In the https://github.com/wazuh/wazuh/issues/13265 epic, we are upgrading the Wazuh embedded Python interpreter to the 3.10 version. There are some files related to the interpreter that should be reviewed and modified in order to avoid conflicts when this epic is merged, for example: https://github.com/wazuh/wazuh-packages/blob/8023b94a374010c6f99d81146bcd76d62e9db5ae/debs/SPECS/wazuh-manager/debian/rules#L214-L219

Tasks

• [x] Review that the changes in the embedded interpreter have their counterpart in the corresponding wazuh-packages files and determine the necessary modifications to be done.
• [x] Modify those files and test that the packages are correctly built.

[mhamra]

UPDATE

• I've reviewed PR 2501
• I've created a virtual machine in AWS.
• I've installed docker ant git in the virtual machine
• I've followed the steps of Wazuh packages generation guide to create RPMs and DEBs using the package-branch enhancement/2499-python3.10-support and wazuh branch fix/13267-Python3.10-test-warnings to run the generate_rpm_package.sh and generate_debian_package.sh scripts.
• The scripts take a long time to generate the .deb and rpm package files
• I've downloaded the deb package to test them in a local environment.
• The generate_rpm_package.sh script finished with errors building the rpm package.

• sed -i s:WAZUH_HOME_TMP:/var/ossec:g src/init/templates/ossec-hids-suse.init
• cp -rp src/init/templates/ossec-hids-suse.init /build_wazuh/rpmbuild/BUILDROOT/wazuh-manager-4.8.0-2.x86_64/var/ossec/packages_files/manager_installation_scripts/src/init/
• cp -rp etc/templates/config/generic/alerts.template etc/templates/config/generic/ar-commands.template etc/templates/config/generic/ar-definitions.template etc/templates/config/generic/auth.template etc/templates/config/generic/cluster.template etc/templates/config/generic/global-ar.template etc/templates/config/generic/global.template etc/templates/config/generic/header-comments.template etc/templates/config/generic/localfile-commands.template etc/templates/config/generic/localfile-logs etc/templates/config/generic/logging.template etc/templates/config/generic/osquery.template etc/templates/config/generic/remote-secure.template etc/templates/config/generic/rootcheck.agent.template etc/templates/config/generic/rootcheck.manager.template etc/templates/config/generic/rule_test.template etc/templates/config/generic/rules.template etc/templates/config/generic/sca.files etc/templates/config/generic/sca.manager.files etc/templates/config/generic/sca.template etc/templates/config/generic/syscheck.agent.template etc/templates/config/generic/syscheck.manager.template etc/templates/config/generic/wodle-ciscat.template etc/templates/config/generic/wodle-syscollector.template etc/templates/config/generic/wodle-vulnerability-detector.manager.template /build_wazuh/rpmbuild/BUILDROOT/wazuh-manager-4.8.0-2.x86_64/var/ossec/packages_files/manager_installation_scripts/etc/templates/config/generic
• cp -rp etc/templates/config/centos/5 etc/templates/config/centos/6 etc/templates/config/centos/7 etc/templates/config/centos/8 etc/templates/config/centos/rootcheck.agent.template etc/templates/config/centos/rootcheck.manager.template etc/templates/config/centos/sca.files /build_wazuh/rpmbuild/BUILDROOT/wazuh-manager-4.8.0-2.x86_64/var/ossec/packages_files/manager_installation_scripts/etc/templates/config/centos
• cp -rp etc/templates/config/rhel/5 etc/templates/config/rhel/6 etc/templates/config/rhel/7 etc/templates/config/rhel/8 etc/templates/config/rhel/9 etc/templates/config/rhel/rootcheck.agent.template etc/templates/config/rhel/rootcheck.manager.template etc/templates/config/rhel/sca.files /build_wazuh/rpmbuild/BUILDROOT/wazuh-manager-4.8.0-2.x86_64/var/ossec/packages_files/manager_installation_scripts/etc/templates/config/rhel
• install -m 0640 src/init/adduser.sh src/init/darwin-addusers.sh src/init/darwin-delete-oldusers.sh src/init/darwin-init.sh src/init/delete-oldusers.sh src/init/dist-detect.sh src/init/functions.sh src/init/fw-check.sh src/init/init.sh src/init/inst-functions.sh src/init/language.sh src/init/pkg_installer.sh src/init/pkg_installer_mac.sh src/init/register_configure_agent.sh src/init/replace_manager_ip.sh src/init/shared.sh src/init/template-select.sh src/init/update.sh src/init/wazuh-client.sh src/init/wazuh-local.sh src/init/wazuh-server.sh /build_wazuh/rpmbuild/BUILDROOT/wazuh-manager-4.8.0-2.x86_64/var/ossec/packages_files/manager_installation_scripts/src/init
• cp src/VERSION /build_wazuh/rpmbuild/BUILDROOT/wazuh-manager-4.8.0-2.x86_64/var/ossec/packages_files/manager_installation_scripts/src/
• cp src/REVISION /build_wazuh/rpmbuild/BUILDROOT/wazuh-manager-4.8.0-2.x86_64/var/ossec/packages_files/manager_installation_scripts/src/
• exit 0 Processing files: wazuh-manager-4.8.0-2.x86_64 error: File not found: /build_wazuh/rpmbuild/BUILDROOT/wazuh-manager-4.8.0-2.x86_64/var/ossec/var/db/mitre.db

RPM build errors: bogus date in %changelog: Thu Feb 24 2020 support info@wazuh.com - 3.11.4 bogus date in %changelog: Mon Aug 8 2019 support support@wazuh.com - 3.9.5 bogus date in %changelog: Mon Jul 12 2019 support support@wazuh.com - 3.9.4 bogus date in %changelog: Mon Jul 02 2019 support support@wazuh.com - 3.9.3 bogus date in %changelog: Mon Jun 11 2019 support support@wazuh.com - 3.9.2 bogus date in %changelog: Mon Jun 01 2019 support support@wazuh.com - 3.9.1 File not found: /build_wazuh/rpmbuild/BUILDROOT/wazuh-manager-4.8.0-2.x86_64/var/ossec/var/db/mitre.db

[mhamra]

UPDATE

I've successfully created the rpm and deb packages using branch 2154-deprecate-centos6-debian7. This branch has the modified files that release python3.9 references to python3.10.

[mhamra]

UPDATE

DEB package installation

I've successfully installed a wazuh server using the package generated with python 3.10

offline DEB package installation

```console ### root@vagrant:~/wazuh-packages/debs# mv /tmp/wazuh-manager_4.8.0-p31_amd64.deb /vagrant root@vagrant:~/wazuh-packages/debs# cd /vagrant root@vagrant:/vagrant# dpkg -i wazuh-manager*.deb Selecting previously unselected package wazuh-manager. (Reading database ... 44775 files and directories currently installed.) Preparing to unpack wazuh-manager_4.8.0-p31_amd64.deb ... Unpacking wazuh-manager (4.8.0-p31) ... Setting up wazuh-manager (4.8.0-p31) ... root@vagrant:/vagrant# systemctl daemon-reload systemctl enable wazuh-manager systemctl start wazuh-manager Created symlink /etc/systemd/system/multi-user.target.wants/wazuh-manager.service → /lib/systemd/system/wazuh-manager.service. root@vagrant:/vagrant# root@vagrant:/vagrant# /var/ossec/ -bash: /var/ossec/: Is a directory root@vagrant:/vagrant# cd /var/ossec/ root@vagrant:/var/ossec# bin/wazuh-control status wazuh-clusterd not running... wazuh-modulesd is running... wazuh-monitord is running... wazuh-logcollector is running... wazuh-remoted is running... wazuh-syscheckd is running... wazuh-analysisd is running... wazuh-maild not running... wazuh-execd is running... wazuh-db is running... wazuh-authd is running... wazuh-agentlessd not running... wazuh-integratord not running... wazuh-dbd not running... wazuh-csyslogd not running... wazuh-apid is running... root@vagrant:/var/ossec# cd framework root@vagrant:/var/ossec/framework# cd python root@vagrant:/var/ossec/framework/python# cd bin root@vagrant:/var/ossec/framework/python/bin# python3 --version Python 3.10.12 root@vagrant:/var/ossec/framework/python/bin# cd .. root@vagrant:/var/ossec/framework/python# cd lib root@vagrant:/var/ossec/framework/python/lib# cd python3.10/ root@vagrant:/var/ossec/framework/python/lib/python3.10# ls abc.py collections dis.py hmac.py mimetypes.py posixpath.py shelve.py subprocess.py turtle.py aifc.py _collections_abc.py distutils html modulefinder.py pprint.py shlex.py sunau.py types.py _aix_support.py colorsys.py doctest.py http multiprocessing profile.py shutil.py symtable.py typing.py antigravity.py _compat_pickle.py email idlelib netrc.py pstats.py signal.py _sysconfigdata__linux_x86_64-linux-gnu.py unittest argparse.py compileall.py encodings imaplib.py nntplib.py pty.py _sitebuiltins.py sysconfig.py urllib ast.py _compression.py ensurepip imghdr.py ntpath.py _py_abc.py site-packages tabnanny.py uuid.py asynchat.py concurrent enum.py importlib nturl2path.py __pycache__ site.py tarfile.py uu.py asyncio config-3.10-x86_64-linux-gnu filecmp.py imp.py numbers.py pyclbr.py smtpd.py telnetlib.py venv asyncore.py configparser.py fileinput.py inspect.py opcode.py py_compile.py smtplib.py tempfile.py warnings.py base64.py contextlib.py fnmatch.py io.py operator.py _pydecimal.py sndhdr.py test wave.py bdb.py contextvars.py fractions.py ipaddress.py optparse.py pydoc_data socket.py textwrap.py weakref.py binhex.py copy.py ftplib.py json os.py pydoc.py socketserver.py this.py _weakrefset.py bisect.py copyreg.py functools.py keyword.py _osx_support.py _pyio.py sqlite3 _threading_local.py webbrowser.py _bootsubprocess.py cProfile.py __future__.py lib2to3 pathlib.py queue.py sre_compile.py threading.py wsgiref bz2.py crypt.py genericpath.py lib-dynload pdb.py quopri.py sre_constants.py timeit.py xdrlib.py calendar.py csv.py getopt.py LICENSE.txt __phello__.foo.py random.py sre_parse.py tkinter xml cgi.py ctypes getpass.py linecache.py pickle.py reprlib.py ssl.py tokenize.py xmlrpc cgitb.py curses gettext.py locale.py pickletools.py re.py statistics.py token.py zipapp.py chunk.py dataclasses.py glob.py logging pipes.py rlcompleter.py stat.py traceback.py zipfile.py cmd.py datetime.py graphlib.py lzma.py pkgutil.py runpy.py stringprep.py tracemalloc.py zipimport.py codecs.py dbm gzip.py mailbox.py platform.py sched.py string.py trace.py zoneinfo codeop.py decimal.py hashlib.py mailcap.py plistlib.py secrets.py _strptime.py tty.py code.py difflib.py heapq.py _markupbase.py poplib.py selectors.py struct.py turtledemo root@vagrant:/var/ossec/framework/python/lib/python3.10# cd site-packages/ root@vagrant:/var/ossec/framework/python/lib/python3.10/site-packages# ls aiohttp dateutil inflection.py pyparsing-2.4.7.dist-info aiohttp-3.8.5.dist-info defusedxml itsdangerous pyparsing.py aiohttp_cache defusedxml-0.6.0.dist-info itsdangerous-2.0.0.dist-info python_dateutil-2.8.1.dist-info aiohttp_cache-2.2.0.dist-info _distutils_hack jinja2 python_jose-3.1.0.dist-info aiohttp_cors distutils-precedence.pth Jinja2-3.0.0.dist-info pythonjsonlogger aiohttp_cors-0.7.0.dist-info docker jmespath python_json_logger-2.0.2.dist-info aiohttp_jinja2 docker-6.0.0.dist-info jmespath-0.9.5.dist-info pytz aiohttp_jinja2-1.5.1.dist-info dockerpycreds jose pytz-2020.1.dist-info aioredis docker_pycreds-0.4.0.dist-info jsonschema pyximport aioredis-1.3.1.dist-info docutils jsonschema-2.6.0.dist-info PyYAML-5.4.1.dist-info aiosignal docutils-0.15.2.dist-info libcst README.txt aiosignal-1.2.0.dist-info ecdsa libcst-0.3.20.dist-info requests api ecdsa-0.16.1.dist-info libfuturize requests-2.31.0.dist-info api-4.8.0.dist-info envparse-0.2.0.dist-info libpasteurize rsa asn1crypto envparse.py markupsafe rsa-4.7.2.dist-info asn1crypto-1.3.0.dist-info examples MarkupSafe-2.1.2.dist-info s3transfer async_timeout flask more_itertools s3transfer-0.4.2.dist-info async_timeout-4.0.2.dist-info Flask-2.2.5.dist-info more_itertools-8.2.0.dist-info scripts attr frozenlist multidict secure attrs-20.3.0.dist-info frozenlist-1.2.0.dist-info multidict-5.2.0.dist-info secure-0.2.1.dist-info azure future mypy_extensions-0.4.3.dist-info setuptools azure_common-1.1.25.dist-info future-0.18.3.dist-info mypy_extensions.py setuptools-65.5.1.dist-info azure_storage_blob-2.1.0.dist-info google numpy six-1.16.0.dist-info azure_storage_common-2.1.0.dist-info google_api_core-1.30.0.dist-info numpy-1.26.0.dist-info six.py boto3 google_api_core-1.30.0-py3.9-nspkg.pth numpy.libs sqlalchemy boto3-1.17.85.dist-info googleapis_common_protos-1.51.0.dist-info openapi_spec_validator SQLAlchemy-1.3.11-py3.10.egg-info botocore googleapis_common_protos-1.51.0-py3.10-nspkg.pth openapi_spec_validator-0.2.6.dist-info tabulate-0.8.9.dist-info botocore-1.20.85.dist-info google_auth-1.28.0.dist-info packaging tabulate.py cachetools google_auth-1.28.0-py3.9-nspkg.pth packaging-20.9.dist-info typing_extensions-3.10.0.2.dist-info cachetools-4.1.0.dist-info google_cloud_core-1.7.1.dist-info past typing_extensions.py certifi google_cloud_core-1.7.1-py3.9-nspkg.pth pathlib-1.0.1.dist-info typing_inspect-0.7.1.dist-info certifi-2023.7.22.dist-info google_cloud_pubsub-2.7.1.dist-info pathlib.py typing_inspect.py cffi google_cloud_pubsub-2.7.1-py3.9-nspkg.pth pip urllib3 cffi-1.15.1.dist-info google_cloud_storage-1.39.0.dist-info pip-23.0.1.dist-info urllib3-1.26.5.dist-info _cffi_backend.cpython-310-x86_64-linux-gnu.so google_cloud_storage-1.39.0-py3.9-nspkg.pth pkg_resources uvloop chardet google_crc32c proto uvloop-0.17.0.dist-info chardet-3.0.4.dist-info google_crc32c-1.1.2.dist-info protobuf-3.19.6.dist-info wazuh charset_normalizer google_resumable_media-1.3.1.dist-info protobuf-3.19.6-py3.9-nspkg.pth wazuh-4.8.0.dist-info charset_normalizer-2.0.4.dist-info google_resumable_media-1.3.1-py3.9-nspkg.pth proto_plus-1.19.0.dist-info websocket click grpc psutil websocket_client-0.57.0.dist-info click-8.1.3.dist-info grpc_google_iam_v1-0.12.3.dist-info psutil-5.9.0.dist-info werkzeug clickclick grpc_google_iam_v1-0.12.3-py3.10-nspkg.pth pyarrow Werkzeug-2.2.3.dist-info clickclick-20.10.2.dist-info grpcio-1.58.0.dist-info pyarrow-13.0.0.dist-info xmltodict-0.12.0.dist-info connexion hiredis pyasn1 xmltodict.py connexion-2.14.2.dist-info hiredis-2.2.3.dist-info pyasn1-0.4.8.dist-info _yaml cryptography idna pyasn1_modules yaml cryptography-41.0.4.dist-info idna-2.9.dist-info pyasn1_modules-0.2.8.dist-info yarl Cython importlib_metadata __pycache__ yarl-1.7.0.dist-info Cython-0.29.21.dist-info importlib_metadata-3.10.1.dist-info pycparser zipp-3.3.2.dist-info cython.py inflection-0.3.1.dist-info pycparser-2.21.dist-info zipp.py root@vagrant:/var/ossec/framework/python/lib/python3.10/site-packages# cd api root@vagrant:/var/ossec/framework/python/lib/python3.10/site-packages/api# ls alogging.py authentication.py constants.py encoder.py middlewares.py __pycache__ spec util.py api_exception.py configuration.py controllers __init__.py models signals.py uri_parser.py validator.py root@vagrant:/var/ossec/framework/python/lib/python3.10/site-packages/api# cd .. root@vagrant:/var/ossec/framework/python/lib/python3.10/site-packages# cd wazuh root@vagrant:/var/ossec/framework/python/lib/python3.10/site-packages/wazuh# ls active_response.py cdb_list.py cluster.py decoder.py __init__.py __main__.py mitre.py rbac rule.py security.py syscheck.py task.py agent.py ciscat.py core event.py logtest.py manager.py __pycache__ rootcheck.py sca.py stats.py syscollector.py vulnerability.py root@vagrant:/var/ossec/framework/python/lib/python3.10/site-packages/wazuh# cd .. root@vagrant:/var/ossec/framework/python/lib/python3.10/site-packages# cd .. root@vagrant:/var/ossec/framework/python/lib/python3.10# cd .. root@vagrant:/var/ossec/framework/python/lib# cd .. root@vagrant:/var/ossec/framework/python# bin/python3 Python 3.10.13 (main, Sep 29 2023, 15:57:40) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import api >>> import wazuh >>> ```

RPM package installation

I've successfully installed a wazuh server using the package generated with python 3.10

offline RPM package installation

```console [root@server-2 vagrant]# rpm -ivh wazuh-manager*.rpm Verifying... ################################# [100%] Preparing... ################################# [100%] Updating / installing... 1:wazuh-manager-4.8.0-p310 ################################# [100%] [root@server-2 vagrant]# systemctl daemon-reload systemctl enable wazuh-manager systemctl start wazuh-manager Created symlink /etc/systemd/system/multi-user.target.wants/wazuh-manager.service → /usr/lib/systemd/system/wazuh-manager.service. [root@server-2 vagrant]# cd /var/ossec/ [root@server-2 ossec]# bin/wazuh-control status wazuh-clusterd not running... wazuh-modulesd is running... wazuh-monitord is running... wazuh-logcollector is running... wazuh-remoted is running... wazuh-syscheckd is running... wazuh-analysisd is running... wazuh-maild not running... wazuh-execd is running... wazuh-db is running... wazuh-authd is running... wazuh-agentlessd not running... wazuh-integratord not running... wazuh-dbd not running... wazuh-csyslogd not running... wazuh-apid is running... [root@server-2 ossec]# framework/python/bin/python3 --version Python 3.10.13 [root@server-2 ossec]# framework/python/bin/python3 Python 3.10.13 (main, Sep 29 2023, 15:57:40) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import wazuh >>> import api >>> ```

[GGP1]

Review

The deb and rpm packages installations using the embedded interpreter files were carried out successfully, all services were running without errors and the version used by the Wazuh Python embedded interpreter was 3.10.13. The goal of the issue was accomplished and I therefore conclude this issue should advance to the final review step.