prometheus-hanadb_exporter installation through zypper does not install latest upstream version

[schniber]

Hello All,

I am currently trying to set up the HANA DB Exporter on a SLES 15 for SAP SP4 Instance running on AWS.

As per the documentation, I'd like to use the configuration option to have the exporter read database credentials from AWS Secrets Manager.

Unfortunately, the service gives me the following error:

2023-03-28 14:33:20,458 ERROR hanadb_exporter Traceback (most recent call last):
  File "/usr/bin/hanadb_exporter", line 9, in <module>
    main.run()
  File "/usr/lib/python3.6/site-packages/hanadb_exporter/main.py", line 134, in run
    timeout=config.get('timeout', 600))
  File "/usr/lib/python3.6/site-packages/hanadb_exporter/db_manager.py", line 105, in start
    kwargs.get('userkey', None), kwargs.get('user', ''), kwargs.get('password', ''))
  File "/usr/lib/python3.6/site-packages/hanadb_exporter/db_manager.py", line 83, in _get_connection_data
    'Provided user data is not valid. userkey or user/password pair must be provided')
ValueError: Provided user data is not valid. userkey or user/password pair must be provided

I have checked the main.py file located in /usr/lib/python3.6/site-packages/hanadb_exporter/main.py and it looks like there's a discrepency between what I have on the instance and the file in here https://github.com/SUSE/hanadb_exporter/blob/master/hanadb_exporter/main.py => It looks like the files are not up to date when installing the prometheus hana db exporter from zypper.

Below is the content of the folder /usr/lib/python3.6/site-packages/hanadb_exporter:

total 36
-rw-r--r-- 1 root root  158 Nov  2  2020 __init__.py
drwxr-xr-x 2 root root  208 Mar 28 14:32 __pycache__
-rw-r--r-- 1 root root 5355 Nov  2  2020 db_manager.py
-rw-r--r-- 1 root root 4624 Nov  2  2020 main.py
-rw-r--r-- 1 root root 7273 Nov  2  2020 prometheus_exporter.py
-rw-r--r-- 1 root root 2811 Nov  2  2020 prometheus_metrics.py
-rw-r--r-- 1 root root 1807 Nov  2  2020 utils.py

=> It is missing the secrets_manager.py file which reinforces my doubts on the installation from zypper (RPM) does not take the latest available version upstream.

FYI, this is the output I am getting when running zypper in:

zypper info prometheus-hanadb_exporter
Refreshing service 'Basesystem_Module_x86_64'.
Refreshing service 'Containers_Module_x86_64'.
Refreshing service 'Desktop_Applications_Module_x86_64'.
Refreshing service 'Development_Tools_Module_x86_64'.
Refreshing service 'Legacy_Module_x86_64'.
Refreshing service 'Public_Cloud_Module_x86_64'.
Refreshing service 'Python_3_Module_x86_64'.
Refreshing service 'SAP_Applications_Module_x86_64'.
Refreshing service 'SUSE_Linux_Enterprise_High_Availability_Extension_x86_64'.
Refreshing service 'SUSE_Linux_Enterprise_Live_Patching_x86_64'.
Refreshing service 'SUSE_Linux_Enterprise_Server_for_SAP_Applications_x86_64'.
Refreshing service 'Server_Applications_Module_x86_64'.
Refreshing service 'Web_and_Scripting_Module_x86_64'.
Loading repository data...
Reading installed packages...
Information for package prometheus-hanadb_exporter:
---------------------------------------------------
Repository     : SLE-Module-SAP-Applications15-SP4-Pool
Name           : prometheus-hanadb_exporter
Version        : 0.7.3+git.1604318097.c2b074f-3.6.1
Arch           : noarch
Vendor         : SUSE LLC <https://www.suse.com/>
Support Level  : Level 3
Installed Size : 124.3 KiB
Installed      : Yes
Status         : up-to-date
Source package : prometheus-hanadb_exporter-0.7.3+git.1604318097.c2b074f-3.6.1.src
Upstream URL   : https://github.com/SUSE/hanadb_exporter
Summary        : SAP HANA database metrics exporter
Description    :
    SAP HANA database metrics exporter 

my config file is as following:

{
    "listen_address": "0.0.0.0",
    "exposition_port": 9668,
    "multi_tenant": true,
    "timeout": 30,
    "hana": {
        "host": "localhost",
        "port": 32213,
        "aws_secret_name": "NAME_OF_THE_SECRET",
        "ssl": true,
        "ssl_validate_cert": false
    },
    "logging": {
        "config_file": "/etc/hanadb_exporter/logging_config.ini"
    }
}

Is there anything I am missing ?

Is there a way to have zypper install the latest upstream version ?

Thanks in advance.

Bests.

[arbulu89]

Hello, Unfortunately we don't have an officially updated version of this package in zypper. I had a look and the development repository is broken. I will see if I can fix it, but I cannot promise anything

[schniber]

Hello @arbulu89,

Alright, thanks again for your support.

Looking forward to get the sync issue fixed. Meanwhile I will explore the manual clone option.

Thanks.

[arbulu89]

Hey @schniber , I managed to fix one upstream repository. It should be aligned with github master/main branch. Keep in mind that this is development repo. https://build.opensuse.org/package/show/network:ha-clustering:sap-deployments:devel/prometheus-hanadb_exporter

There you can choose the distro of your preference. Find the installation commands here: https://software.opensuse.org//download.html?project=network%3Aha-clustering%3Asap-deployments%3Adevel&package=prometheus-hanadb_exporter

[wombelix]

@arbulu89 regarding your comments:

Unfortunately we don't have an officially updated version of this package in zypper. I had a look and the development repository is broken. I will see if I can fix it, but I cannot promise anything

and:

I managed to fix one upstream repository. It should be aligned with github master/main branch. Keep in mind that this is development repo.

My understanding is, that the package is directly supported by SUSE as part of the SLES for SAP subscription:

Repository     : SLE-Module-SAP-Applications15-SP4-Pool
Name           : prometheus-hanadb_exporter
Version        : 0.7.3+git.1604318097.c2b074f-3.6.1
Arch           : noarch
Vendor         : SUSE LLC <https://www.suse.com/>
Support Level  : Level 3

So shouldn't a new official release be triggered soon based on the fix you did? Or what's the way forward to bring the latest version into the official repositories? Also, as soon https://github.com/SUSE/hanadb_exporter/pull/112 is merged, how can we get this fix into an official release?

As @schniber mentioned, this has an impact for all AWS customer when using AWS Secrets Manager also IMDSv2 support is missing right now (https://github.com/SUSE/hanadb_exporter/issues/111) and requires his PR to be fixed.

[diegoakechi]

My understanding is, that the package is directly supported by SUSE as part of the SLES for SAP subscription:

Repository     : SLE-Module-SAP-Applications15-SP4-Pool
Name           : prometheus-hanadb_exporter
Version        : 0.7.3+git.1604318097.c2b074f-3.6.1
Arch           : noarch
Vendor         : SUSE LLC <https://www.suse.com/>
Support Level  : Level 3

So shouldn't a new official release be triggered soon based on the fix you did? Or what's the way forward to bring the latest version into the official repositories? Also, as soon #112 is merged, how can we get this fix into an official release?

What @arbulu89 mentioned refers to the open-source upstream version hosted here on GitHub. For convenience, we provide sometimes RPMs packages from Open Build Service, but there was some problem there and he has fixed that. These are not SLES for SAP official packages and will not receive enterprise support.

As @schniber mentioned, this has an impact for all AWS customer when using AWS Secrets Manager also IMDSv2 support is missing right now (#111) and requires his PR to be fixed.

As soon as the code is validated and merged, we will trigger the internal SUSE process to release the fixes as part of our commercial distro (SLES for SAP) and to have it released on our official repositories ASAP.

If there are urgent customer requests, you can ask them to report the issue via official SUSE support and they will receive assistance to get this via PTF if needed.

[wombelix]

For convenience, we provide sometimes RPMs packages from Open Build Service, but there was some problem there and he has fixed that. These are not SLES for SAP official packages and will not receive enterprise support.

Make sense.

As soon as the code is validated and merged, we will trigger the internal SUSE process to release the fixes as part of our commercial distro (SLES for SAP) and to have it released on our official repositories ASAP.

Great, can you share some insights about the process? One topic from @schniber was that the SLES4SAP package was behind the latest upstream release. What triggers the process to create a new official release? Does that happen every X weeks / months? Is it related to the OBS rpm build mentioned above? Or manually as soon some PRs were merged that make sense for a new release?

If there are urgent customer requests, you can ask them to report the issue via official SUSE support and they will receive assistance to get this via PTF if needed.

Sure that's always an option, thanks for pointing it out, I guess the goal that @schniber had was just to speed up the process a little by contributing directly a fix based on SUSE's upstream first policy :)

[diegoakechi]

As soon as the code is validated and merged, we will trigger the internal SUSE process to release the fixes as part of our commercial distro (SLES for SAP) and to have it released on our official repositories ASAP.

Great, can you share some insights about the process? One topic from @schniber was that the SLES4SAP package was behind the latest upstream release. What triggers the process to create a new official release? Does that happen every X weeks / months? Is it related to the OBS rpm build mentioned above? Or manually as soon some PRs were merged that make sense for a new release?

The process involves prioritization, technical evaluation about regression risks, customer demands and etc + the natural software development process, like backporting into SLE distro, QA, etc.

The best way to influence this is to work with SUSE AWS Partner team and raise the arguments to them, so they can work with Product Management to set the right priorities and expectations.

If there are urgent customer requests, you can ask them to report the issue via official SUSE support and they will receive assistance to get this via PTF if needed.

Sure that's always an option, thanks for pointing it out, I guess the goal that @schniber had was just to speed up the process a little by contributing directly a fix based on SUSE's upstream first policy :)

That is amazing, thanks for that! It is an open-source project and contributions are always welcome.

[wombelix]

I appreciate your feedback and insights, happy to see you actively involved.

SUSE AWS Partner team

Cool thanks, I work as Partner SA for the Linux Alliance Team at AWS, so I guess in addition to reach out to my counterparts at SUSE, the regular SUSE + AWS Engineering Sync would also be fine to bring such topics up then.

[schniber]

Thanks a lot team for your support ! Long live open source and contributions !

[wombelix]

@arbulu89 I think we are good to go to close this issue, new release arrived upstream and in SLES for SAP 15 (SP2, SP3, SP4) repositories based on the latest updated in https://bugzilla.suse.com/show_bug.cgi?id=1210869

SAP Applications Module 15-SP2 (src): prometheus-hanadb_exporter-0.8.0+git.1681379025.bf6cd7c-150200.3.9.1
SAP Applications Module 15-SP3 (src): prometheus-hanadb_exporter-0.8.0+git.1681379025.bf6cd7c-150200.3.9.1
SAP Applications Module 15-SP4 (src): prometheus-hanadb_exporter-0.8.0+git.1681379025.bf6cd7c-150200.3.9.1