Cloudera Manager plugins SSL connect attempt failed | --tls-noverify is ignored

[marbaa]

Hi,

not sure if cloudera plugins are maintained anymore, or there is some support given :)

RHEL 8.4 Cloudera Manager 7.1.7 SP1

In this case Cloudera during installation created own self signed certificates, but --tls-noverify is for some reason ignored (I think). If I specify the path for Cloudera's CA with --ssl-CA-path I get same output. Installed perl version 5.26

First time our RHEL8.x installation, it was pretty pain for me to install all dependecies, but I believe I have all installed.

# .../Nagios-Plugins/check_cloudera_manager_status.pl -H XXX.XXX.XXX.XXX -u xxx -p xxx --tls-noverify --CM-mgmt -vvv
verbose mode on

check_cloudera_manager_status.pl version 0.3  =>  Hari Sekhon Utils version 1.19.6

host:                     xxx
port:                     7180
user:                     xxx
password:                 <omitted>

setting timeout to 10 secs

TLS enabled:              true
TLS noverify:             true
overriding default http port 7180 to default tls port 7183
querying https://xxx:7183/api/v6/cm/service
returned HTML:

Can't connect to xxx:7183 (certificate verify failed)

SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed at /usr/local/share/perl5/LWP/Protocol/http.pm line 50.

http code: 500
message: Can't connect to xxx:7183 (certificate verify failed)
CRITICAL: failed to query Cloudera Manager at 'https://xxx:7183': 500 Can't connect to xxx.xxx.xxx.xxx:7183 (certificate verify failed)

[HariSekhon]

Cloudera is super legacy tech today so I don't support this any more as I've not worked at any company using this tech for a few years now.

I've had a quick look and it should be ignoring the certificate on this line of code:

https://github.com/HariSekhon/lib/blob/b7f172afe78f57c0ca639a66669fee6d864892bc/HariSekhon/ClouderaManager.pm#L158

Could you try adding --debug to see if you can enable any more information from the LWP useragent?

[marbaa]

Yeah, customers somehow still want to use Cloudera. I'm sorry, where to put --debug?

check_cloudera_manager_status.pl doesn't have this option

[HariSekhon]

See if --debug switch on the command line gives you any additional information as it should enable the LWP useragent debug mode.

[marbaa]

check_cloudera_manager_status.pl -vv -H xxx -u xxx -p xxx --tls-noverify --CM-mgmt --debug
2022-10-17 13:19:38 +0200  verbose mode on

2022-10-17 13:19:38 +0200  check_cloudera_manager_status.pl version 0.3  =>  Hari Sekhon Utils version 1.19.6

2022-10-17 13:19:38 +0200  host:                     xxx
2022-10-17 13:19:38 +0200  port:                     7180
2022-10-17 13:19:38 +0200  user:                     xxx
2022-10-17 13:19:38 +0200  password:                 <omitted>
2022-10-17 13:19:38 +0200
2022-10-17 13:19:38 +0200  setting timeout to 10 secs

2022-10-17 13:19:38 +0200  TLS enabled:              true
2022-10-17 13:19:38 +0200  TLS noverify:             true
2022-10-17 13:19:38 +0200  overriding default http port 7180 to default tls port 7183
2022-10-17 13:19:38 +0200  querying https://xxx:7183/api/v6/cm/service
** GET https://xxx:7183/api/v6/cm/service ==> 500 Can't connect to xxx:7183 (certificate verify failed)
2022-10-17 13:19:38 +0200  returned HTML:

Can't connect to 5.202.18.17:7183 (certificate verify failed)

SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed at /usr/local/share/perl5/LWP/Protocol/http.pm line 50.

2022-10-17 13:19:38 +0200  http code: 500
2022-10-17 13:19:38 +0200  message: Can't connect to xxx:7183 (certificate verify failed)
CRITICAL: failed to query Cloudera Manager at 'https://xxx:7183': 500 Can't connect to xxx:7183 (certificate verify failed)

[HariSekhon]

Might be an upstream library bug - you could try the workaround there:

https://linux.debian.bugs.dist.narkive.com/LCDg657A/bug-907853-liblwp-protocol-https-perl-turning-off-hostname-verification-does-not-work

[marbaa]

That helped. I've changed line 158 in ClouderaManager.pm you highligted from

$ua->ssl_opts( verify_hostname => 0 );

to

$ua->ssl_opts( SSL_verify_mode => 0, verify_hostname => 0 );

Output:

# .../Nagios-Plugins/check_cloudera_manager_status.pl -H xxx -u xxx -p xxx --tls-noverify --CM-mgmt
OK: Cloudera Manager Mgmt service state=STARTED

Everything looks good now. Thank you very much for fast support.

[HariSekhon]

I've backported that change now, thanks!