CLI tool elasticsearch-setup-passwords does not work if TLS auth is enabled and certificates reference only domain names and not IPs

[redgryphon]

Elasticsearch version (bin/elasticsearch --version): Version: 7.10.2, Build: default/rpm/747e1cc71def077253878a59143c1f785afa92b9/2021-01-13T00:42:12.435326Z, JVM: 15.0.

Plugins installed: []

JVM version (java -version):

openjdk version "15.0.1" 2020-10-20
OpenJDK Runtime Environment AdoptOpenJDK (build 15.0.1+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 15.0.1+9, mixed mode, sharing)

OS version (uname -a if on a Unix-like system): Linux hostname.example.com 4.18.0-240.10.1.el8_3.x86_64 #1 SMP Mon Jan 18 17:05:51 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior: After successfully configuring Elasticsearch HTTPS support the server is reachable connecting with this curl CLI command curl -vvv --silent --user 'elastic:changeme' 'https://hostname.example.com:9200/'

Steps to reproduce:

Just create certificates for Elasticsearch which don't include the IPs in the Subject Alt Names but only the FQDN (in the example case is hostname.example.com). Install Elasticsearch from the repositories following the official guide and activate the HTTPS support with the options in the elasticsearch.yml file:

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: "/etc/elasticsearch/certs/elasticsearch.key"
xpack.security.http.ssl.certificate: "/etc/elasticsearch/certs/elasticsearch.crt"
xpack.security.http.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]

Use the CLI command elasticsearch-setup-passwords to configure the passwords. It will try to connect to https://IPofthefirstinterfaceofthemachine:9200/_security/_authenticate?pretty. Obviously you get the error:

SSL connection to https://IPofthefirstinterfaceofthemachine:9200/_security/_authenticate?pretty failed: No subject alternative names matching IP address IPofthefirstinterfaceofthemachine found

The part of the code responsible for the generation of the URL is this. What could make everything work automatically is the retrieval of the FQDN from the machine itself as it is shown by the hostname command.

Confirmed workaround: you must need to specify the url option when calling elasticsearch-setup-passwords as in the example: /usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive -u 'https://hostname.example.com:9200/'.

[elasticmachine]

Pinging @elastic/es-security (Team:Security)

[ywangd]

This behaviour is expected and documented as

If xpack.security.http.ssl.enabled is set to true, you must specify an HTTPS URL.

With above being said, I wonder whether we could simplify this by automatically enable certificate verification mode when an URL is not specified, e.g. something like:

./bin/elasticsearch-setup-passwords interactive -E xpack.security.http.ssl.verification_mode=certificate

Since the command sources configuration from the local elasticsearch installation, it might be reasonable to assume that certs from the published IP address are applicable to the IP. I'll raise this to the team and report back.

[redgryphon]

This behaviour is expected and documented as

If xpack.security.http.ssl.enabled is set to true, you must specify an HTTPS URL.

I think that the error that the program prints is somehow cryptic. Why not making it print something like "You must specify a URL since xpack.security.http.ssl.enabled is set to true". It would help a lot troubleshooting people who are good sysadmins but not developers.

With above being said, I wonder whether we could simplify this by automatically enable certificate verification mode when an URL is not specified, e.g. something like:

./bin/elasticsearch-setup-passwords interactive -E xpack.security.http.ssl.verification_mode=certificate

Since the command sources configuration from the local elasticsearch installation, it might be reasonable to assume that certs from the published IP address are applicable to the IP. I'll raise this to the team and report back.

This is not a good idea since it assumes trust over something that clearly should not be trusted, even if it's just SAN configuration of the certificates. At this point just give the sysadmin a useful hint on what went wrong and let them figure out what to do (trivial since the error would point directly to the docs at that point).

[albertzaharovits]

Why not making it print something like "You must specify a URL since xpack.security.http.ssl.enabled is set to true".

This is not strictly required. The tool has smarts to construct the URL for the https scheme as well. But it relies on some ES yml settings to be specified.

I wonder whether we should disable TLS hostname verification (rely on certificate verification only) for URLs connecting to localhost? Can we assume that no one's MITM localhost connections? I think this is a reasonable assumption to make.

[wwalker]

This caused me no end of confusion because the --url is not in the help message. If it were in the help message, or there was a man page, it would have saved me hours and hours.

[root@es-master-dev-003 ~]# /usr/share/elasticsearch/bin/elasticsearch-setup-passwords --help
Sets the passwords for reserved users

Commands
--------
auto - Uses randomly generated passwords
interactive - Uses passwords entered by a user

Non-option arguments:
command

Option             Description
------             -----------
-E <KeyValuePair>  Configure a setting
-h, --help         Show help
-s, --silent       Show minimal output
-v, --verbose      Show verbose output
[root@es-master-dev-003 ~]# /usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto --url https://es-master-dev-003.dev.tgsre.net:9200
[root@es-master-dev-003 ~]# man elasticsearch-setup-passwords
No manual entry for elasticsearch-setup-passwords
[root@es-master-dev-003 ~]#