search

quattor / aii

Automated Installation Infrastructure
www.quattor.org
Apache License 2.0
7 stars 16 forks source link

aii-shellfe cannot verify server certificates when fetching profiles #260

Closed nowack73 closed 7 years ago

nowack73 commented 7 years ago

In Quattor 17.2, aii-shellfe cannot get profiles because of a certificate verification error:

> aii-shellfe --configure XXXX
[INFO] No existing cache /tmp/aii/XXXX/data/aHR0cHM6Ly9xdWF0dG9yMS5waHlzaWsucnd0aC1hYWNoZW4uZGUvcHJvZmlsZXMvaG9tZTMtc2w3LnhtbA==, not specifying the modification date while retrieving
[WARN] Got an unexpected result while retrieving https://xxxxx/profiles/XXXX.xml: 500 Can't connect to xxxxx:443 (SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed)

ncm-aiiserver writes entries for ca_file, cert_file, and key_file into /etc/aii/aii-shellfe.conf, but the temporary configuration of ccm-fetch (/tmp/aii/XXXX/ccm.conf) does not contain these entries. Therefore the new (compared to Quattor 16.8) download method cannot verify the certificate of the server.

In my case, a workaround is to change cdburl in /etc/aii/aii-shellfe.conf from https://xxxx/... to file:///.... in order to access the file locally.

stdweird commented 7 years ago

@nowack73 can you send me --debug 5 output?

nowack73 commented 7 years ago

Here are the debug ouput and some configuration files:

# aii-shellfe --configure xxxxx --debug 5
[DEBUG] Nodes for remove: 
[DEBUG] removelist: 
[DEBUG] Using Net::HTTPS SSL_SOCKET_CLASS IO::Socket::SSL
[DEBUG] Using LWP::UserAgent version 5.833
[DEBUG] LWP::UserAgent is recent enough to support verify_hostname for IO::Socket::SSL
[DEBUG] Using LWP::UserAgent ssl_opts SSL_ca_file: /etc/sindes/certs/ca-quattor.server.some.where.crt SSL_cert_file: /etc/sindes/certs/client_cert.pem SSL_key_file: /etc/sindes/certs/client_privatekey.pem verify_hostname: 1
[DEBUG] update_env PERL_LWP_SSL_VERIFY_HOSTNAME = 1.
[DEBUG] update_env delete PERL_NET_HTTPS_SSL_SOCKET_CLASS.
[DEBUG] update_env PERL_LWP_SSL_VERIFY_HOSTNAME = 1.
[DEBUG] update_env delete PERL_NET_HTTPS_SSL_SOCKET_CLASS.
[DEBUG] Downloading profiles-info: https://quattor.server.some.where/profiles/profiles-info.xml
[DEBUG] Parsing XML file from https://quattor.server.some.where/profiles/profiles-info.xml
[DEBUG] Added xxxxx to the list
[DEBUG] Nodes for configure: xxxxx
[DEBUG] configurelist: xxxxx
[DEBUG] Fetching profile: https://quattor.server.some.where/profiles/xxxxx.xml
[VERB] Opening file /tmp/aii/xxxxx/ccm.conf
[VERB] Running the command: /sbin/restorecon /tmp/aii/xxxxx/ccm.conf
/sbin/restorecon:  Warning no default label for /tmp/aii/xxxxx/ccm.conf
[VERB] File /tmp/aii/xxxxx/ccm.conf was modified
[DEBUG] config file /tmp/aii/xxxxx/ccm.conf  changed.
[DEBUG] base_url is not defined in configuration
[DEBUG] URL is https://quattor.server.some.where/profiles/xxxxx.xml
[DEBUG] No lockfile /tmp/aii/xxxxx/fetch.lock found: no lock
[DEBUG] flock on /tmp/aii/xxxxx/fetch.lock gave has_lock 1
[DEBUG] Writing global lock /tmp/aii/xxxxx/global.lock
[VERB] Opening file /tmp/aii/xxxxx/global.lock
[VERB] Running the command: /sbin/restorecon /tmp/aii/xxxxx/global.lock
/sbin/restorecon:  Warning no default label for /tmp/aii/xxxxx/global.lock
[VERB] File /tmp/aii/xxxxx/global.lock was modified
[VERB] Opening file /tmp/aii/xxxxx/latest.cid
[DEBUG] No reference file/pipe via source option. Returning false.
[VERB] Opening file /tmp/aii/xxxxx/profile.0/profile.url
[DEBUG] No reference file/pipe via source option. Returning false.
[VERB] Not saving file /tmp/aii/xxxxx/profile.0/profile.url
[VERB] Opening file /tmp/aii/xxxxx/profile.0/profile.xml
[DEBUG] No reference file/pipe via source option. Returning false.
[VERB] Not saving file /tmp/aii/xxxxx/profile.0/profile.xml
[DEBUG] Current URL https://quattor.server.some.where/profiles/xxxxx.xml is different from the previous fetched one . Forcing download.
[INFO] No existing cache /tmp/aii/xxxxx/data/aHR0cHM6Ly9xdWF0dG9yMS5waHlzaWsucnd0aC1hYWNoZW4uZGUvcHJvZmlsZXMvYW1zLXduMDUueG1s, not specifying the modification date while retrieving
[VERB] FORCE set, not setting if_modified_since in request
[DEBUG] Using Net::HTTPS SSL_SOCKET_CLASS IO::Socket::SSL
[DEBUG] Using LWP::UserAgent version 5.833
[DEBUG] LWP::UserAgent is recent enough to support verify_hostname for IO::Socket::SSL
[DEBUG] Using LWP::UserAgent ssl_opts verify_hostname: 1
[DEBUG] update_env PERL_LWP_SSL_VERIFY_HOSTNAME = 1.
[DEBUG] update_env delete PERL_NET_HTTPS_SSL_SOCKET_CLASS.
[DEBUG] update_env PERL_LWP_SSL_VERIFY_HOSTNAME = 1.
[DEBUG] update_env delete PERL_NET_HTTPS_SSL_SOCKET_CLASS.
[WARN] Got an unexpected result while retrieving https://quattor.server.some.where/profiles/xxxxx.xml: 500 Can't connect to quattor.server.some.where:443 (SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed)
[DEBUG] https://quattor.server.some.where/profiles/xxxxx.xml: try 1 of 3: sleeping for 30 seconds
[VERB] FORCE set, not setting if_modified_since in request
[DEBUG] Using Net::HTTPS SSL_SOCKET_CLASS IO::Socket::SSL
[DEBUG] Using LWP::UserAgent version 5.833
[DEBUG] LWP::UserAgent is recent enough to support verify_hostname for IO::Socket::SSL
[DEBUG] Using LWP::UserAgent ssl_opts verify_hostname: 1
[DEBUG] update_env PERL_LWP_SSL_VERIFY_HOSTNAME = 1.
[DEBUG] update_env delete PERL_NET_HTTPS_SSL_SOCKET_CLASS.
[DEBUG] update_env PERL_LWP_SSL_VERIFY_HOSTNAME = 1.
[DEBUG] update_env delete PERL_NET_HTTPS_SSL_SOCKET_CLASS.
[WARN] Got an unexpected result while retrieving https://quattor.server.some.where/profiles/xxxxx.xml: 500 Can't connect to quattor.server.some.where:443 (SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed)
[DEBUG] https://quattor.server.some.where/profiles/xxxxx.xml: try 2 of 3: sleeping for 30 seconds
[VERB] FORCE set, not setting if_modified_since in request
[DEBUG] Using Net::HTTPS SSL_SOCKET_CLASS IO::Socket::SSL
[DEBUG] Using LWP::UserAgent version 5.833
[DEBUG] LWP::UserAgent is recent enough to support verify_hostname for IO::Socket::SSL
[DEBUG] Using LWP::UserAgent ssl_opts verify_hostname: 1
[DEBUG] update_env PERL_LWP_SSL_VERIFY_HOSTNAME = 1.
[DEBUG] update_env delete PERL_NET_HTTPS_SSL_SOCKET_CLASS.
[DEBUG] update_env PERL_LWP_SSL_VERIFY_HOSTNAME = 1.
[DEBUG] update_env delete PERL_NET_HTTPS_SSL_SOCKET_CLASS.
[WARN] Got an unexpected result while retrieving https://quattor.server.some.where/profiles/xxxxx.xml: 500 Can't connect to quattor.server.some.where:443 (SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed)
[DEBUG] https://quattor.server.some.where/profiles/xxxxx.xml: try 3 of 3: sleeping for 30 seconds
[ERROR] Failed to fetch profile https://quattor.server.some.where/profiles/xxxxx.xml
[VERB] Running the command: /sbin/restorecon /tmp/aii/xxxxx/latest.cid
/sbin/restorecon:  Warning no default label for /tmp/aii/xxxxx/latest.cid
[VERB] File /tmp/aii/xxxxx/latest.cid was modified
[ERROR] Impossible to fetch profile for xxxxx: failed to download. Skipping.
[ERROR] No nodes left to process after checking for protected hosts
[DEBUG] closing down

ccm.conf used by aii-shellfe:

# cat /tmp/aii/xxxxx/ccm.conf 
cache_root /tmp/aii/xxxxx
json_typed 1
tabcompletion 0

For comparision: ccm.conf generated by ncm-ccm:

# cat /etc/ccm.conf
ca_file /etc/sindes/certs/ca-quattor.server.some.where.crt
cache_root /var/lib/ccm
cert_file /etc/sindes/certs/client_cert.pem
debug 0
force 0
get_timeout 30
key_file /etc/sindes/certs/client_privatekey.pem
lock_retries 3
lock_wait 30
profile https://quattor.server.some.where/profiles/xxxxx.xml
retrieve_retries 3
retrieve_wait 30
world_readable 0

Configuration of aii-shellfe:

# cat /etc/aii/aii-shellfe.conf 
# File generated by ncm-aiiserver
# Do not edit
ca_file = /etc/sindes/certs/ca-quattor.server.some.where.crt
#cdburl = file:///opt/profiles
cdburl = https://quattor.server.some.where/profiles
cert_file = /etc/sindes/certs/client_cert.pem
key_file = /etc/sindes/certs/client_privatekey.pem
nbpdir = /data/aii/nbp/pxelinux.cfg
osinstalldir = /data/aii/www/ks
profile_format = xml
use_fqdn = 0