search

puppetlabs / puppetlabs-peadm

A Puppet module defining Bolt plans used to automate Puppet Enterprise deployments
Apache License 2.0
29 stars 53 forks source link

Converting an existing PE setup with peadm module fails #311

Closed SimonHoenscheid closed 1 year ago

SimonHoenscheid commented 1 year ago

Describe the Bug

Conversion of existing Installation not successful

Expected Behavior

The conversion finishes sucessfully

Steps to Reproduce

Last week we were trying to use the peadm module to convert our existing PE installation in preparation to use it for future upgrades. Testing this on two environments just consisting of one primary server worked fine before.

Here are some basic infos about the environment:

  1. Bolt is running on the workstation. OS is Win10. (installed via chocolatey)

  2. To access the environment, there is a SSH jump host. This jump host is referenced in an SSH config.

  3. We use the following params file:

    {
"primary_host": "puppetserver-02.example.com",
"compiler_hosts": [
"puppetserver-03.example.com",
"puppetserver-04.example.com",
"puppetserver-05.example.com",
"puppetserver-06.example.com",
"puppetserver-04.example-dev.com"
],
"compiler_pool_address": "puppetserver-02.example.com",
"dns_alt_names": [
"puppetserver-lb.example.com",
"puppet"
]
}

  4. puppetserver-03.example.com are in the same network, puppetserver-04.example-dev.com is in a different network, but also reachable via the jump host.

  5. The Inventory.yaml:

    ---
config:
ssh:
ssh-command: 'ssh'
native-ssh: true
tmpdir: '/var/bolt'
IdentityFile: '/c/Users/user_home/.ssh/id_rsa'
run-as: 'root'
sudo-password: 'supersecret'
private-key: "~/.ssh/id_rsa"
host-key-check: false
user: user_name

  6. The bolt plan output:

PS C:\Users\user_name\code\company_peadm> bolt plan run peadm::convert --params '@params.json' --log-level debug
Loaded project from 'C:/Users/user_name/code/company_peadm'
Analytics opt-out is set, analytics will be disabled
Started with 100 max thread(s)
Loading modules from C:/Program Files/Puppet Labs/Bolt/lib/ruby/gems/2.7.0/gems/bolt-3.26.2/bolt-modules;C:/Users/user_name/code/company_peadm/modules;C:/Users/user_name/code/company_peadm/.modules;C:/Program Files/Puppet Labs/Bolt/lib/ruby/gems/2.7.0/gems/bolt-3.26.2/modules
Loaded inventory from C:/Users/user_name/code/company_peadm/inventory.yaml
Did not find config for puppetserver-03.example.com in inventory
Did not find config for puppetserver-04.example.com in inventory
Did not find config for puppetserver-05.example.com in inventory
Did not find config for puppetserver-06.example.com in inventory
Did not find config for puppetserver-04.example-dev.com in inventory
Starting: plan peadm::convert
Starting: plan peadm::convert
Did not find config for puppetserver-02.example.com in inventory
# Gathering information
# Gathering information
Starting: task peadm::cert_data on 6 targets
Starting: task peadm::cert_data on 6 targets
Authentication method 'gssapi-with-mic' (Kerberos) is not available.
Running task peadm::cert_data with '{"_task":"peadm::cert_data"}' on ["puppetserver-05.example.com"]
Running task peadm::cert_data with '{"_task":"peadm::cert_data"}' on ["puppetserver-04.example-dev.com"]
Running task peadm::cert_data with '{"_task":"peadm::cert_data"}' on ["puppetserver-02.example.com"]
Running task peadm::cert_data with '{"_task":"peadm::cert_data"}' on ["puppetserver-04.example.com"]
Running task peadm::cert_data with '{"_task":"peadm::cert_data"}' on ["puppetserver-03.example.com"]
Running task peadm::cert_data with '{"_task":"peadm::cert_data"}' on ["puppetserver-06.example.com"]
Running task 'peadm::cert_data' on puppetserver-05.example.com
Running task 'peadm::cert_data' on puppetserver-02.example.com
Running task 'peadm::cert_data' on puppetserver-03.example.com
Running task 'peadm::cert_data' on puppetserver-04.example-dev.com
Running task 'peadm::cert_data' on puppetserver-04.example.com
Running task 'peadm::cert_data' on puppetserver-06.example.com
{"target":"puppetserver-04.example-dev.com","action":"task","object":null,"status":"failure","value":{"_error":{"details":{"file":"C:/Users/user_name/code/company_peadm/.modules/peadm/plans/convert.pp","line":63},"kind":"puppetlabs.tasks/connect-error","msg":"Failed to connect to puppetserver-04.example-dev.com: Host key verification failed.\n","issue_code":"CONNECT_ERROR"}}}
{"target":"puppetserver-06.example.com","action":"task","object":null,"status":"failure","value":{"_error":{"details":{"file":"C:/Users/user_name/code/company_peadm/.modules/peadm/plans/convert.pp","line":63},"kind":"puppetlabs.tasks/connect-error","msg":"Failed to connect to puppetserver-06.example.com: ssh: connect to host puppetserver-06.example.com port 22: Connection timed out\n","issue_code":"CONNECT_ERROR"}}}
{"target":"puppetserver-03.example.com","action":"task","object":null,"status":"failure","value":{"_error":{"details":{"file":"C:/Users/user_name/code/company_peadm/.modules/peadm/plans/convert.pp","line":63},"kind":"puppetlabs.tasks/connect-error","msg":"Failed to connect to puppetserver-03.example.com: ssh: connect to host puppetserver-03.example.com port 22: Connection timed out\n","issue_code":"CONNECT_ERROR"}}}
{"target":"puppetserver-04.example.com","action":"task","object":null,"status":"failure","value":{"_error":{"details":{"file":"C:/Users/user_name/code/company_peadm/.modules/peadm/plans/convert.pp","line":63},"kind":"puppetlabs.tasks/connect-error","msg":"Failed to connect to puppetserver-04.example.com: ssh: connect to host puppetserver-04.example.com port 22: Connection timed out\n","issue_code":"CONNECT_ERROR"}}}
{"target":"puppetserver-05.example.com","action":"task","object":null,"status":"failure","value":{"_error":{"details":{"file":"C:/Users/user_name/code/company_peadm/.modules/peadm/plans/convert.pp","line":63},"kind":"puppetlabs.tasks/connect-error","msg":"Failed to connect to puppetserver-05.example.com: ssh: connect to host puppetserver-05.example.com port 22: Connection timed out\n","issue_code":"CONNECT_ERROR"}}}
{"target":"puppetserver-02.example.com","action":"task","object":null,"status":"failure","value":{"_error":{"details":{"file":"C:/Users/user_name/code/company_peadm/.modules/peadm/plans/convert.pp","line":63},"kind":"puppetlabs.tasks/connect-error","msg":"Failed to connect to puppetserver-02.example.com: ssh: connect to host puppetserver-02.example.com port 22: Connection timed out\n","issue_code":"CONNECT_ERROR"}}}
Finished: task peadm::cert_data with 6 failures in 21.17 sec
Finished: task peadm::cert_data with 6 failures in 21.17 sec
Finished: plan peadm::convert in 21.22 sec
Finished: plan peadm::convert in 21.22 sec
Failed on puppetserver-02.example.com:
  Failed to connect to puppetserver-02.example.com: ssh: connect to host puppetserver-02.example.com port 22: Connection timed out
Failed on puppetserver-03.example.com:
  Failed to connect to puppetserver-03.example.com: ssh: connect to host puppetserver-03.example.com port 22: Connection timed out
Failed on puppetserver-04.example.com:
  Failed to connect to puppetserver-04.example.com: ssh: connect to host puppetserver-04.example.com port 22: Connection timed out
Failed on puppetserver-05.example.com:
  Failed to connect to puppetserver-05.example.com: ssh: connect to host puppetserver-05.example.com port 22: Connection timed out
Failed on puppetserver-06.example.com:
  Failed to connect to puppetserver-06.example.com: ssh: connect to host puppetserver-06.example.com port 22: Connection timed out
Failed on puppetserver-04.example-dev.com:
  Failed to connect to puppetserver-04.example-dev.com: Host key verification failed.
Failed on 6 targets: puppetserver-02.example.com,puppetserver-03.example.com,puppetserver-04.example.com,puppetserver-05.example.com,puppetserver-06.example.com,puppetserver-04.example-dev.com
Ran on 6 targets
  1. We are seeing multiple connection issues after the first connections via the jump host just worked fine.

Environment

ody commented 1 year ago

@SimonHoenscheid Can you please modify your inventory.yaml file to include an array of FQDNs which you provide in your params.json? Similar to the "Example inventory.yaml Bolt inventory file" that is documented here. When Bolt is unable to find a node in the inventory it attempts to generate the appropriate object and from the log you provided, I suspect it isn't constructing the object in a way that will get you connecting through the jump host.

mcka1n commented 1 year ago

Closing this ticket due to inactivity