SSH Key Pair Module fails with 'context deadline exceeded' error msg

[oaomer]

IBM CloudCollection version: 1.23.0 Terraform Provider version: 1.23.0

Using the following ibm_pi_key module as outlined in the task below to create a new SSH Key pair in a cloud service:

• name: 5- Add new SSH Key if no key was found by that name ibm_pi_key: pi_key_name: "{{ name_ssh_key }}" pi_ssh_key: "{{ ssh_public_key }}" pi_cloud_instance_id: "{{ pi_cloud_instance_id }}"

It failed with the below error. Much appreciate taking a look and getting that fixed. There is additional logic around the task to ensure there is no key with same name exists in the same cloud service; so that is not the issue :) "rc": 1, "stdout": "ibm_pi_key.ansible_20210409-190908: Creating...\n", "stderr": "\nError: Failed to create the key Failed to Create PI Key ansibleSvrKey9 :Post \"https://eu-de.power-iaas.cloud.ibm.com/pcloud/v1/tenants/a4417f93849b4d1deed0253e59ebe168/sshkeys\": context deadline exceeded\n\n on ibm_pi_key_ansible_20210409-190908.tf line 1, in resource \"ibm_pi_key\" \"ansible_20210409-190908\":\n 1: resource ibm_pi_key \"ansible_20210409-190908\" {\n\n\n", "warnings": []}

[surajsub]

I was able to use straight terraform and it worked just fine. This could be just something with the connectivity to the IBM cloud service. I suggest , deleting the key and then trying again please.

[oaomer]

I tried again and it continues to fail... I went ahead and tried the straight terraform code execution and it continues to fail with the same error. does it makes sense to have a common working session to compare and contrast to see why it does not work for me? Adding @Anil-CM @jaywcarman for input from ansible perspective.

[jaywcarman]

@surajsub @oaomer For starters I would compare:

• IBM Cloud Terraform provider version (1.23.0)
• Terraform version (0.12.20)
• IBM Cloud PowerVS endpoint (i.e. what region are you testing against)

@oaomer You could re-run the straight Terraform with the env TF_LOG=TRACE to get some more details for us to look at (see Debugging Terraform docs).

[oaomer]

tf-apply.log

@jaywcarman I have set the TF_LOG to trace and ran the [./terraform apply] and captured the logs in the attached tf-apply.log file. While I am not very well versed in debugging TF, the 20-lines of output in the log file starting from line 555 match exactly what Suraj has shown me in his working example. That lead me to believe that the ansible code is building the terraform state file correctly yet the execution continued to fail for me.

As to which PowerVS endpoint/region, this fails consistently regardless. I have mostly been testing against Frankfort 1 but the attached output was actually from an execution against Washington DC.

[jaywcarman]

All I see from your tf-apply.log is that the sshkeys Power Cloud API call is timing out (no new information there...).

Maybe try increasing the timeout to see if the call eventually receives a response? I think the ibmcloud_timeout parameter controls this. https://cloud.ibm.com/docs/ibm-cloud-provider-for-terraform?topic=ibm-cloud-provider-for-terraform-provider-reference#provider-parameter-ov

[oaomer]

hey Jay; you need to help me understand this better LOL Looked at the doc link you shared above and found out that the default timeout is 60 seconds. Execution never reaches that before it fails. So, I wanted to change the timeout just to prove my theory that it is not an issue of timeout. So, I set an env var IC_TIMEOUT to 200 and timed the next execution of the playbook... That execution completed in ~35 seconds but with success and the ssh key was created!!! Something is not clear to me but I am happy that it works :-)

[jaywcarman]

@hkantare - Could you help us to understand the ibmcloud_timeout provider parameter? Is the value in seconds? Is this a timeout for each API call? @oaomer's trace and experimentation don't seem to align with the provider documentation.