josenk / terraform-provider-esxi

Terraform-provider-esxi plugin
GNU General Public License v3.0
540 stars 154 forks source link

Error: Failed to connect to esxi host: Client Connection Error #141

Closed testpshell closed 3 years ago

testpshell commented 3 years ago

Thank you for this provider @ josenk . I am unable to connect to ESXI for some reason,any help/guidance would be greatly appreciated. Describe the bug Error: Failed to connect to esxi host: Client Connection Error │ │ │ with provider["registry.terraform.io/josenk/esxi"], │ on main.tf line 7, in provider "esxi": │ 7: provider "esxi" {

To Reproduce Steps to reproduce the behavior:

terraform init is working terraform init 2021-05-26T22:33:40.899+0800 [DEBUG] Adding temp file log sink: C:\Users\ADMINI~1\AppData\Local\Temp\1\terraform-log215093271 2021-05-26T22:33:40.942+0800 [INFO] Terraform version: 0.15.4 2021-05-26T22:33:40.943+0800 [INFO] Go runtime version: go1.16.3 2021-05-26T22:33:40.943+0800 [INFO] CLI args: []string{"C:\Binaries\terraform.exe", "init"} 2021-05-26T22:33:40.945+0800 [DEBUG] Attempting to open CLI config file: C:\Users\Administrator\AppData\Roaming\terraform.rc 2021-05-26T22:33:40.945+0800 [DEBUG] File doesn't exist, but doesn't need to. Ignoring. 2021-05-26T22:33:40.956+0800 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins 2021-05-26T22:33:40.956+0800 [DEBUG] ignoring non-existing provider search directory C:\Users\Administrator\AppData\Roaming\terraform.d\plugins 2021-05-26T22:33:40.957+0800 [DEBUG] ignoring non-existing provider search directory C:\Users\Administrator\AppData\Roaming\HashiCorp\Terraform\plugins 2021-05-26T22:33:40.957+0800 [INFO] CLI command args: []string{"init"}

Initializing the backend... 2021-05-26T22:33:40.973+0800 [DEBUG] New state was assigned lineage "b1b50f51-aa15-ad24-65cf-63acbab2756c" 2021-05-26T22:33:42.428+0800 [DEBUG] checking for provisioner in "." 2021-05-26T22:33:42.428+0800 [DEBUG] checking for provisioner in "C:\Binaries" 2021-05-26T22:33:42.429+0800 [INFO] Failed to read plugin lock file .terraform\plugins\windows_amd64\lock.json: open .terraform\plugins\windows_amd64\lock.json: The system cannot find the path specified.

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work.

If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.

  1. terraform plan is failing with below error

    C:_terraform\esxi>terraform plan 2021-05-26T22:33:51.873+0800 [DEBUG] Adding temp file log sink: C:\Users\ADMINI~1\AppData\Local\Temp\1\terraform-log549349591 2021-05-26T22:33:51.918+0800 [INFO] Terraform version: 0.15.4 2021-05-26T22:33:51.919+0800 [INFO] Go runtime version: go1.16.3 2021-05-26T22:33:51.919+0800 [INFO] CLI args: []string{"C:\Binaries\terraform.exe", "plan"} 2021-05-26T22:33:51.923+0800 [DEBUG] Attempting to open CLI config file: C:\Users\Administrator\AppData\Roaming\terraform.rc 2021-05-26T22:33:51.923+0800 [DEBUG] File doesn't exist, but doesn't need to. Ignoring. 2021-05-26T22:33:51.924+0800 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins 2021-05-26T22:33:51.924+0800 [DEBUG] ignoring non-existing provider search directory C:\Users\Administrator\AppData\Roaming\terraform.d\plugins 2021-05-26T22:33:51.925+0800 [DEBUG] ignoring non-existing provider search directory C:\Users\Administrator\AppData\Roaming\HashiCorp\Terraform\plugins 2021-05-26T22:33:51.925+0800 [INFO] CLI command args: []string{"plan"} 2021-05-26T22:33:51.928+0800 [DEBUG] New state was assigned lineage "47888007-9816-af75-cc81-5fef40410425" 2021-05-26T22:33:52.106+0800 [DEBUG] checking for provisioner in "." 2021-05-26T22:33:52.106+0800 [DEBUG] checking for provisioner in "C:\Binaries" 2021-05-26T22:33:52.107+0800 [INFO] Failed to read plugin lock file .terraform\plugins\windows_amd64\lock.json: open .terraform\plugins\windows_amd64\lock.json: The system cannot find the path specified. 2021-05-26T22:33:52.108+0800 [INFO] backend/local: starting Plan operation 2021-05-26T22:33:52.132+0800 [DEBUG] created provider logger: level=debug 2021-05-26T22:33:52.132+0800 [INFO] provider: configuring client automatic mTLS 2021-05-26T22:33:52.207+0800 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe args=[.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe] 2021-05-26T22:33:52.829+0800 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe pid=5964 2021-05-26T22:33:52.829+0800 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe 2021-05-26T22:33:52.905+0800 [INFO] provider.terraform-provider-esxi_v1.8.1.exe: configuring server automatic mTLS: timestamp=2021-05-26T22:33:52.905+0800 2021-05-26T22:33:52.955+0800 [DEBUG] provider: using plugin: version=5 2021-05-26T22:33:52.955+0800 [DEBUG] provider.terraform-provider-esxi_v1.8.1.exe: plugin address: network=tcp address=127.0.0.1:10000 timestamp=2021-05-26T22:33:52.955+0800 2021-05-26T22:33:53.052+0800 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio" 2021-05-26T22:33:53.053+0800 [DEBUG] No provider meta schema returned 2021-05-26T22:33:53.062+0800 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe pid=5964 2021-05-26T22:33:53.062+0800 [DEBUG] provider: plugin exited 2021-05-26T22:33:53.062+0800 [INFO] terraform: building graph: GraphTypeValidate 2021-05-26T22:33:53.063+0800 [DEBUG] ProviderTransformer: "esxi_guest.vmtest01" (terraform.NodeValidatableResource) needs provider["registry.terraform.io/josenk/esxi"] 2021-05-26T22:33:53.064+0800 [DEBUG] ReferenceTransformer: "var.esxi_hostname" references: [] 2021-05-26T22:33:53.064+0800 [DEBUG] ReferenceTransformer: "var.esxi_hostport" references: [] 2021-05-26T22:33:53.064+0800 [DEBUG] ReferenceTransformer: "var.esxi_hostssl" references: [] 2021-05-26T22:33:53.064+0800 [DEBUG] ReferenceTransformer: "var.esxi_username" references: [] 2021-05-26T22:33:53.064+0800 [DEBUG] ReferenceTransformer: "var.esxi_password" references: [] 2021-05-26T22:33:53.064+0800 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/josenk/esxi\"]" references: [var.esxi_password var.esxi_username var.esxi_hostname var.esxi_hostport var.esxi_hostssl] 2021-05-26T22:33:53.064+0800 [DEBUG] ReferenceTransformer: "esxi_guest.vmtest01" references: [] 2021-05-26T22:33:53.065+0800 [DEBUG] Starting graph walk: walkValidate 2021-05-26T22:33:53.066+0800 [DEBUG] created provider logger: level=debug 2021-05-26T22:33:53.066+0800 [INFO] provider: configuring client automatic mTLS 2021-05-26T22:33:53.141+0800 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe args=[.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe] 2021-05-26T22:33:53.145+0800 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe pid=4128 2021-05-26T22:33:53.145+0800 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe 2021-05-26T22:33:53.215+0800 [INFO] provider.terraform-provider-esxi_v1.8.1.exe: configuring server automatic mTLS: timestamp=2021-05-26T22:33:53.215+0800 2021-05-26T22:33:53.265+0800 [DEBUG] provider: using plugin: version=5 2021-05-26T22:33:53.265+0800 [DEBUG] provider.terraform-provider-esxi_v1.8.1.exe: plugin address: address=127.0.0.1:10000 network=tcp timestamp=2021-05-26T22:33:53.265+0800 2021-05-26T22:33:53.357+0800 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio" 2021-05-26T22:33:53.358+0800 [DEBUG] No provider meta schema returned 2021-05-26T22:33:53.374+0800 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe pid=4128 2021-05-26T22:33:53.374+0800 [DEBUG] provider: plugin exited 2021-05-26T22:33:53.374+0800 [INFO] backend/local: plan calling Plan 2021-05-26T22:33:53.374+0800 [INFO] terraform: building graph: GraphTypePlan 2021-05-26T22:33:53.375+0800 [DEBUG] ProviderTransformer: "esxi_guest.vmtest01 (expand)" (terraform.nodeExpandPlannableResource) needs provider["registry.terraform.io/josenk/esxi"] 2021-05-26T22:33:53.375+0800 [DEBUG] ReferenceTransformer: "esxi_guest.vmtest01 (expand)" references: [] 2021-05-26T22:33:53.375+0800 [DEBUG] ReferenceTransformer: "var.esxi_hostssl" references: [] 2021-05-26T22:33:53.375+0800 [DEBUG] ReferenceTransformer: "var.esxi_username" references: [] 2021-05-26T22:33:53.375+0800 [DEBUG] ReferenceTransformer: "var.esxi_password" references: [] 2021-05-26T22:33:53.375+0800 [DEBUG] ReferenceTransformer: "var.esxi_hostname" references: [] 2021-05-26T22:33:53.375+0800 [DEBUG] ReferenceTransformer: "var.esxi_hostport" references: [] 2021-05-26T22:33:53.375+0800 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/josenk/esxi\"]" references: [var.esxi_username var.esxi_hostname var.esxi_hostport var.esxi_hostssl var.esxi_password] 2021-05-26T22:33:53.376+0800 [DEBUG] Starting graph walk: walkPlan 2021-05-26T22:33:53.376+0800 [DEBUG] created provider logger: level=debug 2021-05-26T22:33:53.377+0800 [INFO] provider: configuring client automatic mTLS 2021-05-26T22:33:53.447+0800 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe args=[.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe] 2021-05-26T22:33:53.450+0800 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe pid=5292 2021-05-26T22:33:53.450+0800 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe 2021-05-26T22:33:53.517+0800 [INFO] provider.terraform-provider-esxi_v1.8.1.exe: configuring server automatic mTLS: timestamp=2021-05-26T22:33:53.517+0800 2021-05-26T22:33:53.564+0800 [DEBUG] provider.terraform-provider-esxi_v1.8.1.exe: plugin address: address=127.0.0.1:10000 network=tcp timestamp=2021-05-26T22:33:53.564+0800 2021-05-26T22:33:53.564+0800 [DEBUG] provider: using plugin: version=5 2021-05-26T22:33:53.659+0800 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio" 2021-05-26T22:33:53.660+0800 [DEBUG] No provider meta schema returned 2021-05-26T22:33:53.662+0800 [DEBUG] provider.terraform-provider-esxi_v1.8.1.exe: pid-5292-config.go:18: [validateEsxiCreds] 2021-05-26T22:33:53.662+0800 [DEBUG] provider.terraform-provider-esxi_v1.8.1.exe: pid-5292-esxi_remote_cmds.go:61: [runRemoteSshCommand] :Connectivity test, get vmware version 2021-05-26T22:33:54.493+0800 [DEBUG] provider.terraform-provider-esxi_v1.8.1.exe: pid-5292-esxi_remote_cmds.go:41: [runRemoteSshCommand] Retry connection: 3 2021-05-26T22:33:56.364+0800 [DEBUG] provider.terraform-provider-esxi_v1.8.1.exe: pid-5292-esxi_remote_cmds.go:41: [runRemoteSshCommand] Retry connection: 2 2021-05-26T22:33:58.211+0800 [DEBUG] provider.terraform-provider-esxi_v1.8.1.exe: pid-5292-esxi_remote_cmds.go:41: [runRemoteSshCommand] Retry connection: 1 2021-05-26T22:33:59.226+0800 [DEBUG] provider.terraform-provider-esxi_v1.8.1.exe: pid-5292-esxi_remote_cmds.go:72: [runRemoteSshCommand] Failed err: Client Connection Error 2021-05-26T22:33:59.226+0800 [INFO] backend/local: plan operation completed ╷ │ Error: Failed to connect to esxi host: Client Connection Error │ │ │ with provider["registry.terraform.io/josenk/esxi"], │ on main.tf line 7, in provider "esxi": │ 7: provider "esxi" { │ ╵ 2021-05-26T22:33:59.243+0800 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/josenk/esxi/1.8.1/windows_amd64/terraform-provider-esxi_v1.8.1.exe pid=5292 2021-05-26T22:33:59.243+0800 [DEBUG] provider: plugin exited

I am running a Windows 2016 server as a VM and trying to connect to ESXI host , i tried the same set up from my Mac and it has the same error. I changed the password for ESXI Zxxxxxxxxxxx (no special characters) I can connect from OVA tools to ESXi server with username and password. C:\Users\Administrator>ovftool.exe vi://192.168.99.200 Accept SSL fingerprint (xxxxxxxxxxxx) for host 192.168.99.200 as source type. Fingerprint will be added to the known host file Write 'yes' or 'no' yes Enter login information for source vi://192.168.99.200/ Username: root Password: **** Error: Found wrong kind of object (ResourcePool). Possible completions are: management WIN-MGMT

Expected behavior A clear and concise description of what you expected to happen.

Terraform files Provide a minimalist main.tf and other terraform files that reproduces your issue. If your problem is related to terraform plan/apply/etc, most likely these files will be REQUIRED.

provider "esxi" { esxi_hostname = var.esxi_hostname esxi_hostport = var.esxi_hostport esxi_hostssl = var.esxi_hostssl esxi_username = var.esxi_username esxi_password = var.esxi_password }

terraform { required_providers { esxi = { source = "josenk/esxi" version = "1.8.1" } } }

resource "esxi_guest" "vmtest01" { guest_name = "vmtest01" # Required, Specify the Guest Name disk_store = "DATA" # Required, Specify an existing Disk Store network_interfaces { virtual_network = "VM Network" # Required for each network interface, Specify the Virtual Network name. } }

############################################################################## variable "esxi_hostname" { default = "esxi7" }

variable "esxi_hostport" { default = "22" }

variable "esxi_hostssl" { default = "443" }

variable "esxi_username" { default = "root" }

variable "esxi_password" { default = "Zxxxxxxxxxxx"

Unspecified will prompt

}

Desktop (please complete the following information):

Additional context

testpshell commented 3 years ago

Checked further based on one of the previous issue reported.

UsePAM yes

PasswordAuthentication yes

I ran the terraform plan with tail -f /var/log/auth.log & tail -f /var/log/auth.log but cant see anything in log files

############################################ [root@esxi7:~] tail -f /var/log/auth.log 2021-05-26T15:06:38Z sshd[1051814]: Disconnected from user root 192.168.99.103 port 59938 2021-05-26T15:06:38Z sshd[1051814]: pam_unix(sshd:session): session closed for user root 2021-05-27T01:06:07Z sshd[1056237]: FIPS mode initialized 2021-05-27T01:06:07Z sshd[1056237]: userauth_pubkey: key type ssh-ed25519 not in PubkeyAcceptedKeyTypes [preauth] 2021-05-27T01:06:12Z sshd[1056237]: Accepted keyboard-interactive/pam for root from 192.168.99.103 port 63487 ssh2 2021-05-27T01:06:12Z sshd[1056237]: pam_unix(sshd:session): session opened for user root by (uid=0) 2021-05-27T01:13:21Z sshd[1051535]: FIPS mode initialized 2021-05-27T01:13:21Z sshd[1051535]: userauth_pubkey: key type ssh-ed25519 not in PubkeyAcceptedKeyTypes [preauth] 2021-05-27T01:13:26Z sshd[1051535]: Accepted keyboard-interactive/pam for root from 192.168.99.103 port 63540 ssh2 2021-05-27T01:13:26Z sshd[1051535]: pam_unix(sshd:session): session opened for user root by (uid=0)

#########################################################

[root@esxi7:~] tail -f /var/log/auth.log 2021-05-26T15:06:38Z sshd[1051814]: Disconnected from user root 192.168.99.103 port 59938 2021-05-26T15:06:38Z sshd[1051814]: pam_unix(sshd:session): session closed for user root 2021-05-27T01:06:07Z sshd[1056237]: FIPS mode initialized 2021-05-27T01:06:07Z sshd[1056237]: userauth_pubkey: key type ssh-ed25519 not in PubkeyAcceptedKeyTypes [preauth] 2021-05-27T01:06:12Z sshd[1056237]: Accepted keyboard-interactive/pam for root from 192.168.99.103 port 63487 ssh2 2021-05-27T01:06:12Z sshd[1056237]: pam_unix(sshd:session): session opened for user root by (uid=0) 2021-05-27T01:13:21Z sshd[1051535]: FIPS mode initialized 2021-05-27T01:13:21Z sshd[1051535]: userauth_pubkey: key type ssh-ed25519 not in PubkeyAcceptedKeyTypes [preauth] 2021-05-27T01:13:26Z sshd[1051535]: Accepted keyboard-interactive/pam for root from 192.168.99.103 port 63540 ssh2 2021-05-27T01:13:26Z sshd[1051535]: pam_unix(sshd:session): session opened for user root by (uid=0)

josenk commented 3 years ago

Make sure you have the correct username, password, and hostname/IP address. Maybe there is a special character in the password that is not handled correctly. Try simplifying the password and try again. It's VERY easy to lock out the user if you enter too many incorrect passwords... You may want to disable that feature temporarily while you figure out this issue.

testpshell commented 3 years ago

Make sure you have the correct username, password, and hostname/IP address. Maybe there is a special character in the password that is not handled correctly. Try simplifying the password and try again. It's VERY easy to lock out the user if you enter too many incorrect passwords... You may want to disable that feature temporarily while you figure out this issue. josenk I have referred one of the previous issue "Connection failure to esxi host #133" and i have changed my password to a simple one with characters and letter (Abcdefghij01). My hostname/IP is all correct. I can ssh into the ESXI server from the VM I am running terraform. My ESXI version is [root@esxi7:~] vmware --version VMware ESXi 7.0.0 build-16324942 [root@esxi7:~] . I am really clueless what is causing this error. I am running the Windows Server 2016 VM in the ESXI host and terraform plan from the VM. I tried to use my mac BigSur Version 11.3.1 but getting the same error.

josenk commented 3 years ago

According to #133 , it was resolved by changing the following setting... Did you try that?

After some additional debugging today I finally found root cause
esxi host had following in sshd_config
PasswordAuthentication no
testpshell commented 3 years ago

According to #133 , it was resolved by changing the following setting... Did you try that?

After some additional debugging today I finally found root cause
esxi host had following in sshd_config
PasswordAuthentication no

josenk Thank you for your valuable time. I tried all the steps mentioned in #133 . Still no luck. I can ssh the ESXI and connect to ESXI with ova tool. I am just wondering it has anything to do with ESXI version 7

UsePAM yes PasswordAuthentication yes

`provider "esxi" { esxi_hostname = var.esxi_hostname esxi_hostport = var.esxi_hostport esxi_hostssl = var.esxi_hostssl esxi_username = var.esxi_username esxi_password = var.esxi_password }

terraform { required_providers { esxi = { source = "josenk/esxi" version = "1.8.1" } } }

resource "esxi_guest" "vmtest01" { guest_name = "vmtest01" # Required, Specify the Guest Name disk_store = "DATA" # Required, Specify an existing Disk Store network_interfaces { virtual_network = "VM Network" # Required for each network interface, Specify the Virtual Network name. } }`

`variable "esxi_hostname" { default = "esxi7" }

variable "esxi_hostport" { default = "22" }

variable "esxi_hostssl" { default = "443" }

variable "esxi_username" { default = "root" }

variable "esxi_password" {

}`

josenk commented 3 years ago

I use ESXi 7 without issues. I don't think that's the problem...

When you did the test using ovf_tool, I noticed you specified the host using the IP address, but in variables.tf, you specify "esxi7". Try using the IP address in variables.tf.

testpshell commented 3 years ago

I use ESXi 7 without issues. I don't think that's the problem...

When you did the test using ovf_tool, I noticed you specified the host using the IP address, but in variables.tf, you specify "esxi7". Try using the IP address in variables.tf.

@josenk It really helped I can create resources from both Windows 2016 server and laptop. Once again thank you very much for the time you have spent to help on this issue. Really appreciate for this provider.

 'Plan: 1 to add, 0 to change, 0 to destroy.
  Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

  esxi_guest.vmtest01: Creating...
  esxi_guest.vmtest01: Still creating... [10s elapsed]
  esxi_guest.vmtest01: Creation complete after 13s [id=4]

   Apply complete! Resources: 1 added, 0 changed, 0 destroyed.`