hashicorp / terraform-provider-ignition

Terraform Ignition provider
https://www.terraform.io/docs/providers/ignition/
Mozilla Public License 2.0
39 stars 64 forks source link

Openshift 4.1 UPI installation failed on VMware (terraform version - v0.11.14) #65

Open ghost opened 4 years ago

ghost commented 4 years ago

This issue was originally opened by @arunabhabanerjee as hashicorp/terraform#23640. It was migrated here as a result of the provider split. The original body of the issue is below.


Terraform Version

Terraform v0.11.14

-->

...

Terraform Configuration Files

``// ID identifying the cluster to create. Use your username so that resources created can be tracked back to you. cluster_id = "test"

// Domain of the cluster. This should be "${cluster_id}.${base_domain}". cluster_domain = "test.lab.local"

// Base domain from which the cluster domain is a subdomain. base_domain = "lab.loacal"

// Name of the vSphere server. The dev cluster is on "vcsa.vmware.devcluster.openshift.com". vsphere_server = "10.55.138.36"

// User on the vSphere server. vsphere_user = "administrator@vsphere.local"

// Password of the user on the vSphere server. vsphere_password = "password"

// Name of the vSphere cluster. The dev cluster is "devel". vsphere_cluster = "dev"

// Name of the vSphere data center. The dev cluster is "dc1". vsphere_datacenter = "BTM"

// Name of the vSphere data store to use for the VMs. The dev cluster uses "nvme-ds1". vsphere_datastore = "datastore2"

// Name of the VM template to clone to create VMs for the cluster. The dev cluster has a template named "rhcos-latest". vm_template = "rhcos-latest"

// The machine_cidr where IP addresses will be assigned for cluster nodes. // Additionally, IPAM will assign IPs based on the network ID. machine_cidr = "10.55.24.0/21"

// The number of control plane VMs to create. Default is 3. control_plane_count = 3

// The number of compute VMs to create. Default is 3. compute_count = 3

// URL of the bootstrap ignition. This needs to be publicly accessible so that the bootstrap machine can pull the ignition. bootstrap_ignition_url = "http://10.55.1.5:8080/ignition/bootstrap.ign"

// Ignition config for the control plane machines. You should copy the contents of the master.ign generated by the installer. control_plane_ignition = <<END_OF_MASTER_IGNITION {"ignition":{"config":{"append":[{"source":"https://api-int.test.lab.local:22623/config/master","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"data:text/plain;charset=utf-8;base64,LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFRENDQWZpZ0F3SUJBZ0lJUU5BQXRDbmhUVkl3RFFZSktvWklodmNOQVFFTEJRQXdKakVTTUJBR0ExVUUKQ3hNSmIzQmxibk5vYVdaME1SQXdEZ1lEVlFRREV3ZHliMjkwTFdOaE1CNFhEVEU1TVRJd016RTBNVFF6T1ZvWApEVEk1TVRFek1ERTBNVFF6T1Zvd0pqRVNNQkFHQTFVRUN4TUpiM0JsYm5Ob2FXWjBNUkF3RGdZRFZRUURFd2R5CmIyOTBMV05oTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF1Q3FDSy8xYUtOOTAKQUFTamxQTTZxaFY2SHlPWGtKaFZKYm1YQzErbUthMG9CbnNiUGN4WE14NkFROXhSMSt0djJveFl1Q0lBRlNNUwplb3pIa2cyS2VhU25mQTJYdHVkRGZnaklTNUdsQ3lPRTJ4ZXZrZ1dzU0VVQXV3TmlTTXRRWS9BTWt1YnMyK3VZCmRVQUxUMHhsZWkyUWhoaFN6OXN3allNNitnaUh5K2JZMDZjZmxkbjF0NG5aRGZFeUlENFNaR25WeDBmVzNNZ3QKdVRPU200Q1E0WWpUZDlQd1pKL0RUckpkNG9sVjZHQlNoSXFBSHh0elBiaWZwMjZaTnY5MjRrN2trMFhKUDI5Lwp3MEErWDRPVG1tdnY0RVQ2SVJISDZlMFp6TW9BUkpReVF1ekJMUFhqZ0VGMFBtRDAvS2ZBSFlEc25Rdklza3JICldJa3A3enc5a3dJREFRQUJvMEl3UURBT0JnTlZIUThCQWY4RUJBTUNBcVF3RHdZRFZSMFRBUUgvQkFVd0F3RUIKL3pBZEJnTlZIUTRFRmdRVTI4TEhRTUlMZkNhREVBUGhhcWg0MmpVZjZvUXdEUVlKS29aSWh2Y05BUUVMQlFBRApnZ0VCQUFuQ2dCME9saTl4cmszZkVUK2hCOE9hQWtuRncra2NGVHdGMDNaYS9HeDkydW8wcjJZNlI2bDB2Q1NuCml2VHhoRzBMeDhaTlNYMURlSkFRbDY2YjkrTHF2QVhqdDNqUFpnT1VzRnp5NVBYUnl4emRFMVNPNGRGQ2dZdUcKUHZTYTNObkJaU3JlV0N5Wkp2SWt5S3VjajZRZWhiN0ROUmlYSjJ4WkJSajUya2NGQ29LN1JQODlkNGI4ZlRkNQpjV0dhVDJYcDBsbmJyb0dRWUp2dklmVXZ1N21kdzJ4dUVxMGFFUTFkK00xUnB6eWVxbVRJNTBUcHcxMnBjbnVYCkJnNzI2NVZ3SUxhdVhTWWRxbklnTHJjZU1SejUxbnpaSFdMMHRBMjJMdWRJNE9uNkt0NlJLaXNTQlRsT0ZlYTAKTTQyOFBsVWkrTm9ENm0zUnpmVDhreXFTWS9rPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}} END_OF_MASTER_IGNITION

// Ignition config for the compute machines. You should copy the contents of the worker.ign generated by the installer. compute_ignition = <<END_OF_WORKER_IGNITION {"ignition":{"config":{"append":[{"source":"https://api-int.test.lab.local:22623/config/worker","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"data:text/plain;charset=utf-8;base64,LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFRENDQWZpZ0F3SUJBZ0lJUU5BQXRDbmhUVkl3RFFZSktvWklodmNOQVFFTEJRQXdKakVTTUJBR0ExVUUKQ3hNSmIzQmxibk5vYVdaME1SQXdEZ1lEVlFRREV3ZHliMjkwTFdOaE1CNFhEVEU1TVRJd016RTBNVFF6T1ZvWApEVEk1TVRFek1ERTBNVFF6T1Zvd0pqRVNNQkFHQTFVRUN4TUpiM0JsYm5Ob2FXWjBNUkF3RGdZRFZRUURFd2R5CmIyOTBMV05oTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF1Q3FDSy8xYUtOOTAKQUFTamxQTTZxaFY2SHlPWGtKaFZKYm1YQzErbUthMG9CbnNiUGN4WE14NkFROXhSMSt0djJveFl1Q0lBRlNNUwplb3pIa2cyS2VhU25mQTJYdHVkRGZnaklTNUdsQ3lPRTJ4ZXZrZ1dzU0VVQXV3TmlTTXRRWS9BTWt1YnMyK3VZCmRVQUxUMHhsZWkyUWhoaFN6OXN3allNNitnaUh5K2JZMDZjZmxkbjF0NG5aRGZFeUlENFNaR25WeDBmVzNNZ3QKdVRPU200Q1E0WWpUZDlQd1pKL0RUckpkNG9sVjZHQlNoSXFBSHh0elBiaWZwMjZaTnY5MjRrN2trMFhKUDI5Lwp3MEErWDRPVG1tdnY0RVQ2SVJISDZlMFp6TW9BUkpReVF1ekJMUFhqZ0VGMFBtRDAvS2ZBSFlEc25Rdklza3JICldJa3A3enc5a3dJREFRQUJvMEl3UURBT0JnTlZIUThCQWY4RUJBTUNBcVF3RHdZRFZSMFRBUUgvQkFVd0F3RUIKL3pBZEJnTlZIUTRFRmdRVTI4TEhRTUlMZkNhREVBUGhhcWg0MmpVZjZvUXdEUVlKS29aSWh2Y05BUUVMQlFBRApnZ0VCQUFuQ2dCME9saTl4cmszZkVUK2hCOE9hQWtuRncra2NGVHdGMDNaYS9HeDkydW8wcjJZNlI2bDB2Q1NuCml2VHhoRzBMeDhaTlNYMURlSkFRbDY2YjkrTHF2QVhqdDNqUFpnT1VzRnp5NVBYUnl4emRFMVNPNGRGQ2dZdUcKUHZTYTNObkJaU3JlV0N5Wkp2SWt5S3VjajZRZWhiN0ROUmlYSjJ4WkJSajUya2NGQ29LN1JQODlkNGI4ZlRkNQpjV0dhVDJYcDBsbmJyb0dRWUp2dklmVXZ1N21kdzJ4dUVxMGFFUTFkK00xUnB6eWVxbVRJNTBUcHcxMnBjbnVYCkJnNzI2NVZ3SUxhdVhTWWRxbklnTHJjZU1SejUxbnpaSFdMMHRBMjJMdWRJNE9uNkt0NlJLaXNTQlRsT0ZlYTAKTTQyOFBsVWkrTm9ENm0zUnpmVDhreXFTWS9rPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}} END_OF_WORKER_IGNITION

// Set ipam and ipam_token if you want to use the IPAM server to reserve IP // addresses for the VMs.

// Address or hostname of the IPAM server from which to reserve IP addresses for the cluster machines. // ipam = "139.178.89.254"

// Token to use to authenticate with the IPAM server. // ipam_token = "TOKEN_FOR_THE_IPAM_SERVER"

// Set bootstrap_ip, control_plane_ip, and compute_ip if you want to use static // IPs reserved someone else, rather than the IPAM server.

// The IP address to assign to the bootstrap VM. bootstrap_ip = "10.55.1.10"

// The IP addresses to assign to the control plane VMs. The length of this list // must match the value of control_plane_count. control_plane_ips = ["10.55.1.11", "10.55.1.12", "10.55.1.13"]

// The IP addresses to assign to the compute VMs. The length of this list must // match the value of compute_count. compute_ips = ["10.55.1.14", "10.55.1.15", "10.55.1.16"] root:/home/arunabha/openshift/installer/upi/vsphere> `hcl ...



### Debug Output
<!--
Full debug output can be obtained by running Terraform with the environment variable `TF_LOG=trace`. Please create a GitHub Gist containing the debug output. Please do _not_ paste the debug output in the issue, since debug output is long.

Debug output may contain sensitive information. Please review it before posting publicly, and if you are concerned feel free to encrypt the files using the HashiCorp security public key.
-->

### Crash Output
<!--
# terraform apply -auto-approve
data.ignition_systemd_unit.restart[1]: Refreshing state...
data.ignition_systemd_unit.restart[2]: Refreshing state...
data.ignition_systemd_unit.restart: Refreshing state...
data.ignition_file.hostname: Refreshing state...
data.ignition_systemd_unit.restart[0]: Refreshing state...
data.ignition_file.hostname[2]: Refreshing state...
data.ignition_systemd_unit.restart[0]: Refreshing state...
data.ignition_file.hostname[2]: Refreshing state...
data.ignition_file.hostname[1]: Refreshing state...
data.ignition_file.hostname[1]: Refreshing state...
data.ignition_file.hostname[0]: Refreshing state...
data.ignition_systemd_unit.restart[2]: Refreshing state...
data.ignition_file.hostname[0]: Refreshing state...
data.ignition_systemd_unit.restart[1]: Refreshing state...
data.ignition_file.static_ip[1]: Refreshing state...
data.ignition_file.static_ip[2]: Refreshing state...
data.ignition_file.static_ip[0]: Refreshing state...
data.ignition_file.static_ip[2]: Refreshing state...
data.ignition_file.static_ip: Refreshing state...
data.ignition_file.static_ip[0]: Refreshing state...
data.ignition_config.ign[1]: Refreshing state...
data.ignition_config.ign[2]: Refreshing state...
data.ignition_file.static_ip[1]: Refreshing state...
data.ignition_config.ign[0]: Refreshing state...
data.ignition_config.ign: Refreshing state...
data.ignition_config.ign[2]: Refreshing state...
data.ignition_config.ign[0]: Refreshing state...
data.ignition_config.ign[1]: Refreshing state...
data.vsphere_datacenter.dc: Refreshing state...
data.vsphere_datastore.datastore: Refreshing state...
data.vsphere_virtual_machine.template: Refreshing state...
data.vsphere_network.network: Refreshing state...
data.vsphere_network.network: Refreshing state...
data.vsphere_compute_cluster.compute_cluster: Refreshing state...
data.vsphere_virtual_machine.template: Refreshing state...
data.vsphere_network.network: Refreshing state...
data.vsphere_datastore.datastore: Refreshing state...
data.vsphere_virtual_machine.template: Refreshing state...
data.vsphere_datastore.datastore: Refreshing state...

Error: Error refreshing state: 3 errors occurred:
        * module.control_plane.data.ignition_config.ign: 3 errors occurred:
        * module.control_plane.data.ignition_config.ign[0]: data.ignition_config.ign.0: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'c' after top-level value
        * module.control_plane.data.ignition_config.ign[1]: data.ignition_config.ign.1: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'b' in literal false (expecting 'a')
        * module.control_plane.data.ignition_config.ign[2]: data.ignition_config.ign.2: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'b' after top-level value

        * module.bootstrap.data.ignition_config.ign: 1 error occurred:
        * module.bootstrap.data.ignition_config.ign: data.ignition_config.ign: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'b' after top-level value

        * module.compute.data.ignition_config.ign: 3 errors occurred:
        * module.compute.data.ignition_config.ign[0]: data.ignition_config.ign.0: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'c' after top-level value
        * module.compute.data.ignition_config.ign[1]: data.ignition_config.ign.1: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'e' after top-level value
        * module.compute.data.ignition_config.ign[2]: data.ignition_config.ign.2: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'c' looking for beginning of value

-->

### Expected Behavior
<!--
terraform should able to deploy VM
-->

### Actual Behavior
<!--
terraform is unable to deply VM
-->

### Steps to Reproduce
<!--
Please list the full steps required to reproduce the issue, for example:
1. `terraform init`
2. `terraform apply -auto-approve`
-->

### Additional Context
<!--
Are there anything atypical about your situation that we should know? For example: is Terraform running in a wrapper script or in a CI system? Are you passing any unusual command line options or environment variables to opt-in to non-default behavior?
-->

### References
<!--
Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

- hashicorp/terraform#6017

-->
aaronpi commented 4 years ago

Seeing the same issue here, with ignition provider 1.2.1 and terraform 0.11.4

  • module.jenkins.data.ignition_config.leader: data.ignition_config.leader: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'a' looking for beginning of value

Rolling back/forcing ignition to use 1.1.0, as mentioned in the previous issue for this, solves for now, but that's not a permanent solution

LorbusChris commented 4 years ago

the error message does deliver the fix though: You need to adapt your tf files to use .rendered instead of .id to reference things in data.ignition...

frenchtoasters commented 4 years ago

I am hitting this same issue with the 1.2.1 Ignition provider and terraform 0.12.12, however as @lorbuschris mentioned I am already using the .rendered to access the data. My first thought would be that this has something to do with the how the provider handles ignitions that are rendered and then referenced using count.index.

I am currently referenced the rendered templates in the following manner that is causing this error:

"guestinfo.coreos.config.data" = base64encode(data.ignition_config.aio_ign.*.rendered[count.index])

I have also tested referencing it this way and still getting the same error:

"guestinfo.coreos.config.data" = base64encode(data.ignition_config.aio_ign[count.index].rendered)