search

coreos / tectonic-forum

Apache License 2.0
30 stars 9 forks source link

Tectonic console fail to start: invalid character 'U' looking for beginning of value #200

Open sguyennet opened 7 years ago

sguyennet commented 7 years ago

Issue Report Template

Tectonic Version

1.73

Environment

VMware vSphere 6.5

Expected Behavior

tectonic-console pods in Running state

Actual Behavior

tectonic-console pods in CrashLoopBackOff state

Reproduction Steps

I followed the documentation https://coreos.com/tectonic/docs/latest/install/vmware/vmware-terraform.html

  1. Deploy Tectonic with Terraform
  2. kubectl get pods -n tectonic-system ... tectonic-console-1046407245-3702f 0/1 CrashLoopBackOff 367 20h tectonic-console-1046407245-4v52f 0/1 CrashLoopBackOff 367 20h ...

Other Information

$ kubectl -n tectonic-system logs tectonic-console-1046407245-3702f 2017/09/27 18:53:18 http: Provider config sync failed, retrying in 1s: invalid character 'U' looking for beginning of value 2017/09/27 18:53:19 http: Provider config sync still failing, retrying in 2s: invalid character 'U' looking for beginning of value 2017/09/27 18:53:21 http: Provider config sync still failing, retrying in 4s: invalid character 'U' looking for beginning of value 2017/09/27 18:53:25 http: Provider config sync still failing, retrying in 8s: invalid character 'U' looking for beginning of value 2017/09/27 18:53:33 http: Provider config sync still failing, retrying in 16s: invalid character 'U' looking for beginning of value 2017/09/27 18:53:49 http: Provider config sync still failing, retrying in 32s: invalid character 'U' looking for beginning of value

Here is my terraform.tfvars:

// The e-mail address used to: // 1. login as the admin user to the Tectonic Console. // 2. generate DNS zones for some providers. // // Note: This field MUST be set manually prior to creating the cluster. tectonic_admin_email = "xxx"

// The bcrypt hash of admin user password to login to the Tectonic Console. // Use the bcrypt-hash tool (https://github.com/coreos/bcrypt-tool/releases/tag/v1.0.0) to generate it. // // Note: This field MUST be set manually prior to creating the cluster. tectonic_admin_password_hash = "xxx"

// The base DNS domain of the cluster. It must NOT contain a trailing period. Some // DNS providers will automatically add this if necessary. // // Example: openstack.dev.coreos.systems. // // Note: This field MUST be set manually prior to creating the cluster. // This applies only to cloud platforms. // // [Azure-specific NOTE] // To use Azure-provided DNS, tectonic_base_domain should be set to "" // If using DNS records, ensure that tectonic_base_domain is set to a properly configured external DNS zone. // Instructions for configuring delegated domains for Azure DNS can be found here: https://docs.microsoft.com/en-us/azure/dns/dns-delegate-domain-azure-dns tectonic_base_domain = "mydomain.local"

// (optional) The content of the PEM-encoded CA certificate, used to generate Tectonic Console's server certificate. // If left blank, a CA certificate will be automatically generated. // tectonic_ca_cert = ""

// (optional) The content of the PEM-encoded CA key, used to generate Tectonic Console's server certificate. // This field is mandatory if tectonic_ca_cert is set. // tectonic_ca_key = ""

// (optional) The algorithm used to generate tectonic_ca_key. // The default value is currently recommended. // This field is mandatory if tectonic_ca_cert is set. // tectonic_ca_key_alg = "RSA"

// [ALPHA] If set to true, calico network policy support will be deployed. // WARNING: Enabling an alpha feature means that future updates may become unsupported. // This should only be enabled on clusters that are meant to be short-lived to begin validating the alpha feature. tectonic_calico_network_policy = false

// The Container Linux update channel. // // Examples: stable, beta, alpha tectonic_cl_channel = "stable"

// This declares the IP range to assign Kubernetes pod IPs in CIDR notation. tectonic_cluster_cidr = "10.2.0.0/16"

// The name of the cluster. // If used in a cloud-environment, this will be prepended to tectonic_base_domain resulting in the URL to the Tectonic console. // // Note: This field MUST be set manually prior to creating the cluster. // Warning: Special characters in the name like '.' may cause errors on OpenStack platforms due to resource name constraints. tectonic_cluster_name = "tectonic"

// (optional) This only applies if you use the modules/dns/ddns module. // // Specifies the RFC2136 Dynamic DNS server key algorithm. // tectonic_ddns_key_algorithm = ""

// (optional) This only applies if you use the modules/dns/ddns module. // // Specifies the RFC2136 Dynamic DNS server key name. // tectonic_ddns_key_name = ""

// (optional) This only applies if you use the modules/dns/ddns module. // // Specifies the RFC2136 Dynamic DNS server key secret. // tectonic_ddns_key_secret = ""

// (optional) This only applies if you use the modules/dns/ddns module. // // Specifies the RFC2136 Dynamic DNS server IP/host to register IP addresses to. // tectonic_ddns_server = ""

// (optional) The path of the file containing the CA certificate for TLS communication with etcd. // // Note: This works only when used in conjunction with an external etcd cluster. // If set, the variables tectonic_etcd_servers, tectonic_etcd_client_cert_path, and tectonic_etcd_client_key_path must also be set. // tectonic_etcd_ca_cert_path = "/dev/null"

// (optional) The path of the file containing the client certificate for TLS communication with etcd. // // Note: This works only when used in conjunction with an external etcd cluster. // If set, the variables tectonic_etcd_servers, tectonic_etcd_ca_cert_path, and tectonic_etcd_client_key_path must also be set. // tectonic_etcd_client_cert_path = "/dev/null"

// (optional) The path of the file containing the client key for TLS communication with etcd. // // Note: This works only when used in conjunction with an external etcd cluster. // If set, the variables tectonic_etcd_servers, tectonic_etcd_ca_cert_path, and tectonic_etcd_client_cert_path must also be set. // tectonic_etcd_client_key_path = "/dev/null"

// The number of etcd nodes to be created. // If set to zero, the count of etcd nodes will be determined automatically. // // Note: This is not supported on bare metal. tectonic_etcd_count = "3"

// (optional) List of external etcd v3 servers to connect with (hostnames/IPs only). // Needs to be set if using an external etcd cluster. // // Example: ["etcd1", "etcd2", "etcd3"] // tectonic_etcd_servers = ""

// (optional) If set to true, TLS secure communication for self-provisioned etcd. will be used. // // Note: If tectonic_experimental is set to true this variable has no effect, because the experimental self-hosted etcd always uses TLS. tectonic_etcd_tls_enabled = true

// If set to true, experimental Tectonic assets are being deployed. tectonic_experimental = false

// The path to the tectonic licence file. // You can download the Tectonic license file from your Account overview page at [1]. // // [1] https://account.coreos.com/overview // // Note: This field MUST be set manually prior to creating the cluster unless tectonic_vanilla_k8s is set to true. tectonic_license_path = "/home/sguyennet/tectonic-license.txt"

// The number of master nodes to be created. // This applies only to cloud platforms. tectonic_master_count = "2"

// The path the pull secret file in JSON format. // This is known to be a "Docker pull secret" as produced by the docker login [1] command. // A sample JSON content is shown in [2]. // You can download the pull secret from your Account overview page at [3]. // // [1] https://docs.docker.com/engine/reference/commandline/login/ // // [2] https://coreos.com/os/docs/latest/registry-authentication.html#manual-registry-auth-setup // // [3] https://account.coreos.com/overview // // Note: This field MUST be set manually prior to creating the cluster unless tectonic_vanilla_k8s is set to true. tectonic_pull_secret_path = "/home/sguyennet/config.json"

// This declares the IP range to assign Kubernetes service cluster IPs in CIDR notation. The maximum size of this IP range is /12 tectonic_service_cidr = "10.3.0.0/16"

// The Tectonic statistics collection URL to which to report. tectonic_stats_url = "https://stats-collector.tectonic.com"

// If set to true, a vanilla Kubernetes cluster will be deployed, omitting any Tectonic assets. tectonic_vanilla_k8s = false

// vCenter Cluster used to create VMs under tectonic_vmware_cluster = "Compute-01"

// The domain name which resolves to controller node(s) tectonic_vmware_controller_domain = "tectonic-k8s.mydomain.local"

// Virtual DataCenter to deploy VMs tectonic_vmware_datacenter = "lab"

// The storage LUN used by etcd nodes. In order to use vSphere Datastore Cluster use the syntax DatastoreClusterName/datastore. tectonic_vmware_etcd_datastore = "Datastore-02"

// Default Gateway IP address for etcd nodes(s) tectonic_vmware_etcd_gateway = "10.10.20.1"

// Terraform map of etcd node(s) Hostnames, Example: // tectonic_vmware_etcd_hostnames = { // "0" = "mycluster-etcd-0" // "1" = "mycluster-etcd-1" // "2" = "mycluster-etcd-2" // } tectonic_vmware_etcd_hostnames = { "0" = "tectonic-etcd-0" "1" = "tectonic-etcd-1" "2" = "tectonic-etcd-2" }

// Terraform map of etcd node(s) IP Addresses, Example: // tectonic_vmware_etcd_ip = { // "0" = "192.168.246.10/24" // "1" = "192.168.246.11/24" // "2" = "192.168.246.12/24" // } tectonic_vmware_etcd_ip = { "0" = "10.10.20.210/24" "1" = "10.10.20.211/24" "2" = "10.10.20.212/24" }

// etcd node(s) VM Memory Size in MB tectonic_vmware_etcd_memory = "2048"

// etcd node(s) VM vCPU count tectonic_vmware_etcd_vcpu = "1"

// vSphere Folder to create and add the Tectonic nodes tectonic_vmware_folder = "tectonic"

// The domain name which resolves to Tectonic Ingress (i.e. worker node(s)) tectonic_vmware_ingress_domain = "tectonic-k8s.mydomain.local"

// The storage LUN used by master nodes. In order to use vSphere Datastore Cluster use the syntax DatastoreClusterName/datastore. tectonic_vmware_master_datastore = "Datastore-02"

// Default Gateway IP address for Master nodes(s) tectonic_vmware_master_gateway = "10.10.20.1"

// Terraform map of Master node(s) Hostnames, Example: // tectonic_vmware_master_hostnames = { // "0" = "mycluster-master-0" // "1" = "mycluster-master-1" // } tectonic_vmware_master_hostnames = { "0" = "tectonic-master-0" "1" = "tectonic-master-1" }

// Terraform map of Master node(s) IP Addresses, Example: // tectonic_vmware_master_ip = { // "0" = "192.168.246.20/24" // "1" = "192.168.246.21/24" // } tectonic_vmware_master_ip = { "0" = "10.10.20.220/24" "1" = "10.10.20.221/24" }

// Master node(s) Memory Size in MB tectonic_vmware_master_memory = "2048"

// Master node(s) vCPU count tectonic_vmware_master_vcpu = "1"

// Portgroup to attach the cluster nodes tectonic_vmware_network = "VM Network"

// DNS Server to be used by Virtual Machine(s). Multiple DNS servers can be separated by whitespace. Example: "192.168.1.1 192.168.2.1" tectonic_vmware_node_dns = "10.10.20.1"

// vCenter Server IP/FQDN tectonic_vmware_server = "vcsa.mydomain.local"

// SSH public key to use as an authorized key. Example: "ssh-rsa AAAB3N..." tectonic_vmware_ssh_authorized_key = "ssh-rsa xxx"

// SSH private key file in .pem format corresponding to tectonic_vmware_ssh_authorized_key. If not provided, SSH agent will be used. tectonic_vmware_ssh_private_key_path = ""

// Is the vCenter certificate Self-Signed? Example: tectonic_vmware_sslselfsigned = "true" tectonic_vmware_sslselfsigned = "true"

// Virtual Machine template of CoreOS Container Linux. tectonic_vmware_vm_template = "CoreOS-Container-Linux-1465.6.0"

// Folder for VM template of CoreOS Container Linux. tectonic_vmware_vm_template_folder = "tectonic"

// The storage LUN used by worker nodes. In order to use vSphere Datastore Cluster use the syntax DatastoreClusterName/datastore. tectonic_vmware_worker_datastore = "Datastore-02"

// Default Gateway IP address for Master nodes(s) tectonic_vmware_worker_gateway = "10.10.20.1"

// Terraform map of Worker node(s) Hostnames, Example: // tectonic_vmware_worker_hostnames = { // "0" = "mycluster-worker-0" // "1" = "mycluster-worker-1" // } tectonic_vmware_worker_hostnames = { "0" = "tectonic-worker-0" "1" = "tectonic-worker-1" }

// Terraform map of Worker node(s) IP Addresses, Example: // tectonic_vmware_worker_ip = { // "0" = "192.168.246.30/24" // "1" = "192.168.246.31/24" // } tectonic_vmware_worker_ip = { "0" = "10.10.20.230/24" "1" = "10.10.20.231/24" }

// Worker node(s) Memory Size in MB tectonic_vmware_worker_memory = "2048"

// Worker node(s) vCPU count tectonic_vmware_worker_vcpu = "1"

// The number of worker nodes to be created. // This applies only to cloud platforms. tectonic_worker_count = "2"

lazypower commented 7 years ago

Just stabbing in the dark here, but also inspect the logs coming from both the identity manager and the apiserver.

It feels like something got a weird value and the console is trying to parse an error message when its expecting a JSON response.

pilsy commented 7 years ago

The "U" is coming from a 401 (http status code) "Unauthorised" (actual text output) response, this will be something to do with the certificates (blah is valid for bleh not blah)