search

kubernetes-sigs / cluster-api-provider-ibmcloud

Cluster API Provider for IBM Cloud
https://cluster-api-ibmcloud.sigs.k8s.io
Apache License 2.0
62 stars 82 forks source link

Cannot create cluster by using kind #184

Closed wanghh2000 closed 5 years ago

wanghh2000 commented 5 years ago

/kind bug /sig ibmcloud

What steps did you take and what happened: Error "unable to create control plane machine: timed out waiting for the condition" happened. Steps: 1) Install go and clusterctl and kind. 2) Get userName and apiKey. 3) Run command

./clusterctl create cluster --bootstrap-type kind --provider ibmcloud -c examples/ibmcloud/out/cluster.yaml -m examples/ibmcloud/out/machines.yaml -p examples/ibmcloud/out/provider-components.yaml

4) Output.

root@gummite1:/gopath/src/sigs.k8s.io/cluster-api-provider-ibmcloud/cmd/clusterctl# ./clusterctl create cluster --bootstrap-type kind --provider ibmcloud -c examples/ibmcloud/out/cluster.yaml -m examples/ibmcloud/out/machines.yaml -p examples/ibmcloud/out/provider-components.yaml
I0526 21:17:34.572924    2962 createbootstrapcluster.go:27] Creating bootstrap cluster
I0526 21:18:06.607929    2962 clusterdeployer.go:78] Applying Cluster API stack to bootstrap cluster
I0526 21:18:06.607999    2962 applyclusterapicomponents.go:26] Applying Cluster API Provider Components
I0526 21:18:07.655187    2962 clusterdeployer.go:83] Provisioning target cluster via bootstrap cluster
I0526 21:18:07.662555    2962 applycluster.go:36] Creating cluster object hhwangtest1 in namespace "default"
I0526 21:18:07.672484    2962 clusterdeployer.go:92] Creating control plane  in namespace "default"
I0526 21:18:07.681477    2962 applymachines.go:36] Creating machines in namespace "default"
I0526 21:48:07.715209    2962 createbootstrapcluster.go:36] Cleaning up bootstrap cluster.
F0526 21:48:09.596723    2962 create_cluster.go:61] unable to create control plane machine: timed out waiting for the condition
root@gummite1:/gopath/src/sigs.k8s.io/cluster-api-provider-ibmcloud/cmd/clusterctl#

What did you expect to happen: Hope create cluster success or how to handle the error.

Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.] Saw some values in yaml file create by generate-yaml.sh, and if it need to be changed.

Environment:

jichenjc commented 5 years ago

21:18->21:48 it's time out for 1/2 hour, so please use -v 10 to open debug and see what happened and please refer to docs/ there are a couple of tips to do trouble shooting

wanghh2000 commented 5 years ago

@jichenjc I just apply softlayer account days ago, and "Current Balance" is 0, Is it the reason? Repeating the following messges.

I0527 01:23:25.261801   26372 clusterclient.go:972] Waiting for Machine ibmcloud-master-zn29c to become ready...
I0527 01:23:25.262156   26372 round_trippers.go:419] curl -k -v -XGET  -H "User-Agent: clusterctl/v0.0.0 (linux/amd64) kubernetes/$Format" -H "Accept: application/json, */*" 'https://localhost:39968/apis/cluster.k8s.io/v1alpha1/namespaces/default/machines/ibmcloud-master-zn29c'
I0527 01:23:25.267267   26372 round_trippers.go:438] GET https://localhost:39968/apis/cluster.k8s.io/v1alpha1/namespaces/default/machines/ibmcloud-master-zn29c 200 OK in 5 milliseconds
I0527 01:23:25.267321   26372 round_trippers.go:444] Response Headers:
I0527 01:23:25.267332   26372 round_trippers.go:447]     Content-Type: application/json
I0527 01:23:25.267341   26372 round_trippers.go:447]     Content-Length: 1045
I0527 01:23:25.267349   26372 round_trippers.go:447]     Date: Mon, 27 May 2019 08:23:25 GMT
I0527 01:23:25.267502   26372 request.go:942] Response Body: {"apiVersion":"cluster.k8s.io/v1alpha1","kind":"Machine","metadata":{"creationTimestamp":"2019-05-27T08:09:45Z","generateName":"ibmcloud-master-","generation":1,"labels":{"cluster.k8s.io/cluster-name":"test1","set":"master"},"name":"ibmcloud-master-zn29c","namespace":"default","resourceVersion":"329","selfLink":"/apis/cluster.k8s.io/v1alpha1/namespaces/default/machines/ibmcloud-master-zn29c","uid":"c9c9f7d2-8056-11e9-88cc-0242ac110003"},"spec":{"metadata":{"creationTimestamp":null},"providerSpec":{"value":{"apiVersion":"ibmcloudproviderconfig/v1alpha1","dataCenter":"\u003cdatacenter name\u003e","domain":"\u003cdomain name\u003e","hourlyBillingFlag":true,"kind":"IbmcloudMachineProviderSpec","localDiskFlag":true,"maxMemory":4096,"osReferenceCode":"\u003cthe operating system selection\u003e","sshKeyName":"cluster-api-provider-ibmcloud","sshUserName":"\u003cssh user name\u003e","startCpus":1,"userDataSecret":{"name":"master-user-data","namespace":"ibmcloud-provider-system"}}},"versions":{"controlPlane":"1.14.0","kubelet":"1.14.0"}}}

I0527 01:39:45.266414   26372 clusterclient.go:972] Waiting for Machine ibmcloud-master-zn29c to become ready...
I0527 01:39:45.266531   26372 round_trippers.go:419] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: clusterctl/v0.0.0 (linux/amd64) kubernetes/$Format" 'https://localhost:39968/apis/cluster.k8s.io/v1alpha1/namespaces/default/machines/ibmcloud-master-zn29c'
I0527 01:39:45.269583   26372 round_trippers.go:438] GET https://localhost:39968/apis/cluster.k8s.io/v1alpha1/namespaces/default/machines/ibmcloud-master-zn29c 200 OK in 3 milliseconds
I0527 01:39:45.269600   26372 round_trippers.go:444] Response Headers:
I0527 01:39:45.269606   26372 round_trippers.go:447]     Content-Length: 1045
I0527 01:39:45.269611   26372 round_trippers.go:447]     Date: Mon, 27 May 2019 08:39:45 GMT
I0527 01:39:45.269615   26372 round_trippers.go:447]     Content-Type: application/json
I0527 01:39:45.269879   26372 request.go:942] Response Body: {"apiVersion":"cluster.k8s.io/v1alpha1","kind":"Machine","metadata":{"creationTimestamp":"2019-05-27T08:09:45Z","generateName":"ibmcloud-master-","generation":1,"labels":{"cluster.k8s.io/cluster-name":"test1","set":"master"},"name":"ibmcloud-master-zn29c","namespace":"default","resourceVersion":"329","selfLink":"/apis/cluster.k8s.io/v1alpha1/namespaces/default/machines/ibmcloud-master-zn29c","uid":"c9c9f7d2-8056-11e9-88cc-0242ac110003"},"spec":{"metadata":{"creationTimestamp":null},"providerSpec":{"value":{"apiVersion":"ibmcloudproviderconfig/v1alpha1","dataCenter":"\u003cdatacenter name\u003e","domain":"\u003cdomain name\u003e","hourlyBillingFlag":true,"kind":"IbmcloudMachineProviderSpec","localDiskFlag":true,"maxMemory":4096,"osReferenceCode":"\u003cthe operating system selection\u003e","sshKeyName":"cluster-api-provider-ibmcloud","sshUserName":"\u003cssh user name\u003e","startCpus":1,"userDataSecret":{"name":"master-user-data","namespace":"ibmcloud-provider-system"}}},"versions":{"controlPlane":"1.14.0","kubelet":"1.14.0"}}}
I0527 01:39:45.270884   26372 createbootstrapcluster.go:36] Cleaning up bootstrap cluster.
I0527 01:39:45.271004   26372 kind.go:69] Running: kind [delete cluster --name=clusterapi]
I0527 01:39:47.069562   26372 kind.go:72] Ran: kind [delete cluster --name=clusterapi] Output: Deleting cluster "clusterapi" ...
F0527 01:39:47.069630   26372 create_cluster.go:61] unable to create control plane machine: timed out waiting for the condition
jichenjc commented 5 years ago

I guess not, the account should be ok, did you see the mahcine created in SL GUI? you should see it's created a VM and I guess you might have a ssh key issue that prevent you from get the kubeconfig file 

root@jichen1:/home/cloudusr/go/src/sigs.k8s.io/cluster-api-provider-ibmcloud/cmd/clusterctl# kubectl --kubeconfig=kubeconfig  get pod --all-namespaces
NAMESPACE                  NAME                                          READY   STATUS             RESTARTS   AGE
ibmcloud-provider-system   ibmcloud-machine-controller-0                 1/1     Running            3          4d

kubectl --kubeconfig=kubeconfig log ibmcloud-machine-controller-0 -n ibmcloud-provider-system

check this log to see more info

gyliu513 commented 5 years ago

@wanghh2000 +1 to @jichenjc , please check troubleshooting guidance here https://github.com/kubernetes-sigs/cluster-api-provider-ibmcloud/blob/master/docs/trouble_shooting.md#get-log-of-clusterapi-controller-containers

wanghh2000 commented 5 years ago

Try to find what's the root cause?

gyliu513 commented 5 years ago

Exactly!

wanghh2000 commented 5 years ago

Incorrect values in cluster.yaml and machines.yaml. Thanks.