search

heptio / aws-quickstart

AWS Kubernetes cluster via CloudFormation and kubeadm
Apache License 2.0
223 stars 134 forks source link

Add cloudprovider config for AWS / Fix local dev script #215

Closed stevesloka closed 5 years ago

stevesloka commented 5 years ago

by adding in the controller manager config for AWS cloud provider. Also fixes small nit with local dev script by updating how the S3 bucket gets created.

Signed-off-by: Steve Sloka steves@heptio.com

stevesloka commented 5 years ago

I realized that my ELBs are now getting created but no instances are tied to them, so something else is still incorrect with the configuration.

stevesloka commented 5 years ago

Seeing this error in Controller manager: 


route_controller.go:120] Couldn't reconcile node routes: error listing routes: unable to find route table for AWS cluster: kubernetes```
chuckha commented 5 years ago

It looks like we aren't tagging the route table with the correct tags in the cloud formation template. Should have tag-key=kubernetes.io/cluster/<cluster-id>...am I reading that code right?

https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/aws/aws.go#L744 from https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/aws/tags.go#L251 from https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/aws/aws_routes.go#L44

I'd need to spend more time on it though to make sure it's doing what I think it's doing.

chuckha commented 5 years ago

@stevesloka try tagging your route table with the suggested tag above and rerunning, does that fix it?

stevesloka commented 5 years ago

@chuckha where do I add that? To the cloudformation template?

chuckha commented 5 years ago

oh i was thinking directly in the aws console

vincepri commented 5 years ago

+1 to tag in the cloudformation template, although I'm not sure how this was working before, the code @chuckha linked has been there for a while.

chuckha commented 5 years ago

@stevesloka can you add steps to reproduce so I can try this out when I get a moment?

stevesloka commented 5 years ago

@chuckha all I did was deploy the quickstart, then try and create a service type "LoadBalancer". What I got was pending for the state of it as well as errors in Controller manager stating the cloud provider was not yet defined. So I added that (In this PR) and the ELB gets created, but the instances weren't added and they did not have proper security groups to allow traffic to the nodes.

jhamilton1 commented 5 years ago

@stevesloka @chuckha I just wanted to add that we have a customer experiencing the same error. They have bootstrapped with just kubeadm.

route_controller.go:120] Couldn't reconcile node routes: error listing routes: unable to find route table for AWS cluster: kubernetes

vincepri commented 5 years ago

@stevesloka @chuckha @jhamilton1 @detiber I'll go ahead and merge this, I have another PR ready that I want to base on this that should fix the issue. I'll run CI and sonobuoy and PR against upstream. I'll add more details on the steps I did to debug.