search

kubernetes / cloud-provider-aws

Cloud provider for AWS
https://cloud-provider-aws.sigs.k8s.io/
Apache License 2.0
381 stars 300 forks source link

cloud-provider-aws does not Prefer CLI Arguments for Configuring Kubernetes #840

Closed aauren closed 1 month ago

aauren commented 6 months ago

What happened:

I attempted to pass --master and --kubeconfig to cloud-provider-aws and didn't see any change to the output of the provider. This resulted in cloud-provider-aws not being configured correctly in my cluster.

This resulted in the following error:

F0128 01:25:22.339300      26 main.go:84] unable to execute command: unable to load configmap based request-header-client-ca-file: Get "https://10.96.0.1:443/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication": dial tcp 10.96.0.1:443: i/o timeout

In the above, 10.96.0.1 is what the kube-apiserver would be reachable at, if my CNI was up and proxying requests to it's ClusterIP.

What you expected to happen:

I expected it to adhere to the CLI arguments that I passed.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

In my case, I found that this was caused by aws-cloud-controller-manager preferring the environment variables that Kubernetes sets:

instead of the CLI parameters passed.

This will be surprising for most users as other CLI tooling from Kubernetes like kubectl prefer the CLI arguments over environment variables.

It also makes it incredibly difficult to bootstrap a new cluster where the CNI is dependent on information from the cloud provider plugin in order to function correctly. As it will prefer the ClusterIP of the Kubernetes API server rather than the configuration that the user is attempting to override it with and result in a race condition.

Environment:

/kind bug

k8s-ci-robot commented 6 months ago

This issue is currently awaiting triage.

If cloud-provider-aws contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
kmala commented 6 months ago

aws cloud provider uses the upstream kubernetes cloud library and the error is coming from the upstream code. It is using a different flag to override the kube config . Can you adding additional flag authentication-kubeconfig and verify if it works. We can create an issue upstream and see how they would like to proceed.

aauren commented 6 months ago

Adding --authentication-kubeconfig seemed to make the configuration work as well.

Can I ask why cloud-provider-aws has two separate flags for this? --kubeconfig is the normal flag so I wasn't expecting to find another flag for passing the kubeconfig again.

k8s-triage-robot commented 3 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 2 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 1 month ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 1 month ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes/cloud-provider-aws/issues/840#issuecomment-2198262256): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.