search

kubernetes-sigs / karpenter

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
Apache License 2.0
461 stars 158 forks source link

Documenting how to implement another cloud provider #754

Open achetronic opened 2 years ago

achetronic commented 2 years ago

Is an existing page relevant? No pages for anything

What karpenter features are relevant?

How should the docs be improved? Generic documentation about interfaces and what should be implemented in a generic way would help, instead of just giving an implementation for AWS with a lot of specific cases. Done in this way it is impossible to collaborate crafting the providers for other clouds. With this, I mean, with no documentation pages and no documentation comments in the code, how is it supposed to guess what is for Karpenter and what is just for AWS?

Community Note

ellistarn commented 2 years ago

Hi @achetronic, this isn't something we've prioritized yet. Implementing another cloud provider is a pretty big undertaking. While Karpenter is designed to support this (see: https://github.com/aws/karpenter/blob/main/pkg/cloudprovider/types.go), I expect there will be some changes as we work towards support for another provider.

Off the top of my head:

  1. changes to helm release processes and getting started guides
  2. separation of the docs site into a separate repository, and a multi-vendor docs strategy
  3. separation of the aws cloud provider code (/pkg/cloudprovider/aws) into a separate repository from karpenter core

It sounds like you might be interested in contributing support for another cloud provider. If you'd like to chat about this further, I recommend joining us at https://github.com/aws/karpenter/blob/main/WORKING_GROUP.md, or reaching out on slack.

achetronic commented 2 years ago

Hi @achetronic, this isn't something we've prioritized yet. Implementing another cloud provider is a pretty big undertaking. While Karpenter is designed to support this (see: https://github.com/aws/karpenter/blob/main/pkg/cloudprovider/types.go), I expect there will be some changes as we work towards support for another provider.

Off the top of my head:

1. changes to helm release processes and getting started guides

2. separation of the docs site into a separate repository, and a multi-vendor docs strategy

3. separation of the aws cloud provider code (/pkg/cloudprovider/aws) into a separate repository from karpenter core

It sounds like you might be interested in contributing support for another cloud provider. If you'd like to chat about this further, I recommend joining us at https://github.com/aws/karpenter/blob/main/WORKING_GROUP.md, or reaching out on slack.

Hello @ellistarn, I have seen that interfaces but there are some specific things for AWS and I should have to implement them as a null or a dirty other thing because they are inside the generic interface (about AWS Neuron and so)

I could implement the cloudprovider for Gcloud and for DO, but I would need to deattach the AWS related things to take the classes and only implement them for gcloud.

Would you consider to add a section into the API spec to reference a secret on a namespace to get, for example, the credentials for other providers?

ellistarn commented 2 years ago

Agreed that we need to factor out AWSNeuron and NvidiaGPU into a generic instancetype.Resources(). We've talked about it, and it should be relatively straightforward -- I think @bwagner5 might've made some progress on it.

In the short term, you can make progress on this by returning 0 for resource types not supported by your cloud provider.

For the API, each cloud provider implements the Provider runtime.RawExtensions (https://github.com/aws/karpenter/blob/3a9fa808b6eb2a42e17603135838ec5a75817399/pkg/apis/provisioning/v1alpha5/constraints.go#L42). This struct is just raw bytes that gets serialized according to the provider that the binary was built with. For example, AWS's provider API is here: https://github.com/aws/karpenter/blob/3a9fa808b6eb2a42e17603135838ec5a75817399/pkg/cloudprovider/aws/apis/v1alpha1/provider.go#L35.

ellistarn commented 2 years ago

I'd love to chat about all of this on slack if you are able to.

achetronic commented 2 years ago

I'd love to chat about all of this on slack if you are able to.

Hello there! Of course, I would like to talk about details on Slack to implement, at least, the basic support for another provider. Are you available on the Kubernetes Slack?

ellistarn commented 2 years ago

You can find me here: https://kubernetes.slack.com/archives/C02SFFZSA2K

github-actions[bot] commented 2 years ago

Labeled for closure due to inactivity in 10 days.

till commented 1 year ago

Hey 👋. I am interested in this as well. Did anything materialize from the conversation on Slack?

ellistarn commented 8 months ago

@tallaxes, would you be interested in helping out on this one?

tallaxes commented 8 months ago

@tallaxes, would you be interested in helping out on this one?

Yes, absolutely - assuming you are referring to ironing out (and maybe ultimately documenting) what it takes to implement a provider. I may not have a ton of bandwidth for this, but would be happy to contribute, provide feedback, point out (and maybe help pin down) expectations or requirements that are not clear, etc. (This would also be a good opportunity to revisit/sharpen some of the assumptions and decisions in Azure provider ...)

ellistarn commented 7 months ago

I love to review an azure provider design so we can make sure everything is lining up nicely.

k8s-triage-robot commented 4 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 3 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 2 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 2 months ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-sigs/karpenter/issues/754#issuecomment-2128853973): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
jonathan-innis commented 2 months ago

/reopen

k8s-ci-robot commented 2 months ago

@jonathan-innis: Reopened this issue.

In response to [this](https://github.com/kubernetes-sigs/karpenter/issues/754#issuecomment-2129877401): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
jonathan-innis commented 2 months ago

/remove-lifecycle rotten

jonathan-innis commented 2 months ago

/triage accepted