search

backstage / backstage

Backstage is an open framework for building developer portals
https://backstage.io/
Apache License 2.0
26.89k stars 5.58k forks source link

[RFC] Future kubernetes plugin contributions #2857

Closed mclarke47 closed 1 year ago

mclarke47 commented 3 years ago

RFC future kubernetes plugin contributions

Now that https://github.com/spotify/backstage/issues/2461 is complete we can consider how to proceed with future Kubernetes contributions!

I have itemized a bunch of ideas below. They are just suggestions I would encourage users to share their use cases if they don’t see it covered.

Future contributions should try to stick to the following principles:

Service orientated

The kubernetes UI plugin is currently service orientated. Meaning it is focused on an Entity’s resources on Kubernetes. A cluster orientated version could exist but would probably not be present in the service catalog. For this reason certain Kubernetes Resources are probably not suitable for the current Kubernetes UI plugin, for example the apiGroups:

These resources are more suitable for a cluster administrator and as such should be surfaced in a different way which is still currently hypothetical.

Multi-cluster first

Using multiple clusters is very common and should be supported as a core concept, not an afterthought. Future contributions should consider:

How scalable it would be to fan out operations to multiple clusters including things like:

For this reason we should try to use the Kubernetes API whenever possible as it is fairly vendor agnostic.

Support CRDs

Kinds like Deployments/CronJobs/ingresses are widespread but they are not the only way to use Kubernetes. Many users use tools which lean heavily on CRDs:

It should be possible for users to contribute UIs for different CRDs and display what is appropriate for their use cases.

Future contribution ideas

moustafab commented 3 years ago

@mclarke47 I'd love to contribute to some of this effort. Specifically the arbitrary label selector lookup would be really interesting for me. Let me know how to get started, if I should open a separate issue or how to go about that.

regicsolutions commented 3 years ago

Plus 1 for Istio, ability to look at a service map that show's all the different versions of your services similar to Kiali where you can visualize the traffic and re-route where it makes sense. Example is you have 3 versions of a microservice and you want to re-route the traffic from v2 to v3

braghettos commented 3 years ago

I would like to know your opinion regarding the ability to also deploy a helm chart via this plugin.

benjdlambert commented 3 years ago

Hey @braghettos, thinking that maybe helm should be it's own plugin and not really tied to the k8s underlying API.

Helm is an abstraction on the top of k8s, so I think it deserves it's own space to manage this. But it might depend on the k8s api to talk to tiller or what have you.

https://github.com/backstage/backstage/issues/3905 for reference

braghettos commented 3 years ago

Is it possible to dinamically add a kubernetes cluster in the app-config.yaml file? Otherwise I would need to restart the application every time a new k8s cluster is provisioned.

braghettos commented 3 years ago

Would it be possible to add also secrets to the K8S resources UIs?

benjdlambert commented 3 years ago

Is it possible to dinamically add a kubernetes cluster in the app-config.yaml file? Otherwise I would need to restart the application every time a new k8s cluster is provisioned.

That's the case right now yes, if you have dynamic clusters, then maybe we need to think about some way to do cluster discovery, maybe refreshing kubeconfig or something. Haven't really thought of this.

Would it be possible to add also secrets to the K8S resources UIs?

I'd assume this is pretty easy to implement, however we would probably not display the secret in the frontend yet. Until we have someway to validate that the user has the correct identity to see this secret or something

mclarke47 commented 3 years ago

@braghettos I actually had thought about integrations with cloud providers to pull down cluster information dynamically, perhaps even filtering by some tags.

This is why I added the configuration value clusterLocatorMethods I was hoping to expand this with perhaps GKE EKS AKS etc...

This would be a great contribution.

mclarke47 commented 3 years ago

As @benjdlambert says we actually were going to include secrets but it probably needs some thought in how it is done.

braghettos commented 3 years ago

That's the case right now yes, if you have dynamic clusters, then maybe we need to think about some way to do cluster discovery, maybe refreshing kubeconfig or something. Haven't really thought of this.

I guess that especially for companies that rely on a public cloud provider Kubernetes clusters can be provisioned and wasted quickly so Backstage could expose an API where POSTING a new kubernetes cluster and saving it on the database could help having a dynamic list of clusters and kubeconfigs to check.

I'd assume this is pretty easy to implement, however we would probably not display the secret in the frontend yet. Until we have someway to validate that the user has the correct identity to see this secret or something

I agree with you because the Kubernetes plugin uses the kubeconfig file for Backstage and users should not be able to use it. Would it be possible to launch a Kubectl command within the frontend? Or let user authenticate with their own authentication provider and let kubectl use that tokens?

benjdlambert commented 3 years ago

@mclarke47 yep - totally forgot about the clusterLocator stuff, @braghettos see @mclarke47's response! 👍

braghettos commented 3 years ago

So for GKE invoking this API: https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1/projects.locations.clusters/list - using this client library: https://github.com/googleapis/nodejs-cloud-container but I need to know the project

benjdlambert commented 3 years ago

I guess you'd probably need to add some config specific to the provider also in the config schema then, that's where you could have this project ID or something

mclarke47 commented 3 years ago

@benjdlambert yes, I was considering turning clusterLocatorMethods into an array of objects rather than an array of strings so that you could include, projects and any necessary secrets (if using a different account to list the clusters than retrieve K8S objects)

regicsolutions commented 3 years ago

Kiali has been a fantastic tool for service mesh observability, may want to consider adding that to the list of future contributions.

https://github.com/kiali/kiali-ui

FF86517C-2336-4EB4-AFD2-F1A13491D478

braghettos commented 3 years ago

@mclarke47 regarding the contribution for https://github.com/backstage/backstage/issues/2857#issuecomment-767453431 , should we open an issue only on that and wait for feedbacks from the community? I'm evaluating as well if I can help on this.

Fox32 commented 3 years ago

Would be cool to support ArgoCD App CRDs. That would make a custom plugin for that unnecessary.

mclarke47 commented 3 years ago

Custom UIs for CRDs can now be contributed to https://github.com/backstage/backstage/tree/master/plugins/kubernetes/src/components/CustomResources 🎉

Fox32 commented 3 years ago

Custom UIs for CRDs can now be contributed to https://github.com/backstage/backstage/tree/master/plugins/kubernetes/src/components/CustomResources 🎉

It would be cool if that code would allow configuration from the outside so that one could write interfaces for custom CRDs in own code. Right now you always have to modify the plugin itself if I have seen it correctly?

mclarke47 commented 3 years ago

@Fox32 that's a good idea, especially as orgs might have closed source CRDs they want to add UIs for but don't want it to live in a public repo. 🤔 We could maybe dynamically import CRD components if they are available with an expected name.

benjdlambert commented 3 years ago

@mclarke47 you could add an API to the plugin-kubernetes apiRef that registers components to CRD id's I guess, and have a map that you can look up in for internally and use those.

That way people can override the ArgoCD Rollouts one too if they had some other implementation or whatever. Not that it would be necessary.

We have a similar use case coming up for people providing their own components in the scaffolder for the frontend forms and wanting to render their custom components instead of the default ones.

dhenneke commented 3 years ago

The CRD schema also supports additional printer columns that could be used to provide a reasonable default for all CRDs. But the plugin must be able to read the CRDs from the cluster to do this.

Example: The Application CRD of ArgoCD specifies the following:

additionalPrinterColumns:
  - jsonPath: .status.sync.status
    name: Sync Status
    type: string
  - jsonPath: .status.health.status
    name: Health Status
    type: string
  - jsonPath: .status.sync.revision
    name: Revision
    priority: 10
    type: string

Octant displays it as follows: image

mclarke47 commented 3 years ago

Printer columns would be a better default CRD ui than what I did! I might look into that.

OneCricketeer commented 3 years ago

RE: Multi-cluster

Would that be propagated via kubefed, [2], or just a static list of configured clusters?

mclarke47 commented 3 years ago

@OneCricketeer We currently support 2 methods of cluster discovery:

More details at https://backstage.io/docs/features/kubernetes/configuration#clusterlocatormethods

regicsolutions commented 3 years ago

We have a use case around Idling applications in lower environments in an effort to reduce cloud costs and maybe having Backstage act as the front end to enable developers to "Un-idle" there services once they enter an idle state due to inactivity for X hours or days using a service-idler

Backstage would only orchestrate Un-idling which in essence would consist of scaling up a DeploymentConfig: $ oc scale --replicas=1 dc <dc_name>

Would love to hear what others think about this idea.

keattang commented 2 years ago

Hi,

I'm interested in contributing to this project and am interested in enabling the ability to load cluster information from a user’s kubeconfig. I've created this PR: https://github.com/backstage/backstage/pull/6333 as a proof of concept. I would like to know whether this is the correct approach and if not, what is, before continuing to work on it. Thanks for your time.

manusant commented 2 years ago

Is it possible to dinamically add a kubernetes cluster in the app-config.yaml file? Otherwise I would need to restart the application every time a new k8s cluster is provisioned.

I've just raised a feature request especially targeting this since this was a topic we are working on my company. 6967

Please have a look and also provide some feedback. I'll appreciate.

hwinkel commented 2 years ago

Hi, I would consider bringing Composite Resources (XRDs) as used in crossplane.io to the party. Particularly as they're addressing the same use case as Backstage: Platform Use Cases.

Where Backstage is the UI of the User/Developer Portal, crossplane.io is the platform's composite API (CRD, XRD) provider.

devth commented 2 years ago

What's the next step for this? Should we break the RFC into small tasks to be done? I'd love to contribute!

regicsolutions commented 2 years ago

@mclarke47 came across a cool helm-ui react component, what are your thoughts on having something like this as a custom helm resource?

image

imballinst commented 1 year ago

Custom UIs for CRDs can now be contributed to https://github.com/backstage/backstage/tree/master/plugins/kubernetes/src/components/CustomResources 🎉

It would be cool if that code would allow configuration from the outside so that one could write interfaces for custom CRDs in own code. Right now you always have to modify the plugin itself if I have seen it correctly?

hi @Fox32, not sure if this is related, but I forked and made a branch containing a somewhat PoC of this: https://github.com/backstage/backstage/compare/master...imballinst:backstage:poc. I am using the Deployment core resource as the example here. Would you like to lend me some thoughts about the API/design/approach? Thanks!

So in my PoC, if I want to customize how my DeploymentAccordion looks, I can provide a <CustomKubernetesComponentsContext.Provider> as a wrapper to <EntityKubernetesContent />. The provider has a value of a map between the core resource's kind to the component used for the accordion (the custom component should have the same props as the default one).

<CustomKubernetesComponentsContext.Provider
  value={{
    Deployment: CustomDeploymentAccordion,
  }}
>

If it's provided, then it'll use the custom component, as the picture below where I removed the target CPU usage in Fixture 2:

image

If I keep it as it is like in Fixture 1:

image

If this makes sense, I think I can try to expand it for custom resources as well, so if we provide the Custom Resource's kind in the context provider, it will try to render the passed component. If the context is not provided or the kind is not found in the map, then it'll default to the current switch case (ArgoRollouts/DefaultCustomResource). Please let me know what you think. Thanks!

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

jkleinlercher commented 1 year ago

I want to add a new view for a customresource like ArgoRollouts in https://github.com/backstage/backstage/tree/master/plugins/kubernetes/src/components/CustomResources. What is the best approach here to contribute and test locally?

mclarke47 commented 1 year ago

Hey @jkleinlercher this needs to be implemented https://github.com/backstage/backstage/issues/5511

jkleinlercher commented 1 year ago

Hey @mclarke47 ! Can‘t I extend it the same way as ArgoRollout? The CRD is public so everyone would benefit from it.

mclarke47 commented 1 year ago

@jkleinlercher sure, then it just needs to be added here https://github.com/backstage/backstage/tree/master/plugins/kubernetes/src/components/CustomResources

jkleinlercher commented 1 year ago

@mclarke47 and how can I test it and give others a prerelease before it is merged in backstage main branch? Can I fork just the kubernetes plugin and build it independently in my repo? Sorry for this question but I have no experience with those plugins and npm packages.

mclarke47 commented 1 year ago

Yes, you should be able to fork and build your own version

jessesanford commented 10 months ago

@mclarke47 is this actually completed?

mclarke47 commented 10 months ago

@jessesanford this is an RFC, not an issue tracking the addition of all these features, so I would consider it completed after the comments have been gathered.