Closed timothysc closed 4 years ago
As discussed in Slack, pls review https://master.cluster-api.sigs.k8s.io/clusterctl/commands/config-cluster.html#variables and https://master.cluster-api.sigs.k8s.io/clusterctl/configuration.html#variables. We don't have anything other than variable substitution (no conditionals, no loops, etc.), but hopefully this is enough for now.
/milestone Next
Big +1 on this, having defaults, conditionals and loops available will make templates a much better experience.
Also, rendering the template server-side will make it easier to adopt from other programming languages, as currently the clusterctl client library is only available in go.
So our current user experience could use some work, I'll try to update the ask to outline in more detail the workflow I'm thinking about.
@frapposelli if you're looking for server-side rendering, it's possible, but because we're CRD-based, it would end up being a request/response pattern and not a blocking call.
Some initial considerations: There are concerns that are common across templating engines:
The last point is where template engines are generally not adequate, because they are designed to receive in input a list of values in some format (e.g. Helm, ytt).
Instead, what is required for improving the user experience around "workload cluster template/flavors" is something more than a list of values; it is required to have the list of variables included in each template, where each variable have at least:
How to fill the gap between list of values and list of variables in a template agnostic way requires some design work.
Let's move the discussion to a proposal google doc, we can do a POC using FeatureGates
in CAPI as v0.3.0-rc.1
/kind proposal
A small, but potentially very beneficial improvement for now would be the ability for providers to provide a variable description manifest, so that
➜ clusterctl config cluster test --infrastructure=vsphere:v0.0.0
Error: failed to perform variable substitution: value for variables [CLUSTER_CIDR, K8S_IMAGE_REPOSITORY, KUBERNETES_VERSION, SERVICE_CIDR, SERVICE_DOMAIN, SSH_AUTHORIZED_KEY, VSPHERE_DATACENTER, VSPHERE_DATASTORE, VSPHERE_DISK_GIB, VSPHERE_FOLDER_VAR, VSPHERE_MEM_MIB, VSPHERE_NETWORK, VSPHERE_NUM_CPUS, VSPHERE_RESOURCE_POOL, VSPHERE_SERVER, VSPHERE_TEMPLATE] is not set. Please set the value using os environment variables or the clusterctl config file
can be more informative. Wouldn't necessarily require the adoption of a templating engine.
In the past, at least with Ruby, have used i18n packages for messages and ended up with things that looked like the following:
options:
- name: AWS_ACCOUNT
description: The AWS Account ID for the ${cluster} cluster
messages:
invalid: |-
To find your AWS account ID number in the AWS Management Console, click on
Support in the navigation bar in the upper-right, and then click Support
Center. Your currently signed in account ID appears in the upper-right
corner below the Support menu.
For further information, see,
http://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html
We at WeaveWorks have a set of templates as helm charts that could be utilised. Currently updating these for alpha3 and would consider contributing them and making them available on a helm repository.
We at WeaveWorks have a set of templates as helm charts that could be utilised. Currently updating these for alpha3 and would consider contributing them and making them available on a helm repository.
Essentially what @casibbald is referring to is complete set of templates not only for clusterapi.yaml, but also working method of standing up the cluster fully equipped with selected components like CNI, Ingress controller, Policy agent, Monitoring and Logging stack etc by just pointing to Git dir with templates. We are working with WeaveWorks to apply that work at Deutsche Telekom Technik and working together to expand it to full GitOps based cluster management.
@vincepri @timothysc We could organize dedicated Zoom to walk you through our use case. I originally intended to do that in person in Amsterdam, but Corona....
I'm interested too, and I imagine @sedefsavas will be as well.
My vision is to use Sedef's work in #2395 to get a minimal set of things installed as soon as a cluster is created:
Getting the "addon manager(s)" installed would get the managers/operators configured for things like the CNI provider that we had to install manually (a bit of chicken & egg), CoreDNS, kube-proxy, and whatever else you may want in all your clusters (monitoring, logging, etc).
I know the above isn't necessarily about cluster templating, but it relates to your suggestion @vukg.
Great! We will plan for it together with WeaveWorks partners and ping the community via Slack when we have things in place (couple of days).
We intend to make whole that concept open and contribute it.
It seems that it will take us a bit more time to make migration from 0.2.0 to 0.3.0 than we originally anticipated. We will come back to demo once we have stable setup.
I'm interested in this discussion as well
Interested as well.
@ncdc I wonder if what you're describing could also apply for out-of-tree cloud provider managers and CSI (related to https://github.com/kubernetes-sigs/cluster-api-provider-azure/issues/456#issuecomment-605171003). In this case we'd want all clusters with a certain K8s version (eg. 1.18+) to install the CCM and CSI addons.
We have it running for all components and providers and fully managed by helm
On Fri, 27 Mar 2020 at 22:41, Cecile Robert-Michon notifications@github.com wrote:
Interested as well.
@ncdc https://github.com/ncdc I wonder if what you're describing could also apply for out-of-tree cloud provider managers and CSI (related to kubernetes-sigs/cluster-api-provider-azure#456 (comment) https://github.com/kubernetes-sigs/cluster-api-provider-azure/issues/456#issuecomment-605171003). In this case we'd want all clusters with a certain K8s version (eg. 1.18+) to install the CCM and CSI addons.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kubernetes-sigs/cluster-api/issues/2339#issuecomment-605344513, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA6C3Y2LZZSTQY7SGYKAADRJUTQRANCNFSM4KVMYTNQ .
We would be ready to demonstrate together with @casibbald from Weaveworks our use case with CAPI incl. cluster templating, bootstrapping of workload clusters and installation of standard components at bootstrapping time using Helm and few other things.
If you agree I would go for dedicated call. I will reach out via Slack to fix suitable slot.
@ncdc @fabriziopandini @timothysc @vincepri @sedefsavas
Hi @vukg, if you'd like to demo something, I would encourage you to attend the weekly Cluster API Office Hours meeting, Wednesdays at 1pm ET. You can sign up for a demo by adding it in the "Demos/POCs" section here
@devigned and I have been thinking about this overall problem lately, wondering if anyone on this thread has thoughts on https://github.com/kubernetes-sigs/cluster-api-provider-azure/pull/540 since it's related.
It would also be useful to provide templating functionality for clusterctl init
as well.
For example, it would be useful to template the infrastructure-components.yaml for a provider to change the way the provider is configured.
One concrete example of this is the desire to use a different identity type when initializing a CAPZ provider. Many of the cloud providers offer credentialed access or usage of an implicit identity provisioned on the node. If the configuration was templated, it would be easy to produce one template that allowed a user of clusterctl init
to specify the desired configuration as input and produce a yaml with the desired identity configuration.
/assign @wfernandes
/lifecycle active
As per April 27th 2020 community guidelines, this project follows the process outlined in https://github.com/kubernetes-sigs/cluster-api/blob/master/CONTRIBUTING.md#proposal-process-caep for large features or changes.
Following those guidelines, I'll go ahead and close this issue for now and defer to contributors interested in pushing the proposal forward to open a collaborative document proposal instead, and follow the process as described.
/close
@vincepri: Closing this issue.
User Story
As a admin I would like to provide my team with a set of well defined templates w/defaults so users only have to input a limited subset of variables when standing up a cluster.
Detailed Description
As of today, the clusterapi.yaml can be intimidating for users to consume, which poses a barrier to broader adoption. An easy way to mitigate this problem is to create well defined flavor.templates that have a limited set of user input variables. This pattern is very common in the kubernetes community, e.g helm charts.
The purpose of this is issue is to enable clusterctl to create a pluggable template model for flavors. For the purpose of simplicity we may want to start with templates that are relatively simple and well defined go-lang libraries and support e.g. https://github.com/k14s/ytt with the option of creating plugable templates for operators who prefer their own secret sauce.
Anything else you would like to add:
On creation of the flavor only the template variables are presented as possible input options in order to simplify the user experience.
This would be highly valuable to the end user community.
/cc @fabriziopandini /kind feature