Have a process to identify new kubernetes features to use and deprecations to stop using

rata commented 4 years ago

Each Kubernetes release happens every ~3 months and features are added at a very high speed. From the 1.18 release blog post:

Kubernetes 1.18 consists of 38 enhancements: 15 enhancements are moving to stable, 11 enhancements in beta, and 12 enhancements in alpha

I think we should have some kind of process to make we review the new features and think of ways that might benefit Lokomotive if we adopt them.

Also, deprecations happen often and we should be on top of them.

I'm not sure which can be a reasonable way to do this. But just to say something, I propose ew start with:

Every time a new Kubernetes version is released a new issue is created to review it's changelog
More than one person is assigned to it, as there are so many things that I expect one person to sometimes miss something
Each person assigned creates issues for Urgent Upgrade Notes. Hopefully most of them are already fixed as this or another process is in place to be on top of them.
Each person creates issues on adopting/exposing new features added, mentioned in the rest of the Changelog (obviously communicating between them to avoid duplicated issues). It might not be straight forward to see how to adopt/expose a feature, but the idea is to capture the new feature and later polish it. No need to do that at this stage, as that can be too much time consuming. If the issue needs polishing can be indicated with a label (need to create one) and we can decide when we want to allocate time for this. Probably a label for k8s version makes sense too, so maybe something like k8s/1.18 and proposed/needs-polishing (totally open for discussion, of course)
Each person assigned to the issue checks the Kubernetes blog, as some features usually have a blog post about them. The blog post has usually extended explanations and ideas mentioned there, so it can be useful to see how to adopt/expose the feature. After reading them, modify/create issues with ideas on how to adopt/expose the features, if any.
The issue to review the changelog of the new kubernetes release is created from a template, with all things people need to do and is updated with other things that people found interesting (if any). The idea is, if someone found an interesting way to read more about features (like sysdig blog post are usually great, so check them too, or kubernetes started to have an awesome podcast, review other projects like kops release notes, etc.) the template with things to do is updated so we don't rely on people but on a process and anyone can do it the next time. To start with, this template can be on a shared google docs, access to modify can be granted on a case-by-case basis to external collaborators that want to modify it.

There are two tricky things about this procedure:

Every time a Kubernetes version is released, we need to manually create an issue and remind ourselves. This can be solved with subscriptions to Kubernetes discuss blog for the announcements and creating the issues as email arrives, or a calendar event, or both
Polish issues. I think we can include this as tasks we work and that is all.

I think there is a common underlying issue on "how to handle external events" in general, the external event being a new Kubernetes minor release, a Kubernetes security release, a component release, a dependency security release/upgrade, etc. All of this is with a common end goal: keep what we have up to date.

I think that, while there is a common thing underneath, those concerns can be treated differently as usually what we need to do is quite different for those cases. For components we might want to make sure to subscribe to security announces (and general announcements) too and create issues as email arrive, but for a dependency security upgrade we may want to rely on GitHub notifications that scans the code and warns about using versions with known issues, etc.

So, before anyone mentions that, I'd prefer to start small (like, start with this) and create different issues for different concerns. If we later see we can easily treat them the same way, we can merge later. But I don't want to have "finding a way that works for everything" get in the way of starting with this basic stuff.

What do you think @iaguis ?

surajssd commented 4 years ago

I think this can be automated using OPA, I will put some details about this tomorrow morning.

Edit: What I mentioned is only about handling deprecations.

surajssd commented 4 years ago

Using the tool conftest and the policies in this repo, here is the output on our config:

$ lokoctl component render-manifest | docker run --rm -i quay.io/swade1987/deprek8ion:1.1.7 conftest test -p /policies -
2020/04/08 06:24:46 info: skipping unknown hook: "crd-install"
2020/04/08 06:24:46 info: skipping unknown hook: "crd-install"
2020/04/08 06:24:46 info: skipping unknown hook: "crd-install"
2020/04/08 06:24:46 info: skipping unknown hook: "crd-install"
2020/04/08 06:24:46 info: skipping unknown hook: "crd-install"
WARN - ClusterRole/openebs-maya-operator: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRoleBinding/openebs-maya-operator: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - MutatingWebhookConfiguration/prometheus-operator-admission: API admissionregistration.k8s.io/v1beta1 is deprecated in Kubernetes 1.19, use admissionregistration.k8s.io/v1 instead.
WARN - Role/prometheus-operator-grafana: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRole/prometheus-operator-kube-state-metrics: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - CustomResourceDefinition/servicemonitors.monitoring.coreos.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - CustomResourceDefinition/podmonitors.monitoring.coreos.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - ValidatingWebhookConfiguration/prometheus-operator-admission: API admissionregistration.k8s.io/v1beta1 is deprecated in Kubernetes 1.19, use admissionregistration.k8s.io/v1 instead.
WARN - CustomResourceDefinition/alertmanagers.monitoring.coreos.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - ClusterRoleBinding/prometheus-operator-kube-state-metrics: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - RoleBinding/prometheus-operator-grafana: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - CustomResourceDefinition/prometheuses.monitoring.coreos.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - CustomResourceDefinition/prometheusrules.monitoring.coreos.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - CustomResourceDefinition/ingressroutes.contour.heptio.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - CustomResourceDefinition/tlscertificatedelegations.contour.heptio.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - CustomResourceDefinition/httpproxies.projectcontour.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - CustomResourceDefinition/tlscertificatedelegations.projectcontour.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - RoleBinding/contour: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - Role/contour-certgen: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRoleBinding/contour: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRole/contour: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - Role/contour-leaderelection: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - RoleBinding/contour-leaderelection: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - CustomResourceDefinition/certificaterequests.cert-manager.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - ClusterRole/cert-manager-cainjector: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRoleBinding/cert-manager-cainjector: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - Role/cert-manager-cainjector:leaderelection: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - RoleBinding/cert-manager-cainjector:leaderelection: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - MutatingWebhookConfiguration/cert-manager-webhook: API admissionregistration.k8s.io/v1beta1 is deprecated in Kubernetes 1.19, use admissionregistration.k8s.io/v1 instead.
WARN - CustomResourceDefinition/certificates.cert-manager.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - CustomResourceDefinition/clusterissuers.cert-manager.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - ClusterRole/cert-manager-controller-challenges: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRole/cert-manager-controller-clusterissuers: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRole/cert-manager-controller-certificates: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRole/cert-manager-controller-ingress-shim: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRole/cert-manager-controller-orders: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRole/cert-manager-controller-issuers: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRoleBinding/cert-manager-controller-challenges: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRoleBinding/cert-manager-controller-issuers: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRoleBinding/cert-manager-controller-orders: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRoleBinding/cert-manager-controller-ingress-shim: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRoleBinding/cert-manager-controller-certificates: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - ClusterRoleBinding/cert-manager-controller-clusterissuers: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - Role/cert-manager:leaderelection: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - RoleBinding/cert-manager:leaderelection: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - CustomResourceDefinition/orders.acme.cert-manager.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - ValidatingWebhookConfiguration/cert-manager-webhook: API admissionregistration.k8s.io/v1beta1 is deprecated in Kubernetes 1.19, use admissionregistration.k8s.io/v1 instead.
WARN - ClusterRoleBinding/cert-manager-webhook:auth-delegator: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - RoleBinding/cert-manager-webhook:webhook-authentication-reader: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.
WARN - CustomResourceDefinition/challenges.acme.cert-manager.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - CustomResourceDefinition/issuers.cert-manager.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.
WARN - Ingress/httpbin: API extensions/v1beta1 for Ingress is deprecated from Kubernetes 1.20, use networking.k8s.io/v1beta1 instead.
FAIL - Ingress/httpbin: Ingress annotation kubernetes.io/ingress.class has been deprecated in 1.18, use spec.IngressClassName instead.
--------------------------------------------------------------------------------
PASS: 774/827
WARN: 52/827
FAIL: 1/827

Above is not exhaustive, this only covers following components. If we run it in CI we will know if all the components are fine.

$ grep 'component "'
packet-cluster.lokocfg
43:component "openebs-operator" {}
50:component "openebs-storage-class" {
57:component "prometheus-operator" {
62:component "contour" {
66:component "metallb" {
73:component "cert-manager" {
80:component "httpbin" {

We can automate this in CI using the tool conftest. We can maintain our own profiles inspired by the repo: https://github.com/swade1987/deprek8ion/tree/master/policies. Not just for such deprecation tests but also for general unit testing of the configuration.

rata commented 4 years ago

Oh, that is very nice!

In any case, I think we need manual actions for the urgent upgrade procedure and adding/exposing new features to lokomotive, right? Like new way to auth the kubelet (not a real example, just an example), etc.

On Wed, Apr 8, 2020 at 1:30 AM Suraj Deshmukh notifications@github.com wrote:

Using the tool conftest https://github.com/instrumenta/conftest and the policies in this repo https://github.com/swade1987/deprek8ion, here is the output on our config:

$ lokoctl component render-manifest | docker run --rm -i quay.io/swade1987/deprek8ion:1.1.7 conftest test -p /policies -2020/04/08 06:24:46 info: skipping unknown hook: "crd-install"2020/04/08 06:24:46 info: skipping unknown hook: "crd-install"2020/04/08 06:24:46 info: skipping unknown hook: "crd-install"2020/04/08 06:24:46 info: skipping unknown hook: "crd-install"2020/04/08 06:24:46 info: skipping unknown hook: "crd-install"WARN - ClusterRole/openebs-maya-operator: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRoleBinding/openebs-maya-operator: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - MutatingWebhookConfiguration/prometheus-operator-admission: API admissionregistration.k8s.io/v1beta1 is deprecated in Kubernetes 1.19, use admissionregistration.k8s.io/v1 instead.WARN - Role/prometheus-operator-grafana: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRole/prometheus-operator-kube-state-metrics: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - CustomResourceDefinition/servicemonitors.monitoring.coreos.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - CustomResourceDefinition/podmonitors.monitoring.coreos.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - ValidatingWebhookConfiguration/prometheus-operator-admission: API admissionregistration.k8s.io/v1beta1 is deprecated in Kubernetes 1.19, use admissionregistration.k8s.io/v1 instead.WARN - CustomResourceDefinition/alertmanagers.monitoring.coreos.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - ClusterRoleBinding/prometheus-operator-kube-state-metrics: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - RoleBinding/prometheus-operator-grafana: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - CustomResourceDefinition/prometheuses.monitoring.coreos.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - CustomResourceDefinition/prometheusrules.monitoring.coreos.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - CustomResourceDefinition/ingressroutes.contour.heptio.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - CustomResourceDefinition/tlscertificatedelegations.contour.heptio.com: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - CustomResourceDefinition/httpproxies.projectcontour.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - CustomResourceDefinition/tlscertificatedelegations.projectcontour.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - RoleBinding/contour: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - Role/contour-certgen: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRoleBinding/contour: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRole/contour: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - Role/contour-leaderelection: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - RoleBinding/contour-leaderelection: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - CustomResourceDefinition/certificaterequests.cert-manager.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - ClusterRole/cert-manager-cainjector: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRoleBinding/cert-manager-cainjector: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - Role/cert-manager-cainjector:leaderelection: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - RoleBinding/cert-manager-cainjector:leaderelection: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - MutatingWebhookConfiguration/cert-manager-webhook: API admissionregistration.k8s.io/v1beta1 is deprecated in Kubernetes 1.19, use admissionregistration.k8s.io/v1 instead.WARN - CustomResourceDefinition/certificates.cert-manager.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - CustomResourceDefinition/clusterissuers.cert-manager.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - ClusterRole/cert-manager-controller-challenges: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRole/cert-manager-controller-clusterissuers: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRole/cert-manager-controller-certificates: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRole/cert-manager-controller-ingress-shim: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRole/cert-manager-controller-orders: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRole/cert-manager-controller-issuers: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRoleBinding/cert-manager-controller-challenges: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRoleBinding/cert-manager-controller-issuers: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRoleBinding/cert-manager-controller-orders: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRoleBinding/cert-manager-controller-ingress-shim: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRoleBinding/cert-manager-controller-certificates: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - ClusterRoleBinding/cert-manager-controller-clusterissuers: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - Role/cert-manager:leaderelection: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - RoleBinding/cert-manager:leaderelection: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - CustomResourceDefinition/orders.acme.cert-manager.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - ValidatingWebhookConfiguration/cert-manager-webhook: API admissionregistration.k8s.io/v1beta1 is deprecated in Kubernetes 1.19, use admissionregistration.k8s.io/v1 instead.WARN - ClusterRoleBinding/cert-manager-webhook:auth-delegator: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - RoleBinding/cert-manager-webhook:webhook-authentication-reader: API rbac.authorization.k8s.io/v1beta1 is deprecated from Kubernetes 1.20, use rbac.authorization.k8s.io/v1 instead.WARN - CustomResourceDefinition/challenges.acme.cert-manager.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - CustomResourceDefinition/issuers.cert-manager.io: API apiextensions.k8s.io/v1beta1 for CustomResourceDefinition is deprecated in 1.19, use apiextensions.k8s.io/v1 instead.WARN - Ingress/httpbin: API extensions/v1beta1 for Ingress is deprecated from Kubernetes 1.20, use networking.k8s.io/v1beta1 instead.FAIL - Ingress/httpbin: Ingress annotation kubernetes.io/ingress.class has been deprecated in 1.18, use spec.IngressClassName instead.--------------------------------------------------------------------------------PASS: 774/827WARN: 52/827FAIL: 1/827

Above is not exhaustive, this only covers following components. If we run it in CI we will know if all the components are fine.

$ grep 'component "'packet-cluster.lokocfg43:component "openebs-operator" {}50:component "openebs-storage-class" {57:component "prometheus-operator" {62:component "contour" {66:component "metallb" {73:component "cert-manager" {80:component "httpbin" {

We can automate this in CI using the tool conftest. We can maintain our own profiles inspired by the repo: https://github.com/swade1987/deprek8ion/tree/master/policies. Not just for such deprecation tests but also for general unit testing of the configuration.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/kinvolk/lokomotive/issues/276#issuecomment-610744997, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAARJTILAYTAAYS2VMHLMUTRLP4VTANCNFSM4MDIBRIQ .

-- Rodrigo Campos

Kinvolk GmbH | Adalbertstr.6a, 10999 Berlin | tel: +491755589364 Geschäftsführer/Directors: Alban Crequy, Chris Kühl, Iago López Galeiras Registergericht/Court of registration: Amtsgericht Charlottenburg Registernummer/Registration number: HRB 171414 B Ust-ID-Nummer/VAT ID number: DE302207000

kinvolk / lokomotive

Have a process to identify new kubernetes features to use and deprecations to stop using #276

-- Rodrigo Campos