make CRD from v1beta1 to v1

[jichenjc]

What this PR does / why we need it:

make CRD become v1 of virtualcluster/config/crd/tenancy.x-k8s.io_clusterversions.yaml

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): Fixes #

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jichenjc To complete the pull request process, please assign charleszheng44 after the PR has been reviewed. You can assign the PR to them by writing /assign @charleszheng44 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[virtualcluster/OWNERS](https://github.com/kubernetes-sigs/cluster-api-provider-nested/blob/main/virtualcluster/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[jichenjc]

/test pull-cluster-api-provider-nested-test

[jichenjc]

weird , my local test success ..

[Fei-Guo]

/retest

[Fei-Guo]

By the way, will this cause compatibility problem for old K8S version? When is the CRD API version promoted to v1?

[k8s-ci-robot]

@jichenjc: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-cluster-api-provider-nested-test	c548343ad1dcb03ae364090164eaa0050376d238	link	/test pull-cluster-api-provider-nested-test

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).

[christopherhein]

weird , my local test success ..

This could be a version difference w/ envtest and the binaries that are used to validate. @Fei-Guo 1.22 - https://kubernetes.io/docs/reference/using-api/deprecation-guide/#customresourcedefinition-v122

[jichenjc]

weird , my local test success ..

This could be a version difference w/ envtest and the binaries that are used to validate. @Fei-Guo 1.22 - https://kubernetes.io/docs/reference/using-api/deprecation-guide/#customresourcedefinition-v122

yeah, 1.22

Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
customresourcedefinition.apiextensions.k8s.io/clusterversions.tenancy.x-k8s.io created

my local test succeed

Ran 0 of 0 Specs in 6.684 seconds
SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestAPIs (6.68s)
PASS
ok      sigs.k8s.io/cluster-api-provider-nested/controlplane/nested/controllers (cached)
?       sigs.k8s.io/cluster-api-provider-nested/hack/boilerplate/test   [no test files]

[jichenjc]

/hold

my bad, I run test only on nested folder, not on virtual cluster folder so need further test locally before submit

[jichenjc]

root@jjtest1:~# kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/cluster-api-provider-nested/master/virtualcluster/config/crd/tenancy.x-k8s.io_clusterversions.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/cluster-api-provider-nested/master/virtualcluster/config/crd/tenancy.x-k8s.io_virtualclusters.yaml
error: unable to recognize "https://raw.githubusercontent.com/kubernetes-sigs/cluster-api-provider-nested/master/virtualcluster/config/crd/tenancy.x-k8s.io_clusterversions.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"

FYI, use latest k8s cluster has problem so this is something need to be fixed before 1.22 GA

[christopherhein]

@jichenjc are you still trying to get this fixed up?

[jichenjc]

@jichenjc are you still trying to get this fixed up?

yes, still working on , if any comment/help can be provided that will be helpful

[christopherhein]

@jichenjc are you still trying to get this fixed up?

yes, still working on , if any comment/help can be provided that will be helpful

Sounds good, I'll try to take a look Tuesday if you haven't figured it out. Something that might help is #131 to help remove some of the CRDs now that we have the CAPN provisioner we might be able to wholly shift to CAPN and remove the native provisioner.

[jichenjc]

I narrowed down to here https://github.com/kubernetes-sigs/cluster-api-provider-nested/blob/main/virtualcluster/pkg/controller/controllers/virtualcluster_controller_test.go#L48 but still not able to fix the problem

clusterVersion created here returns empty when Get function called, means this object not created successfully I believe 1) we should error out here first (can be done in further PR) 2) I think the new CRD v1 is different to previous v1beta1 is multiple versions are added, somewhere need reflect but need help..

[jichenjc]

I think it's due to I manually updated this file, it should be regenerated though there are some other places might need update as well, still checking but unforunately my env is broken recently

[jichenjc]

I gave up ..after fixed above issue , saw another one .. no idea what it happened though the code looks fine maybe someone can help fix this later on

2021-07-09T14:45:24.685+0800    ERROR   controllers.VirtualCluster      fail to create virtualcluster   {"vc": "virtualcluster-samplewcghg", "retrytimes": 1, "error": "StatefulSet.apps \"etcd\" is invalid: spec.template.metadata.labels: Invalid value: map[string]string(nil): `selector` does not match template `labels`"}