search

karmada-io / karmada

Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration
https://karmada.io
Apache License 2.0
4.11k stars 805 forks source link

update kube-apiserver and kube-controller-manager default version #1107

Closed Shike-Ada closed 2 years ago

Shike-Ada commented 2 years ago

What type of PR is this?

/kind feature

What this PR does / why we need it: Upgrade k8s version to v1.21.7 Which issue(s) this PR fixes: Fixes #

Special notes for your reviewer: The latest version of K8s image with v1.21.x is 7. Does this PR introduce a user-facing change?:

RainbowMango commented 2 years ago

Found a failing test:

porting workloads testing
/home/runner/work/karmada/karmada/test/e2e/porting_workloads_test.go:19
  porting workloads from legacy clusters testing
  /home/runner/work/karmada/karmada/test/e2e/porting_workloads_test.go:21
    porting Deployments from legacy clusters testing
    /home/runner/work/karmada/karmada/test/e2e/porting_workloads_test.go:40
------------------------------
[MCS] Multi-Cluster Service testing Connectivity testing 
  Export Service from source-clusters, import Service to destination-clusters
  /home/runner/work/karmada/karmada/test/e2e/mcs_test.go:260
STEP: Creating ClusterPropagationPolicy(serviceexports-c6g-policy)
STEP: Creating ClusterPropagationPolicy(serviceimports-9st-policy)
STEP: Check if crd(multicluster.x-k8s.io/v1alpha1/ServiceExport) present on member clusters
I1214 15:16:29.577424   40089 customresourcedefine.go:58] Waiting for crd present on cluster(member1)
I1214 15:16:34.624492   40089 customresourcedefine.go:58] Waiting for crd present on cluster(member2)
I1214 15:16:39.663811   40089 customresourcedefine.go:58] Waiting for crd present on cluster(member3)
STEP: Check if crd(multicluster.x-k8s.io/v1alpha1/ServiceImport) present on member clusters
I1214 15:16:39.670935   40089 customresourcedefine.go:58] Waiting for crd present on cluster(member1)
I1214 15:16:39.678695   40089 customresourcedefine.go:58] Waiting for crd present on cluster(member2)
I1214 15:16:39.686116   40089 customresourcedefine.go:58] Waiting for crd present on cluster(member3)
STEP: Creating Deployment(karmadatest-wcg/hello-2jh)
I1214 15:16:39.692804   40089 mcs_test.go:227] Create Deployment(karmadatest-wcg/hello-2jh) in member1 cluster
I1214 15:16:39.701538   40089 mcs_test.go:230] Create Service(karmadatest-wcg/hello-2jh) in member1 cluster
STEP: Creating Service(karmadatest-wcg/hello-2jh)
STEP: Wait Service(karmadatest-wcg/hello-2jh)'s EndpointSlice exist in member1 cluster
STEP: Create ServiceExport(karmadatest-wcg/hello-2jh)
STEP: Creating PropagationPolicy(karmadatest-wcg/export-hello-2jh-policy)
STEP: Wait EndpointSlices collected to namespace(karmadatest-wcg) in controller-plane
STEP: Create ServiceImport(karmadatest-wcg/hello-2jh)
STEP: Creating PropagationPolicy(karmadatest-wcg/import-hello-2jh-policy)
STEP: Wait derived-service(karmadatest-wcg/derived-hello-2jh) exist in member2 cluster
STEP: Wait EndpointSlices have been imported to member2 cluster
I1214 15:21:54.827869   40089 mcs_test.go:253] Delete Deployment(karmadatest-wcg/hello-2jh) in member1 cluster
STEP: Removing Deployment(karmadatest-wcg/hello-2jh)
STEP: Removing Service(karmadatest-wcg/hello-2jh)
I1214 15:21:54.841631   40089 mcs_test.go:256] Delete Service(karmadatest-wcg/hello-2jh) in member1 cluster
STEP: Removing ClusterPropagationPolicy(serviceimports-9st-policy)
STEP: Removing ClusterPropagationPolicy(serviceexports-c6g-policy)

• Failure [325.532 seconds]
[MCS] Multi-Cluster Service testing
/home/runner/work/karmada/karmada/test/e2e/mcs_test.go:178
  Connectivity testing
  /home/runner/work/karmada/karmada/test/e2e/mcs_test.go:220
    Export Service from source-clusters, import Service to destination-clusters [It]
    /home/runner/work/karmada/karmada/test/e2e/mcs_test.go:260

    Timed out after 300.003s.
    Expected
        <int>: 0
    to equal
        <int>: 1

    /home/runner/work/karmada/karmada/test/e2e/mcs_test.go:319
------------------------------
SSSSSSSSSSSS

Summarizing 1 Failure:

[Fail] [MCS] Multi-Cluster Service testing Connectivity testing [It] Export Service from source-clusters, import Service to destination-clusters 
/home/runner/work/karmada/karmada/test/e2e/mcs_test.go:319

Ran 19 of 31 Specs in 804.028 seconds
FAIL! -- 18 Passed | 1 Failed | 0 Pending | 12 Skipped
--- FAIL: TestE2E (804.12s)
FAIL
RainbowMango commented 2 years ago

/lgtm /approve

RainbowMango commented 2 years ago

/hold @algebra2k @jrkeen Should we update the chart version as well? (since we do changed templates)

jrkeen commented 2 years ago

/hold @algebra2k @jrkeen Should we update the chart version as well? (since we do changed templates)

I think it`s necessary @Shike-Ada

Shike-Ada commented 2 years ago

Okay, I will try it soon.

RainbowMango commented 2 years ago

Just update it to 0.0.3 should be ok. By the way, could you please help check why the E2E is falling?

Shike-Ada commented 2 years ago

I think this may require modifying the K8S parameter, and I will try to inspect it soon.

RainbowMango commented 2 years ago

You can ignore the FOSSA check. There must be something wrong with the service. I'll look at it.

Shike-Ada commented 2 years ago

The e2e test can't pass and I do not know how to fix it. Maybe it exists incompatibility problem.

RainbowMango commented 2 years ago

We've fixed several failing tests. So please rebase your code and try again.

Shike-Ada commented 2 years ago

It still doesn't work, and e2e test failed in the same function again..

XiShanYongYe-Chang commented 2 years ago

It still doesn't work, and e2e test failed in the same function again.

Hi, @Shike-Ada. With the current pr, it's bound to happen. You can run hack/run-e2e.sh in you local env.

To avoid running the case in full, you can specify a topic:

ginkgo -v -race -failFast -focus="XXX" ./test/e2e/

just modify the line in hack/run-e2e.sh:

https://github.com/karmada-io/karmada/blob/7dcb197f9e62be4a361bf5f29e56802f6321663f/hack/run-e2e.sh#L37

Shike-Ada commented 2 years ago

Sorry, I don't know rebase operation in the past and have known it now. And In my local env, I do not have enough resource to run it. Maybe I would use the online test.

Shike-Ada commented 2 years ago

image Although having skipped a lot of use cases, a error has occurred.

RainbowMango commented 2 years ago

Maybe @XiShanYongYe-Chang could help take a look since he is the test owner.

XiShanYongYe-Chang commented 2 years ago

Maybe @XiShanYongYe-Chang could help take a look since he is the test owner.

@Shike-Ada, ok, let me have a look at my local site.

XiShanYongYe-Chang commented 2 years ago

Hi, @Shike-Ada discovery.k8s.io/v1beta1 EndpointSlices are deprecated in favor of discovery.k8s.io/v1, we can refer to https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md

So you need to update like this:

--- a/test/e2e/mcs_test.go
+++ b/test/e2e/mcs_test.go
@@ -232,7 +232,7 @@ var _ = ginkgo.Describe("[MCS] Multi-Cluster Service testing", func() {

                        ginkgo.By(fmt.Sprintf("Wait Service(%s/%s)'s EndpointSlice exist in %s cluster", demoService.Namespace, demoService.Name, serviceExportClusterName), func() {
                                gomega.Eventually(func(g gomega.Gomega) (int, error) {
-                                       endpointSlices, err := exportClusterClient.DiscoveryV1beta1().EndpointSlices(demoService.Namespace).List(context.TODO(), metav1.ListOptions{
+                                       endpointSlices, err := exportClusterClient.DiscoveryV1().EndpointSlices(demoService.Namespace).List(context.TODO(), metav1.ListOptions{
                                                LabelSelector: labels.Set{discoveryv1beta1.LabelServiceName: demoService.Name}.AsSelector().String(),
                                        })
                                        g.Expect(err).NotTo(gomega.HaveOccurred())

Update DiscoveryV1beta1() to DiscoveryV1() in all place.

XiShanYongYe-Chang commented 2 years ago

Besides, we use discoveryv1beta1 in the controller, we also need to update it to discoveryv1.

XiShanYongYe-Chang commented 2 years ago

Hi @Shike-Ada, you can directly cherry-pick my commit: https://github.com/XiShanYongYe-Chang/karmada/commit/047e0d7997f93de95c23405751ff794f41870722

I have passed the CI on my local site.

Shike-Ada commented 2 years ago

Thanks for your help!@XiShanYongYe-Chang ! I will cherry-pick it in this day.

XiShanYongYe-Chang commented 2 years ago

@Shike-Ada , It looks ok.

/cc @RainbowMango

RainbowMango commented 2 years ago

/lgtm /approve

karmada-bot commented 2 years ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: RainbowMango

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/karmada-io/karmada/blob/master/OWNERS)~~ [RainbowMango] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
RainbowMango commented 2 years ago

/hold cancel