search

karmada-io / karmada

Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration
https://karmada.io
Apache License 2.0
4.11k stars 805 forks source link

add e2etest for aggregated api endpoint #1400

Closed wwwnay closed 2 years ago

wwwnay commented 2 years ago

What type of PR is this?

What this PR does / why we need it: Add e2e test case for aggregated-api-endpoint Which issue(s) this PR fixes: Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE
karmada-bot commented 2 years ago

@wwwnay: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
karmada-bot commented 2 years ago

Welcome @wwwnay! It looks like this is your first PR to karmada-io/karmada 🎉

RainbowMango commented 2 years ago

Wow! Big thanks!!! @wwwnay

/assign @XiShanYongYe-Chang

XiShanYongYe-Chang commented 2 years ago

@wwwnay thanks!

Maybe you can signoff refer to https://github.com/karmada-io/karmada/pull/1400/checks?check_run_id=5331140890

XiShanYongYe-Chang commented 2 years ago

Hi @wwwnay, have you occur any problems, maybe we can discuss.

wwwnay commented 2 years ago

@XiShanYongYe-Chang I have update ,please review

XiShanYongYe-Chang commented 2 years ago

Hi @wwwnay, have you tested it on your local site? I test it failed on my side.

wwwnay commented 2 years ago

Hi @wwwnay, have you tested it on your local site? I test it failed on my side.

yes,the test have passed on my side what is the error report ------------------------------ aggregatedapi testing Serviceaccount Unified authentication Test Serviceaccount Unified authentication Test Grant no permission to Serviceaccount in member2 cluster. /root/go/src/karmada/test/e2e/aggregatedapi_test.go:135 STEP: Creating serviceaccount(tom) in membercluster (member1) STEP: Creating serviceaccount(tom) in karmadaCluster STEP: Creating ClusterRole(cluster-proxy-clusterrole) STEP: Creating ClusterRole(cluster-proxy-clusterrolebinding) STEP: Remove sa(tom) STEP: Remove sa(tom) STEP: Remove ClusterRole(cluster-proxy-clusterrole) STEP: Remove ClusterRole(cluster-proxy-clusterrolebinding) clusterroles.rbac.authorization.k8s.io "tom" not found clusterrolebindings.rbac.authorization.k8s.io "tom" not found • Ran 5 of 5 Specs in 2.556 seconds SUCCESS! -- 5 Passed | 0 Failed | 0 Pending | 0 Skipped PASS

XiShanYongYe-Chang commented 2 years ago

Passed | 0 Failed | 0 Pending | 0 Skipped PASS

No cases appear to be executed.

You can directly use commend hack/run-e2e.sh under karmada path.

In order to focus on someone target, you can update like this:

--- a/hack/run-e2e.sh
+++ b/hack/run-e2e.sh
@@ -38,7 +38,7 @@ export KUBECONFIG=${KARMADA_APISERVER_KUBECONFIG}
 export PULL_BASED_CLUSTERS=${PULL_BASED_CLUSTERS}

 set +e
-ginkgo -v -race -failFast ./test/e2e/
+ginkgo -v -race -failFast -focus="aggregatedapi testing" ./test/e2e/
 TESTING_RESULT=$?

Error info on my side:

------------------------------
aggregatedapi testing Serviceaccount Unified authentication Test Serviceaccount Unified authentication Test 
  Serviceaccount  access in the member1 cluster
  /root/go/src/github.com/karmada-io/karmada/test/e2e/aggregatedapi_test.go:107
STEP: Creating serviceaccount(tom) in membercluster (member1)
STEP: Creating serviceaccount(tom) in karmadaCluster
STEP: Creating ClusterRole(cluster-proxy-clusterrole)
STEP: Creating ClusterRole(cluster-proxy-clusterrolebinding)
STEP: Remove sa(tom)
STEP: Remove sa(tom)
STEP: Remove ClusterRole(cluster-proxy-clusterrole)
STEP: Remove ClusterRole(cluster-proxy-clusterrolebinding)
clusterroles.rbac.authorization.k8s.io "tom" not found
clusterrolebindings.rbac.authorization.k8s.io "tom" not found

• Failure [0.150 seconds]
aggregatedapi testing
/root/go/src/github.com/karmada-io/karmada/test/e2e/aggregatedapi_test.go:13
  Serviceaccount Unified authentication Test
  /root/go/src/github.com/karmada-io/karmada/test/e2e/aggregatedapi_test.go:105
    Serviceaccount Unified authentication Test
    /root/go/src/github.com/karmada-io/karmada/test/e2e/aggregatedapi_test.go:106
      Serviceaccount  access in the member1 cluster [It]
      /root/go/src/github.com/karmada-io/karmada/test/e2e/aggregatedapi_test.go:107

      Expected
          <int>: 403
      to equal
          <int>: 200

      /root/go/src/github.com/karmada-io/karmada/test/e2e/aggregatedapi_test.go:110
------------------------------
wwwnay commented 2 years ago

Passed | 0 Failed | 0 Pending | 0 Skipped PASS

No cases appear to be executed.

You can directly use commend hack/run-e2e.sh under karmada path.

In order to focus on someone target, you can update like this:

--- a/hack/run-e2e.sh
+++ b/hack/run-e2e.sh
@@ -38,7 +38,7 @@ export KUBECONFIG=${KARMADA_APISERVER_KUBECONFIG}
 export PULL_BASED_CLUSTERS=${PULL_BASED_CLUSTERS}

 set +e
-ginkgo -v -race -failFast ./test/e2e/
+ginkgo -v -race -failFast -focus="aggregatedapi testing" ./test/e2e/
 TESTING_RESULT=$?

Error info on my side:

------------------------------
aggregatedapi testing Serviceaccount Unified authentication Test Serviceaccount Unified authentication Test 
  Serviceaccount  access in the member1 cluster
  /root/go/src/github.com/karmada-io/karmada/test/e2e/aggregatedapi_test.go:107
STEP: Creating serviceaccount(tom) in membercluster (member1)
STEP: Creating serviceaccount(tom) in karmadaCluster
STEP: Creating ClusterRole(cluster-proxy-clusterrole)
STEP: Creating ClusterRole(cluster-proxy-clusterrolebinding)
STEP: Remove sa(tom)
STEP: Remove sa(tom)
STEP: Remove ClusterRole(cluster-proxy-clusterrole)
STEP: Remove ClusterRole(cluster-proxy-clusterrolebinding)
clusterroles.rbac.authorization.k8s.io "tom" not found
clusterrolebindings.rbac.authorization.k8s.io "tom" not found

• Failure [0.150 seconds]
aggregatedapi testing
/root/go/src/github.com/karmada-io/karmada/test/e2e/aggregatedapi_test.go:13
  Serviceaccount Unified authentication Test
  /root/go/src/github.com/karmada-io/karmada/test/e2e/aggregatedapi_test.go:105
    Serviceaccount Unified authentication Test
    /root/go/src/github.com/karmada-io/karmada/test/e2e/aggregatedapi_test.go:106
      Serviceaccount  access in the member1 cluster [It]
      /root/go/src/github.com/karmada-io/karmada/test/e2e/aggregatedapi_test.go:107

      Expected
          <int>: 403
      to equal
          <int>: 200

      /root/go/src/github.com/karmada-io/karmada/test/e2e/aggregatedapi_test.go:110
------------------------------

thanks, I run about 20 times and failed 2 times ,i will check what happened when it failed

wwwnay commented 2 years ago

/retest

karmada-bot commented 2 years ago

@wwwnay: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to [this](https://github.com/karmada-io/karmada/pull/1400#issuecomment-1066606927): >/retest Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
XiShanYongYe-Chang commented 2 years ago

Hi @wwwnay , retest doesn't work now. You can sign off refer to https://github.com/karmada-io/karmada/pull/1400/checks?check_run_id=5535871399

Have you test it sucessful on you local site?

Maybe you need to squash your commits and remove merge commit.

wwwnay commented 2 years ago

yes,i run it successful on my local site.

wwwnay commented 2 years ago 
Running Suite: E2E Suite
========================
Random Seed: 1647254966
Will run 5 of 5 specs

I0314 18:49:31.251656 2317773 cluster.go:162] Got 3 member cluster and all in ready state.
aggregatedapi testing when Serviceaccount access test Serviceaccount access test
  Serviceaccount  access in the member1 cluster
  /root/go/src/karmada/test/e2e/aggregatedapi_test.go:104
STEP: Creating serviceaccount(tom) in membercluster (member1)
STEP: Creating serviceaccount(tom) in karmadaCluster
STEP: Creating ClusterRole(cluster-proxy-clusterrole)
STEP: Creating ClusterRoleBinding(cluster-proxy-clusterrolebinding)
STEP: wait for clusterRoleBinding(cluster-proxy-clusterrolebinding) present
STEP: wait for secret (tom) present in cluster
STEP: Remove serviceaccount(tom) in membercluster (member1)
STEP: Remove serviceaccount(tom) in karmadaCluster
STEP: Remove ClusterRole(cluster-proxy-clusterrole)
STEP: Remove ClusterRole(cluster-proxy-clusterrolebinding)
•
------------------------------
aggregatedapi testing when Serviceaccount access test Serviceaccount access test
  Serviceaccount have no  access in the member2 cluster
  /root/go/src/karmada/test/e2e/aggregatedapi_test.go:112
STEP: Creating serviceaccount(tom) in membercluster (member1)
STEP: Creating serviceaccount(tom) in karmadaCluster
STEP: Creating ClusterRole(cluster-proxy-clusterrole)
STEP: Creating ClusterRoleBinding(cluster-proxy-clusterrolebinding)
STEP: wait for clusterRoleBinding(cluster-proxy-clusterrolebinding) present
STEP: wait for secret (tom) present in cluster
STEP: Remove serviceaccount(tom) in membercluster (member1)
STEP: Remove serviceaccount(tom) in karmadaCluster
STEP: Remove ClusterRole(cluster-proxy-clusterrole)
STEP: Remove ClusterRole(cluster-proxy-clusterrolebinding)
•
------------------------------
aggregatedapi testing when Serviceaccount access test Serviceaccount access test
  Serviceaccount  does not have any permissions in the member1 cluster
  /root/go/src/karmada/test/e2e/aggregatedapi_test.go:121
STEP: Creating serviceaccount(tom) in membercluster (member1)
STEP: Creating serviceaccount(tom) in karmadaCluster
STEP: Creating ClusterRole(cluster-proxy-clusterrole)
STEP: Creating ClusterRoleBinding(cluster-proxy-clusterrolebinding)
STEP: wait for clusterRoleBinding(cluster-proxy-clusterrolebinding) present
STEP: wait for secret (tom) present in cluster
STEP: Remove serviceaccount(tom) in membercluster (member1)
STEP: Remove serviceaccount(tom) in karmadaCluster
STEP: Remove ClusterRole(cluster-proxy-clusterrole)
STEP: Remove ClusterRole(cluster-proxy-clusterrolebinding)
•
------------------------------
aggregatedapi testing when Grant permission to Serviceaccount in member
  Grant permission to Serviceaccount in member1 cluster.
  /root/go/src/karmada/test/e2e/aggregatedapi_test.go:142
STEP: Creating serviceaccount(tom) in membercluster (member1)
STEP: Creating serviceaccount(tom) in karmadaCluster
STEP: Creating ClusterRole(cluster-proxy-clusterrole)
STEP: Creating ClusterRoleBinding(cluster-proxy-clusterrolebinding)
STEP: Creating member clusterrole(tom) in clustermember(member1)
STEP: Creating member clusterrolebinding(tom)
STEP: wait for clusterRoleBinding(cluster-proxy-clusterrolebinding) present
STEP: wait for clusterRoleBinding(tom) present in member(member1) cluster
STEP: wait for secret (tom) present in cluster
STEP: Remove clusterRole(tom) in member(member1)
STEP: Remove clusterRoleBinding(tom) in member(member1)
STEP: Remove serviceaccount(tom) in membercluster (member1)
STEP: Remove serviceaccount(tom) in karmadaCluster
STEP: Remove ClusterRole(cluster-proxy-clusterrole)
STEP: Remove ClusterRole(cluster-proxy-clusterrolebinding)
•
------------------------------
aggregatedapi testing when Grant permission to Serviceaccount in member
  Grant no permission to Serviceaccount in member2 cluster.
  /root/go/src/karmada/test/e2e/aggregatedapi_test.go:151
STEP: Creating serviceaccount(tom) in membercluster (member1)
STEP: Creating serviceaccount(tom) in karmadaCluster
STEP: Creating ClusterRole(cluster-proxy-clusterrole)
STEP: Creating ClusterRoleBinding(cluster-proxy-clusterrolebinding)
STEP: Creating member clusterrole(tom) in clustermember(member1)
STEP: Creating member clusterrolebinding(tom)
STEP: wait for clusterRoleBinding(cluster-proxy-clusterrolebinding) present
STEP: wait for clusterRoleBinding(tom) present in member(member1) cluster
STEP: wait for secret (tom) present in cluster
STEP: Remove clusterRole(tom) in member(member1)
STEP: Remove clusterRoleBinding(tom) in member(member1)
STEP: Remove serviceaccount(tom) in membercluster (member1)
STEP: Remove serviceaccount(tom) in karmadaCluster
STEP: Remove ClusterRole(cluster-proxy-clusterrole)
STEP: Remove ClusterRole(cluster-proxy-clusterrolebinding)
•
Ran 5 of 5 Specs in 4.783 seconds
SUCCESS! -- 5 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
XiShanYongYe-Chang commented 2 years ago

Hi @wwwnay, are you still working with this e2e test?

wwwnay commented 2 years ago

Hi @wwwnay, are you still working with this e2e test?

ah.sorry, I didn't realized I was pushed failed. 

I0321 18:42:53.786107  125910 cluster.go:162] Got 3 member cluster and all in ready state.
aggregatedapi testing when Serviceaccount access test Serviceaccount access test
  Serviceaccount  access in the member1 cluster
  /root/go/src/karmada/test/e2e/aggregatedapi_test.go:65
STEP: Creating serviceaccount(tom) in membercluster (member1) in namespace karmadatest-7z6
STEP: Creating serviceaccount(tom) in karmadaCluster in namespace karmadatest-7z6
STEP: Creating ClusterRole(cluster-proxy-clusterrole)
STEP: Creating ClusterRoleBinding(tom)
STEP: wait for serviceaccount (tom) present in karmadacluster  namespace karmadatest-7z6
STEP: wait for serviceaccount (tom) present in member(member1) cluster in namespace karmadatest-7z6
STEP: wait for clusterRoleBinding(tom) present
STEP: wait for secret (tom) present in cluster  in namespace karmadatest-7z6
STEP: Remove serviceaccount(tom) in membercluster (member1)
STEP: Remove serviceaccount(tom) in karmadaCluster
STEP: Remove ClusterRole(cluster-proxy-clusterrole)
STEP: Remove ClusterRole(tom)
•
------------------------------
aggregatedapi testing when Serviceaccount access test Serviceaccount access test
  Serviceaccount have no  access in the member2 cluster
  /root/go/src/karmada/test/e2e/aggregatedapi_test.go:74
STEP: Creating serviceaccount(tom) in membercluster (member1) in namespace karmadatest-7z6
STEP: Creating serviceaccount(tom) in karmadaCluster in namespace karmadatest-7z6
STEP: Creating ClusterRole(cluster-proxy-clusterrole)
STEP: Creating ClusterRoleBinding(tom)
STEP: wait for serviceaccount (tom) present in karmadacluster  namespace karmadatest-7z6
STEP: wait for serviceaccount (tom) present in member(member1) cluster in namespace karmadatest-7z6
STEP: wait for clusterRoleBinding(tom) present
STEP: wait for secret (tom) present in cluster  in namespace karmadatest-7z6
STEP: Remove serviceaccount(tom) in membercluster (member1)
STEP: Remove serviceaccount(tom) in karmadaCluster
STEP: Remove ClusterRole(cluster-proxy-clusterrole)
STEP: Remove ClusterRole(tom)
•
------------------------------
aggregatedapi testing when Serviceaccount access test Serviceaccount access test
  Serviceaccount  does not have any permissions in the member1 cluster
  /root/go/src/karmada/test/e2e/aggregatedapi_test.go:84
STEP: Creating serviceaccount(tom) in membercluster (member1) in namespace karmadatest-7z6
STEP: Creating serviceaccount(tom) in karmadaCluster in namespace karmadatest-7z6
STEP: Creating ClusterRole(cluster-proxy-clusterrole)
STEP: Creating ClusterRoleBinding(tom)
STEP: wait for serviceaccount (tom) present in karmadacluster  namespace karmadatest-7z6
STEP: wait for serviceaccount (tom) present in member(member1) cluster in namespace karmadatest-7z6
STEP: wait for clusterRoleBinding(tom) present
STEP: wait for secret (tom) present in cluster  in namespace karmadatest-7z6
STEP: Remove serviceaccount(tom) in membercluster (member1)
STEP: Remove serviceaccount(tom) in karmadaCluster
STEP: Remove ClusterRole(cluster-proxy-clusterrole)
STEP: Remove ClusterRole(tom)
•
------------------------------
aggregatedapi testing when Grant permission to Serviceaccount in member
  Grant permission to Serviceaccount in member1 cluster.
  /root/go/src/karmada/test/e2e/aggregatedapi_test.go:107
STEP: Creating serviceaccount(tom) in membercluster (member1) in namespace karmadatest-7z6
STEP: Creating serviceaccount(tom) in karmadaCluster in namespace karmadatest-7z6
STEP: Creating ClusterRole(cluster-proxy-clusterrole)
STEP: Creating ClusterRoleBinding(tom)
STEP: Creating member clusterrole(tom) in clustermember(member1)
STEP: Creating member clusterrolebinding(tom)
STEP: wait for serviceaccount (tom) present in karmadacluster  namespace karmadatest-7z6
STEP: wait for serviceaccount (tom) present in member(member1) cluster in namespace karmadatest-7z6
STEP: wait for clusterRoleBinding(tom) present
STEP: wait for clusterRoleBinding(tom) present in member(member1) cluster
STEP: wait for secret (tom) present in cluster  in namespace karmadatest-7z6
STEP: Remove clusterRole(tom) in member(member1)
STEP: Remove clusterRoleBinding(tom) in member(member1)
STEP: Remove serviceaccount(tom) in membercluster (member1)
STEP: Remove serviceaccount(tom) in karmadaCluster
STEP: Remove ClusterRole(cluster-proxy-clusterrole)
STEP: Remove ClusterRole(tom)
•
------------------------------
aggregatedapi testing when Grant permission to Serviceaccount in member
  Grant no permission to Serviceaccount in member2 cluster.
  /root/go/src/karmada/test/e2e/aggregatedapi_test.go:117
STEP: Creating serviceaccount(tom) in membercluster (member1) in namespace karmadatest-7z6
STEP: Creating serviceaccount(tom) in karmadaCluster in namespace karmadatest-7z6
STEP: Creating ClusterRole(cluster-proxy-clusterrole)
STEP: Creating ClusterRoleBinding(tom)
STEP: Creating member clusterrole(tom) in clustermember(member1)
STEP: Creating member clusterrolebinding(tom)
STEP: wait for serviceaccount (tom) present in karmadacluster  namespace karmadatest-7z6
STEP: wait for serviceaccount (tom) present in member(member1) cluster in namespace karmadatest-7z6
STEP: wait for clusterRoleBinding(tom) present
STEP: wait for clusterRoleBinding(tom) present in member(member1) cluster
STEP: wait for secret (tom) present in cluster  in namespace karmadatest-7z6
STEP: Remove clusterRole(tom) in member(member1)
STEP: Remove clusterRoleBinding(tom) in member(member1)
STEP: Remove serviceaccount(tom) in membercluster (member1)
STEP: Remove serviceaccount(tom) in karmadaCluster
STEP: Remove ClusterRole(cluster-proxy-clusterrole)
STEP: Remove ClusterRole(tom)
•
Ran 5 of 5 Specs in 3.778 seconds
SUCCESS! -- 5 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
RainbowMango commented 2 years ago

There is a conflict?

RainbowMango commented 2 years ago

@wwwnay Thanks for doing this.

Would you mind I and @XiShanYongYe-Chang help with you and moving this PR forward? (I can push code to your branch)

wwwnay commented 2 years ago

@wwwnay Thanks for doing this.

Would you mind I and @XiShanYongYe-Chang help with you and moving this PR forward? (I can push code to your branch)

Please help me with this, thanks.

RainbowMango commented 2 years ago

Ok. @XiShanYongYe-Chang is working on it and after that, I'll help push the changes to your branch. Just wait for a while.

XiShanYongYe-Chang commented 2 years ago

Hi @RainbowMango @wwwnay, I refactor the related code on my fork repository:

https://github.com/XiShanYongYe-Chang/karmada/tree/pr1400

XiShanYongYe-Chang commented 2 years ago

Hi @wwwnay, can you have a look?

wwwnay commented 2 years ago

Hi @wwwnay, can you have a look?

thanks, it's fine with me , much better than what i do

XiShanYongYe-Chang commented 2 years ago

thanks, it's fine with me , much better than what i do

Thanks! thank you very much too! Would you like to add more E2E or UT for karmada?

wwwnay commented 2 years ago

thanks, it's fine with me , much better than what i do

Thanks! thank you very much too! Would you like to add more E2E or UT for karmada?

yes,i am interested， i will keep trying to add E2E or UT for karmada

RainbowMango commented 2 years ago

@wwwnay, thanks again. Here is the Github workflow guideline it might help you.

XiShanYongYe-Chang commented 2 years ago

@wwwnay, thanks again. Here is the Github workflow guideline it might help you.

Doc site: https://github.com/karmada-io/karmada/blob/master/docs/contributors/guide/github-workflow.md

RainbowMango commented 2 years ago

Sorry, I missed the link. thanks @XiShanYongYe-Chang

wwwnay commented 2 years ago

@wwwnay, thanks again. Here is the Github workflow guideline it might help you.

Doc site: https://github.com/karmada-io/karmada/blob/master/docs/contributors/guide/github-workflow.md

@RainbowMango ,I will fllow the workflow guideline next time . @XiShanYongYe-Chang thanks again for help me with this.

karmada-bot commented 2 years ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: RainbowMango, XiShanYongYe-Chang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[test/OWNERS](https://github.com/karmada-io/karmada/blob/master/test/OWNERS)~~ [RainbowMango,XiShanYongYe-Chang] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment