TestAddCredentialFinalizers UT randomly fails

[RolandMa1986]

Describe the Bug

TestAddCredentialFinalizers UT randomly fails.

ref: https://github.com/kubesphere/kubesphere/runs/2771946724?check_suite_focus=true

Versions Used KubeSphere: 3.1.0 Kubernetes: (If KubeSphere installer used, you can skip this)

Environment How many nodes and their hardware configuration:

For example: CentOS 7.5 / 3 masters: 8cpu/8g; 3 nodes: 8cpu/16g (and other info are welcomed to help us debugging)

How To Reproduce

I0608 08:40:31.354345   28018 devopscredential_controller.go:220] namespace 'test-123/test' in work queue no longer exists 
--- FAIL: TestAddCredentialFinalizers (0.00s)
    devopscredential_controller_test.go:191: 1 additional expected actions:[{ActionImpl:{Namespace:test-123 Verb:update Resource:/v1, Resource=secrets Subresource:} Object:&Secret{ObjectMeta:{test  test-123    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[credential.devops.kubesphere.io/autosync:true credential.devops.kubesphere.io/syncstatus:successful] [] [finalizers.kubesphere.io/credential]  []},Data:map[string][]byte{},Type:credential.devops.kubesphere.io/test,StringData:map[string]string{},Immutable:nil,}}]
    devopscredential_controller_test.go:200:  credential &Secret{ObjectMeta:{test  test-123    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[credential.devops.kubesphere.io/autosync:true credential.devops.kubesphere.io/syncstatus:successful] [] [finalizers.kubesphere.io/credential]  []},Data:map[string][]byte{},Type:credential.devops.kubesphere.io/test,StringData:map[string]string{},Immutable:nil,} not match 
         &Secret{ObjectMeta:{test  test-123    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[credential.devops.kubesphere.io/autosync:true] [] []  []},Data:map[string][]byte{},Type:credential.devops.kubesphere.io/test,StringData:map[string]string{},Immutable:nil,}
FAIL

Expected behavior UT test should always pass.

/kind bug /area devops /cc @kubesphere/devops-admin

[JohnNiang]

Thanks for reaching out.

IMO, the unstable test may be introduced by https://github.com/kubesphere/kubesphere/pull/3944, but I'm not sure. You were supposed to debug the test with your changed codes.

[RolandMa1986]

@JohnNiang First of all, It not related to kubesphere/kubesphere#3944. When checking the details, you can found that we upgraded protobuf in kubesphere/kubesphere#3944. but the UT won't call any protobuf related functions. All the UT is using fake clients. Secondly, It's only happened one time in the pipeline. The same commit triggers 2 builds, but one successfully and another fails. When I force to push the commit again with the same changes. Both Go kubesphere/kubesphere#2605 and BuildMultiArch kubesphere/kubesphere#81 build successfully .

BTW, I never reproduced locally.

It looks more like an environmental issue or race condition. rather than a new issue introduced by kubesphere/kubesphere#3944.

Regards, Roland

[JohnNiang]

@RolandMa1986

Thanks for your correction. We will pay attention to this issue later.

[JohnNiang]

/cc @kubesphere/sig-devops

[ks-ci-bot]

@RolandMa1986: The label(s) area/devops cannot be applied, because the repository doesn't have them.

In response to [this](https://github.com/kubesphere/ks-devops/issues/135): >**Describe the Bug** > >TestAddCredentialFinalizers UT randomly fails. > >ref: https://github.com/kubesphere/kubesphere/runs/2771946724?check_suite_focus=true > >**Versions Used** >KubeSphere: 3.1.0 >Kubernetes: (If KubeSphere installer used, you can skip this) > >**Environment** >How many nodes and their hardware configuration: > >For example: CentOS 7.5 / 3 masters: 8cpu/8g; 3 nodes: 8cpu/16g >(and other info are welcomed to help us debugging) > >**How To Reproduce** > >``` >I0608 08:40:31.354345 28018 devopscredential_controller.go:220] namespace 'test-123/test' in work queue no longer exists >--- FAIL: TestAddCredentialFinalizers (0.00s) > devopscredential_controller_test.go:191: 1 additional expected actions:[{ActionImpl:{Namespace:test-123 Verb:update Resource:/v1, Resource=secrets Subresource:} Object:&Secret{ObjectMeta:{test test-123 0 0001-01-01 00:00:00 +0000 UTC map[] map[credential.devops.kubesphere.io/autosync:true credential.devops.kubesphere.io/syncstatus:successful] [] [finalizers.kubesphere.io/credential] []},Data:map[string][]byte{},Type:credential.devops.kubesphere.io/test,StringData:map[string]string{},Immutable:nil,}}] > devopscredential_controller_test.go:200: credential &Secret{ObjectMeta:{test test-123 0 0001-01-01 00:00:00 +0000 UTC map[] map[credential.devops.kubesphere.io/autosync:true credential.devops.kubesphere.io/syncstatus:successful] [] [finalizers.kubesphere.io/credential] []},Data:map[string][]byte{},Type:credential.devops.kubesphere.io/test,StringData:map[string]string{},Immutable:nil,} not match > &Secret{ObjectMeta:{test test-123 0 0001-01-01 00:00:00 +0000 UTC map[] map[credential.devops.kubesphere.io/autosync:true] [] [] []},Data:map[string][]byte{},Type:credential.devops.kubesphere.io/test,StringData:map[string]string{},Immutable:nil,} >FAIL >``` > >**Expected behavior** >UT test should always pass. > >/kind bug >/area devops >/cc @kubesphere/devops-admin Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.