search

openshift-online / maestro

Maestro Service Repo
Apache License 2.0
8 stars 15 forks source link

the integration test is flaky #75

Closed clyang82 closed 2 months ago

clyang82 commented 3 months ago 
[unnamed-0] W0402 03:09:36.367828   13984 logger.go:129]   Updating status for stale resource; disregard as the latest version is: 1 
[unnamed-0]     resource_test.go:276: 
[unnamed-0]         Timed out after 10.000s.
[unnamed-0]         Expected success, but got an error:
[unnamed-0]             <*errors.errorString | 0xc0006daf10>: 
[unnamed-0]             unexpected work version 0
[unnamed-0]             {
[unnamed-0]                 s: "unexpected work version 0",
[unnamed-0]             }
[unnamed-0] --- FAIL: TestResourcePatch (10.04s)
[unnamed-0] FAIL test/integration.TestResourcePatch (10.04s)
[unnamed-0] PASS test/integration.TestResourcePaging (0.10s)
[unnamed-0] PASS test/integration.TestResourceListSearch (0.08s)
[unnamed-0] PASS test/integration.TestUpdateResourceWithRacingRequests (2.09s)
[unnamed-0] PASS test/integration.TestResourceFromGRPC (4.07s)
[unnamed-0] I0402 03:09:50.360324   13984 api_server.go:157] Web server terminated
[unnamed-0] E0402 03:09:50.360476   13984 logger.go:129]   error obtaining the non blocking advisory lock for id maestro-instances-pulse-check type instances, context canceled 
[unnamed-0] I0402 03:09:50.360560   13984 healthcheck_server.go:60] HealthCheck server terminated
[unnamed-0] E0402 03:09:50.360604   13984 baseclient.go:90] the cloudevents client is disconnected, read tcp [::1]:44552->[::1]:1883: use of closed network connection
[unnamed-0] E0402 03:09:50.360727   13984 logger.go:129]   error obtaining the instance lock: error obtaining the non blocking advisory lock for id maestro-instances-pulse-check type instances, context canceled 
[unnamed-0] FAIL test/integration
[unnamed-0] 
[unnamed-0] === Failed
[unnamed-0] === FAIL: test/integration TestResourcePatch (10.04s)
[unnamed-0] {"level":"info","ts":1712027374.0121367,"logger":"fallback","caller":"v2@v2.0.0-20231030012137-0836a524e995/protocol.go:124","msg":"subscribing to topics: map[sources/maestro/consumers/55b65dc4-c5d1-4b66-874b-9bf4a6f47d47/sourceevents:{1 0 false false}]"}
[unnamed-0] W0402 03:09:34.012331   13984 agentoptions.go:63] the agent broadcast topic not set, fall back to the agent events topic
[unnamed-0] I0402 03:09:34.013173   13984 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/a5d6e605-f92b-43d0-9b8f-6f3e980ddda7"
[unnamed-0] I0402 03:09:34.022670   13984 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/foo"
[unnamed-0] I0402 03:09:34.024138   13984 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/a5d6e605-f92b-43d0-9b8f-6f3e980ddda7"
[unnamed-0] E0402 03:09:36.324796   13984 agentclient.go:168] failed to resync manifestsstatus, failed to find a codec for event io.open-cluster-management.works.v1alpha1.manifestbundles
[unnamed-0] W0402 03:09:36.367828   13984 logger.go:129]   Updating status for stale resource; disregard as the latest version is: 1 
[unnamed-0]     resource_test.go:276: 
[unnamed-0]         Timed out after 10.000s.
[unnamed-0]         Expected success, but got an error:
[unnamed-0]             <*errors.errorString | 0xc0006daf10>: 
[unnamed-0]             unexpected work version 0
[unnamed-0]             {
[unnamed-0]                 s: "unexpected work version 0",
[unnamed-0]             }
[unnamed-0] 
[unnamed-0] DONE 15 tests, 1 failure in 167.649s
[unnamed-0] make: *** [Makefile:214: test-integration] Error 1
clyang82 commented 2 months ago

/assign

working on ...

clyang82 commented 2 months ago 
[unnamed-0] === RUN   TestResourcePost
[unnamed-0] {"level":"info","ts":1712671495.9543536,"logger":"fallback","caller":"v2@v2.0.0-20231030012137-0836a524e995/protocol.go:124","msg":"subscribing to topics: map[sources/maestro/consumers/cluster1/sourceevents:{1 0 false false}]"}
[unnamed-0] W0409 14:04:55.954584   12372 agentoptions.go:63] the agent broadcast topic not set, fall back to the agent events topic
[unnamed-0] I0409 14:04:55.955813   12372 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "POST"/"/api/maestro/v1/resources"
[unnamed-0] I0409 14:04:55.976286   12372 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "POST"/"/api/maestro/v1/resources"
[unnamed-0]     resource_test.go:116: 
[unnamed-0]         Timed out after 10.000s.
[unnamed-0]         Expected success, but got an error:
[unnamed-0]             <*errors.errorString | 0xc000b63770>: 
[unnamed-0]             unexpected work list []
[unnamed-0]             {
[unnamed-0]                 s: "unexpected work list []",
[unnamed-0]             }
[unnamed-0] --- FAIL: TestResourcePost (10.23s)
[unnamed-0] FAIL test/integration.TestResourcePost (10.23s)
[unnamed-0] PASS test/integration.TestResourcePatch (1.11s)
[unnamed-0] PASS test/integration.TestResourcePaging (0.72s)
[unnamed-0] PASS test/integration.TestResourceListSearch (0.46s)
[unnamed-0] PASS test/integration.TestUpdateResourceWithRacingRequests (1.19s)
[unnamed-0] PASS test/integration.TestResourceFromGRPC (4.17s)
[unnamed-0] I0409 14:05:13.630490   12372 api_server.go:157] Web server terminated
[unnamed-0] I0409 14:05:13.630697   12372 healthcheck_server.go:60] HealthCheck server terminated
[unnamed-0] FAIL test/integration
[unnamed-0] 
[unnamed-0] === Failed
[unnamed-0] === FAIL: test/integration TestResourcePost (10.23s)
[unnamed-0] {"level":"info","ts":1712671495.9543536,"logger":"fallback","caller":"v2@v2.0.0-20231030012137-0836a524e995/protocol.go:124","msg":"subscribing to topics: map[sources/maestro/consumers/cluster1/sourceevents:{1 0 false false}]"}
[unnamed-0] W0409 14:04:55.954584   12372 agentoptions.go:63] the agent broadcast topic not set, fall back to the agent events topic
[unnamed-0] I0409 14:04:55.955813   12372 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "POST"/"/api/maestro/v1/resources"
[unnamed-0] I0409 14:04:55.976286   12372 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "POST"/"/api/maestro/v1/resources"
[unnamed-0]     resource_test.go:116: 
[unnamed-0]         Timed out after 10.000s.
[unnamed-0]         Expected success, but got an error:
[unnamed-0]             <*errors.errorString | 0xc000b63770>: 
[unnamed-0]             unexpected work list []
[unnamed-0]             {
[unnamed-0]                 s: "unexpected work list []",
[unnamed-0]             }
[unnamed-0] 
[unnamed-0] DONE 15 tests, 1 failure in 620.821s
[unnamed-0] make: *** [Makefile:214: test-integration] Error 1
clyang82 commented 2 months ago 
=== RUN   TestUpdateResourceWithRacingRequests
I0416 09:13:54.961063   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.964926   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.965155   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.965949   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.965951   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.966326   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.966326   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.966695   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.967393   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.967736   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.969067   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.969417   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.969546   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.970049   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.970435   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.971694   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.971848   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.972438   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.973696   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
I0416 09:13:54.976063   13594 authz_middleware_mock.go:18] Mock authz allows <any>/<any> for "PATCH"/"/api/maestro/v1/resources/c52b86d3-a33c-4cb2-a888-4890dba3cecd"
    resource_test.go:471: 
        Timed out after 5.000s.
        Expected success, but got an error:
            <*errors.errorString | 0xc000f1eae0>: 
            there are 1 unreleased advisory lock
            {
                s: "there are 1 unreleased advisory lock",
            }
--- FAIL: TestUpdateResourceWithRacingRequests (5.09s)
clyang82 commented 2 months ago 
[unnamed-0] === RUN   TestPulseServer
[unnamed-0] {"level":"info","ts":1713218075.4088767,"logger":"fallback","caller":"v2@v2.0.0-20231030012137-0836a524e995/protocol.go:124","msg":"subscribing to topics: map[sources/maestro/consumers/cluster1/sourceevents:{1 0 false false}]"}
[unnamed-0] W0415 21:54:35.408978   12842 agentoptions.go:63] the agent broadcast topic not set, fall back to the agent events topic
[unnamed-0]     pulse_server_test.go:127: 
[unnamed-0]         Timed out after 5.000s.
[unnamed-0]         Expected success, but got an error:
[unnamed-0]             <*errors.errorString | 0xc00041de90>: 
[unnamed-0]             resource status is empty
[unnamed-0]             {
[unnamed-0]                 s: "resource status is empty",
[unnamed-0]             }
[unnamed-0] --- FAIL: TestPulseServer (9.12s)