search

riverqueue / river

Fast and reliable background jobs in Go
https://riverqueue.com
Mozilla Public License 2.0
3.37k stars 89 forks source link

Flaky Client stop test #154

Closed bgentry closed 6 months ago

bgentry commented 8 months ago

From this run:

--- FAIL: Test_Client (0.00s)
    --- FAIL: Test_Client/Stopped ([10](https://github.com/riverqueue/river/actions/runs/7544000573/job/20536298471#step:8:11).14s)
        logger.go:225: time=2024-01-16T15:42:32.755Z level=INFO msg="River client successfully started"
        logger.go:225: time=2024-01-16T15:42:32.757Z level=INFO msg="Election change received" is_leader=false
        logger.go:225: time=2024-01-16T15:42:32.757Z level=INFO msg="producer: Producer started" queue=default
        logger.go:225: time=2024-01-16T15:42:32.757Z level=INFO msg="producer: Run loop started"
        logger.go:225: time=2024-01-16T15:42:32.859Z level=INFO msg="Client[github.com/jackc/pgx/v5.Tx]: Hard stop started; cancelling all work"
        logger.go:225: time=2024-01-16T15:42:32.876Z level=INFO msg="producer: Run loop stopped"
        logger.go:225: time=2024-01-16T15:42:32.876Z level=INFO msg="producer: Producer stopped" queue=default num_completed_jobs=1
        client_test.go:466: context deadline exceeded
        logger.go:225: time=2024-01-16T15:42:37.860Z level=INFO msg="Client[github.com/jackc/pgx/v5.Tx]: Stop started"
        client_test.go:156: 
                Error Trace:    /home/runner/work/river/river/client_test.go:156
                                            /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:[11](https://github.com/riverqueue/river/actions/runs/7544000573/job/20536298471#step:8:12)69
                                            /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:[13](https://github.com/riverqueue/river/actions/runs/7544000573/job/20536298471#step:8:14)47
                                            /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:[15](https://github.com/riverqueue/river/actions/runs/7544000573/job/20536298471#step:8:16)89
                                            /opt/hostedtoolcache/go/1.21.5/x64/src/runtime/panic.go:541
                                            /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:999
                                            /opt/hostedtoolcache/go/1.21.5/x64/src/testing/testing.go:1076
                                            /home/runner/work/river/river/client_test.go:466
                Error:          Received unexpected error:
                                context deadline exceeded
                Test:           Test_Client/Stopped
FAIL
FAIL    github.com/riverqueue/river [20](https://github.com/riverqueue/river/actions/runs/7544000573/job/20536298471#step:8:21).060s

cc @brandur

brandur commented 6 months ago

@bgentry I've run hundreds of test runs in the last couple weeks and I haven't seen this one at all — I think one of the service refactors over the last month may have inadvertently patched it up. Going to close for now, but feel free to reopen if you see it again.