riverqueue / river

Fast and reliable background jobs in Go
https://riverqueue.com
Mozilla Public License 2.0
3.55k stars 93 forks source link

SharedTx flaky test #321

Closed bgentry closed 6 months ago

bgentry commented 6 months ago

Encountered this one here:

panic: sharedtx: Timed out trying to acquire lock on SharedTx

goroutine 6611 [running]:
github.com/riverqueue/river/internal/riverinternaltest/sharedtx.(*SharedTx).lock(0xc00037aed0)
    /home/runner/work/river/river/internal/riverinternaltest/sharedtx/shared_tx.go:117 +0xf6
github.com/riverqueue/river/internal/riverinternaltest/sharedtx.(*SharedTx).QueryRow(0xc00037aed0, {0xd0b910, 0xc0000c6460}, {0xc3f489, 0x26c}, {0xc000230340, 0x4, 0x4})
    /home/runner/work/river/river/internal/riverinternaltest/sharedtx/shared_tx.go:89 +0x5d
github.com/riverqueue/river/riverdriver/riverpgxv5/internal/dbsqlc.(*Queries).JobDeleteBefore(0x6fc2398b5?, {0xd0b910, 0xc0000c6460}, {0x7fad4cb0c540, 0xc00037aed0}, 0xc000297a30)
    /home/runner/work/river/river/riverdriver/riverpgxv5/internal/dbsqlc/river_job.sql.go:127 +0x24d
github.com/riverqueue/river/riverdriver/riverpgxv5.(*Executor).JobDeleteBefore(0xc00041c9f0, {0xd0b910, 0xc0000c6460}, 0xc0002fd9f0)
    /home/runner/work/river/river/riverdriver/riverpgxv5/river_pgx_v5_driver.go:103 +0x296
github.com/riverqueue/river/internal/maintenance.(*JobCleaner).runOnce.func1({0xd0b8a0, 0xc0000c82d0}, 0xc00026[20](https://github.com/riverqueue/river/actions/runs/8870315848/job/24351966280?pr=301#step:8:21)80)
    /home/runner/work/river/river/internal/maintenance/job_cleaner.go:150 +0x3e9
github.com/riverqueue/river/internal/maintenance.(*JobCleaner).runOnce(0xc000262080, {0xd0b8a0, 0xc0000c82d0})
    /home/runner/work/river/river/internal/maintenance/job_cleaner.go:161 +0x[22](https://github.com/riverqueue/river/actions/runs/8870315848/job/24351966280?pr=301#step:8:23)5
github.com/riverqueue/river/internal/maintenance.(*JobCleaner).Start.func1()
    /home/runner/work/river/river/internal/maintenance/job_cleaner.go:120 +0x3cc
created by github.com/riverqueue/river/internal/maintenance.(*JobCleaner).Start in goroutine 6588
    /home/runner/work/river/river/internal/maintenance/job_cleaner.go:104 +0x185
FAIL    github.com/riverqueue/river/internal/maintenance    8.194s
brandur commented 6 months ago

Doh.