search

pingcap / tidb

TiDB - the open-source, cloud-native, distributed SQL database designed for modern applications.
https://pingcap.com
Apache License 2.0
37.2k stars 5.84k forks source link

unstable test in the TestUseDB #36628

Closed hawkingrei closed 1 year ago

hawkingrei commented 2 years ago

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

[2022-07-27T11:13:11.064Z] fatal error: runtime·unlock: lock count
[2022-07-27T11:13:11.064Z] fatal error: runtime·unlock: lock count
[2022-07-27T11:13:11.064Z] 
[2022-07-27T11:13:11.064Z] runtime stack:
[2022-07-27T11:13:11.064Z] runtime.throw({0x4045b9e?, 0x7f9567543388?})
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:992 +0x71
[2022-07-27T11:13:11.064Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.064Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.064Z] runtime.unlock(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.064Z] runtime.printunlock()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.064Z] runtime.throw.func1()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.064Z] runtime.throw({0x4045b9e?, 0x7f9567543420?})
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.064Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.064Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.064Z] runtime.unlock(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.064Z] runtime.printunlock()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.064Z] runtime.throw.func1()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.064Z] runtime.throw({0x4045b9e?, 0x7f95675434b8?})
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.064Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.064Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.064Z] runtime.unlock(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.064Z] runtime.printunlock()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.064Z] runtime.throw.func1()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.064Z] runtime.throw({0x4045b9e?, 0x7f9567543550?})
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.064Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.064Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.064Z] runtime.unlock(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.064Z] runtime.printunlock()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.064Z] runtime.throw.func1()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.064Z] runtime.throw({0x4045b9e?, 0x7f95675435e8?})
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.064Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.064Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.064Z] runtime.unlock(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.064Z] runtime.printunlock()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.064Z] runtime.throw.func1()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.064Z] runtime.throw({0x4045b9e?, 0x7f9567543680?})
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.064Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.064Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.064Z] runtime.unlock(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.064Z] runtime.printunlock()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.064Z] runtime.throw.func1()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.064Z] runtime.throw({0x4045b9e?, 0x7f9567543718?})
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.064Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.064Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.064Z] runtime.unlock(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.064Z] runtime.printunlock()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.064Z] runtime.throw.func1()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.064Z] runtime.throw({0x4045b9e?, 0x7f95675437b0?})
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.064Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.064Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.064Z] runtime.unlock(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.064Z] runtime.printunlock()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.064Z] runtime.throw.func1()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.064Z] runtime.throw({0x4045b9e?, 0x7f9567543848?})
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.064Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.064Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.064Z] runtime.unlock(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.064Z] runtime.printunlock()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.064Z] runtime.throw.func1()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.064Z] runtime.throw({0x4045b9e?, 0x7f95675438e0?})
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.064Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.064Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.064Z] runtime.unlock(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.064Z] runtime.printunlock()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.064Z] runtime.throw.func1()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.064Z] runtime.throw({0x4045b9e?, 0x7f9567543978?})
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.064Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.064Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.064Z] runtime.unlock(...)
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.064Z] runtime.printunlock()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.064Z] runtime.throw.func1()
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.064Z] runtime.throw({0x4045b9e?, 0x7f9567543a10?})
[2022-07-27T11:13:11.064Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f9567543aa8?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f9567543b40?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f9567543bd8?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f9567543c70?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f9567543d08?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f9567543da0?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f9567543e38?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f9567543ed0?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f9567543f68?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f9567544000?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f9567544098?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f9567544130?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] runtime.throw({0x4045b9e?, 0x7f95675441c8?})
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:985 +0x46
[2022-07-27T11:13:11.065Z] runtime.unlock2(0x1b?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:126 +0x7a
[2022-07-27T11:13:11.065Z] runtime.unlockWithRank(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lockrank_off.go:31
[2022-07-27T11:13:11.065Z] runtime.unlock(...)
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/lock_futex.go:111
[2022-07-27T11:13:11.065Z] runtime.printunlock()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/print.go:80 +0x3b
[2022-07-27T11:13:11.065Z] runtime.throw.func1()
[2022-07-27T11:13:11.065Z]  GOROOT/src/runtime/panic.go:986 +0x55
[2022-07-27T11:13:11.065Z] 
[2022-07-27T11:13:11.065Z] goroutine 1 [chan receive]:
[2022-07-27T11:13:11.065Z] testing.(*T).Run(0xc000f581a0, {0x400413c?, 0x1719945?}, 0x413d998)
[2022-07-27T11:13:11.065Z]  GOROOT/src/testing/testing.go:1487 +0x37a
[2022-07-27T11:13:11.065Z] testing.runTests.func1(0xc000f24ea0?)
[2022-07-27T11:13:11.065Z]  GOROOT/src/testing/testing.go:1839 +0x6e
[2022-07-27T11:13:11.065Z] testing.tRunner(0xc000f581a0, 0xc000e5fb28)
[2022-07-27T11:13:11.065Z]  GOROOT/src/testing/testing.go:1439 +0x102
[2022-07-27T11:13:11.066Z] testing.runTests(0xc000f5a000?, {0xc000f24de0, 0x2, 0x2}, {0x203000?, 0x203000?, 0x0?})
[2022-07-27T11:13:11.066Z]  GOROOT/src/testing/testing.go:1837 +0x457
[2022-07-27T11:13:11.066Z] testing.(*M).Run(0xc000f5a000)
[2022-07-27T11:13:11.066Z]  GOROOT/src/testing/testing.go:1719 +0x5d9
[2022-07-27T11:13:11.066Z] go.uber.org/goleak.VerifyTestMain({0x45a2a00?, 0xc000f5a000?}, {0xc0003e9e28, 0x5, 0x5})
[2022-07-27T11:13:11.066Z]  external/org_uber_go_goleak/testmain.go:53 +0x3d
[2022-07-27T11:13:11.066Z] privilege/privileges/privileges_test_test.TestMain(0x3b63d40?)
[2022-07-27T11:13:11.066Z]  privilege/privileges/main_test.go:38 +0x345
[2022-07-27T11:13:11.066Z] main.main()
[2022-07-27T11:13:11.066Z]  bazel-out/k8-fastbuild/bin/privilege/privileges/privileges_test_/testmain.go:286 +0x495
[2022-07-27T11:13:11.066Z] 
[2022-07-27T11:13:11.066Z] goroutine 90 [select]:
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/ddl.(*ddl).limitDDLJobs(0xc0009dc480)
[2022-07-27T11:13:11.066Z]  ddl/ddl_worker.go:278 +0x167
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/util.(*WaitGroupWrapper).Run.func1()
[2022-07-27T11:13:11.066Z]  util/wait_group_wrapper.go:33 +0x6b
[2022-07-27T11:13:11.066Z] created by github.com/pingcap/tidb/util.(*WaitGroupWrapper).Run
[2022-07-27T11:13:11.066Z]  util/wait_group_wrapper.go:31 +0x8d
[2022-07-27T11:13:11.066Z] 
[2022-07-27T11:13:11.066Z] goroutine 54 [select]:
[2022-07-27T11:13:11.066Z] go.opencensus.io/stats/view.(*worker).start(0xc0006f0700)
[2022-07-27T11:13:11.066Z]  external/io_opencensus_go/stats/view/worker.go:276 +0xad
[2022-07-27T11:13:11.066Z] created by go.opencensus.io/stats/view.init.0
[2022-07-27T11:13:11.066Z]  external/io_opencensus_go/stats/view/worker.go:34 +0x8d
[2022-07-27T11:13:11.066Z] 
[2022-07-27T11:13:11.066Z] goroutine 55 [chan receive]:
[2022-07-27T11:13:11.066Z] github.com/golang/glog.(*loggingT).flushDaemon(0x0?)
[2022-07-27T11:13:11.066Z]  external/com_github_golang_glog/glog.go:882 +0x6a
[2022-07-27T11:13:11.066Z] created by github.com/golang/glog.init.0
[2022-07-27T11:13:11.066Z]  external/com_github_golang_glog/glog.go:410 +0x1bf
[2022-07-27T11:13:11.066Z] 
[2022-07-27T11:13:11.066Z] goroutine 98 [select]:
[2022-07-27T11:13:11.066Z] github.com/pingcap/badger.Open.func4()
[2022-07-27T11:13:11.066Z]  external/com_github_pingcap_badger/db.go:342 +0x155
[2022-07-27T11:13:11.066Z] created by github.com/pingcap/badger.Open
[2022-07-27T11:13:11.066Z]  external/com_github_pingcap_badger/db.go:339 +0x11f8
[2022-07-27T11:13:11.066Z] 
[2022-07-27T11:13:11.066Z] goroutine 89 [semacquire]:
[2022-07-27T11:13:11.066Z] sync.runtime_Semacquire(0xc000fecc88?)
[2022-07-27T11:13:11.066Z]  GOROOT/src/runtime/sema.go:56 +0x25
[2022-07-27T11:13:11.066Z] sync.(*WaitGroup).Wait(0x44?)
[2022-07-27T11:13:11.066Z]  GOROOT/src/sync/waitgroup.go:136 +0x52
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*dbWriter).Write(0xc000f129c0, {0x45c54a0?, 0xc000eb9500})
[2022-07-27T11:13:11.066Z]  store/mockstore/unistore/tikv/write.go:222 +0x95
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*MVCCStore).Commit(0xc000eb8180, 0xc000f5bb80, {0xc000b0dec0?, 0x1, 0x1}, 0x608fd70b1540001, 0x608fd70b1580000)
[2022-07-27T11:13:11.066Z]  store/mockstore/unistore/tikv/mvcc.go:1002 +0x2e8
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*Server).KvCommit(0xc000f12a40, {0x399db1e?, 0x6?}, 0xc000cd59a0)
[2022-07-27T11:13:11.066Z]  store/mockstore/unistore/tikv/server.go:363 +0x18e
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/store/mockstore/unistore.(*RPCClient).SendRequest(0xc000d60870, {0x45bf518, 0xc000c76c30}, {0xc000eb00a0, 0x6}, 0xc00078ca00, 0xc000c76b10?)
[2022-07-27T11:13:11.066Z]  store/mockstore/unistore/rpc.go:183 +0xfa9
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/store/mockstore.(*clientRedirector).SendRequest(0x0?, {0x45bf518?, 0xc000c76c30?}, {0xc000eb00a0?, 0x0?}, 0x0?, 0x0?)
[2022-07-27T11:13:11.066Z]  store/mockstore/redirector.go:72 +0x107
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/internal/client.interceptedClient.SendRequest({{0x45b84c0?, 0xc000d60ed0?}}, {0x45bf518, 0xc000c76c30}, {0xc000eb00a0, 0x6}, 0x0?, 0x6fc23ac00)
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/internal/client/client_interceptor.go:42 +0x186
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/internal/client.reqCollapse.SendRequest({{0x45bddd0?, 0xc000eae690?}}, {0x45bf518, 0xc000c76c30}, {0xc000eb00a0, 0x6}, 0xc000fed788?, 0x15e9a17?)
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/internal/client/client_collapse.go:74 +0xd6
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/internal/locate.(*RegionRequestSender).sendReqToRegion(0xc000fee6b8, 0xc000fcba70, 0xc000c5aa10, 0xc00078ca00, 0xc000fedef8?)
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/internal/locate/region_request.go:1184 +0x86f
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/internal/locate.(*RegionRequestSender).SendReqCtx(0xc000fee6b8, 0xc000fcba70, 0xc00078ca00, {0x2, 0x1, 0x1}, 0x6fc23ac00, 0x0, {0x0, 0x0, ...})
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/internal/locate/region_request.go:1017 +0x10f4
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/internal/locate.(*RegionRequestSender).SendReq(...)
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/internal/locate/region_request.go:233
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/txnkv/transaction.actionCommit.handleSingleBatch({0xc5}, 0xc000193080, 0xc000fcba70, {{0x2, 0x1, 0x1}, {0x45d7f80, 0xc000b15240}, 0x1})
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/txnkv/transaction/commit.go:97 +0x7cd
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/txnkv/transaction.(*twoPhaseCommitter).doActionOnBatches(0xc000193080, 0xc000fcba70, {0x45bde60?, 0xc000b08738}, {0xc000c76d50?, 0x1, 0xc000feec08?})
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/txnkv/transaction/2pc.go:988 +0x329
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/txnkv/transaction.(*twoPhaseCommitter).doActionOnGroupMutations(0xc000193080, 0xc000fcba70, {0x45bde60, 0xc000b08738?}, {0xc000c76d20, 0x1, 0x0?})
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/txnkv/transaction/2pc.go:921 +0x895
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/txnkv/transaction.(*twoPhaseCommitter).doActionOnMutations(0xc000193080, 0xc000c76c30?, {0x45bde60?, 0xc000b08738}, {0x45d7f80, 0xc000b15160})
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/txnkv/transaction/2pc.go:745 +0xc7
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/txnkv/transaction.(*twoPhaseCommitter).commitMutations(0x1519852?, 0xc000fcba70, {0x45d7f80, 0xc000b15160})
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/txnkv/transaction/commit.go:221 +0x22a
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/txnkv/transaction.(*twoPhaseCommitter).commitTxn(0xc000193080, {0x45bf518?, 0xc000c76c30}, 0xc0005d7500)
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/txnkv/transaction/2pc.go:1610 +0x1bc
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/txnkv/transaction.(*twoPhaseCommitter).execute(0xc000193080, {0x45bf518?, 0xc000c76c30})
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/txnkv/transaction/2pc.go:1601 +0x2835
[2022-07-27T11:13:11.066Z] github.com/tikv/client-go/v2/txnkv/transaction.(*KVTxn).Commit(0xc000cbdcc0, {0x45bf518, 0xc000c76b10?})
[2022-07-27T11:13:11.066Z]  external/com_github_tikv_client_go_v2/txnkv/transaction/txn.go:448 +0xdcf
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/store/driver/txn.(*tikvTxn).Commit(0xc000c76b70, {0x45bf518?, 0xc000c76b10?})
[2022-07-27T11:13:11.066Z]  store/driver/txn/txn_driver.go:80 +0x38
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/kv.RunInNewTxn({0x45bf518, 0xc000c76b10}, {0x45df218, 0xc000f12ec0}, 0x1, 0xc000ff05c8)
[2022-07-27T11:13:11.066Z]  kv/txn.go:156 +0x79d
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/ddl.(*ddl).genGlobalIDs(0xc0009dc480, 0x1)
[2022-07-27T11:13:11.066Z]  ddl/ddl.go:754 +0xb3
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/ddl.(*ddl).CreateSchemaWithInfo(0xc000ff0700?, {0x4617d48, 0xc000db26c0}, 0xc000c5a7e0, 0x1)
[2022-07-27T11:13:11.066Z]  ddl/ddl_api.go:188 +0x1c5
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/ddl.(*ddl).CreateSchema(0x4617d48?, {0x4617d48, 0xc000db26c0}, 0xc000eb9200)
[2022-07-27T11:13:11.066Z]  ddl/ddl_api.go:153 +0x579
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/executor.(*DDLExec).executeCreateDatabase(0xc000f5a960, 0xc000c76990?)
[2022-07-27T11:13:11.066Z]  executor/ddl.go:236 +0x69
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/executor.(*DDLExec).Next(0xc000f5a960, {0x45bf518?, 0xc000c76960?}, 0x0?)
[2022-07-27T11:13:11.066Z]  executor/ddl.go:150 +0x3f1
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/executor.Next({0x45bf518, 0xc000c76960}, {0x45c25e0, 0xc000f5a960}, 0xc000cd5180)
[2022-07-27T11:13:11.066Z]  executor/executor.go:319 +0x55e
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/executor.(*ExecStmt).next(0xc0002b6780, {0x45bf518, 0xc000c76960}, {0x45c25e0, 0xc000f5a960}, 0x0?)
[2022-07-27T11:13:11.066Z]  executor/adapter.go:898 +0x7e
[2022-07-27T11:13:11.066Z] github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor(0xc0002b6780, {0x45bf518, 0xc000c76960}, {0x45c25e0?, 0xc000f5a960})
[2022-07-27T11:13:11.066Z]  executor/adapter.go:698 +0x611
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay(0xc0002b6780, {0x45bf518, 0xc000c76960}, {0x45c25e0?, 0xc000f5a960?}, 0x0)
[2022-07-27T11:13:11.067Z]  executor/adapter.go:548 +0x258
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/executor.(*ExecStmt).Exec(0xc0002b6780, {0x45bf518, 0xc000c76960})
[2022-07-27T11:13:11.067Z]  executor/adapter.go:497 +0xea5
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/session.runStmt({0x45bf518, 0xc000c76630}, 0xc000db26c0, {0x45ccb00, 0xc0002b6780?})
[2022-07-27T11:13:11.067Z]  session/session.go:2133 +0x6c9
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/session.(*session).ExecuteStmt(0xc000db26c0, {0x45bf518, 0xc000c76630}, {0x45d2668?, 0xc000eb9200})
[2022-07-27T11:13:11.067Z]  session/session.go:1999 +0xbb7
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/session.(*session).ExecuteInternal(0xc000db26c0, {0x45bf518, 0xc000c76630}, {0x405f70e, 0x22}, {0x0, 0x0, 0x0})
[2022-07-27T11:13:11.067Z]  session/session.go:1503 +0x42b
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/session.mustExecute({0x461d1f0, 0xc000db26c0}, {0x405f70e, 0x22}, {0x0, 0x0, 0x0})
[2022-07-27T11:13:11.067Z]  session/bootstrap.go:2110 +0xdc
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/session.doDDLWorks({0x461d1f0, 0xc000db26c0})
[2022-07-27T11:13:11.067Z]  session/bootstrap.go:1932 +0x45
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/session.bootstrap({0x461d1f0?, 0xc000db26c0?})
[2022-07-27T11:13:11.067Z]  session/bootstrap.go:446 +0x2f3
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/session.runInBootstrapSession({0x45df218, 0xc000f12ec0}, 0x413a338)
[2022-07-27T11:13:11.067Z]  session/session.go:2794 +0x22f
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/session.BootstrapSession({0x45df218, 0xc000f12ec0})
[2022-07-27T11:13:11.067Z]  session/session.go:2679 +0x2b7
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/testkit.bootstrap({0x45e4128?, 0xc000f58340}, {0x45df218?, 0xc000f12ec0}, 0x0?)
[2022-07-27T11:13:11.067Z]  testkit/mockstore.go:71 +0x85
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/testkit.CreateMockStoreAndDomain({0x45e4128?, 0xc000f58340}, {0x0?, 0x0?, 0x0?})
[2022-07-27T11:13:11.067Z]  testkit/mockstore.go:64 +0xa5
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/testkit.CreateMockStore({0x45e4128?, 0xc000f58340}, {0x0, 0x0, 0x0})
[2022-07-27T11:13:11.067Z]  testkit/mockstore.go:56 +0x2be
[2022-07-27T11:13:11.067Z] privilege/privileges/privileges_test_test.createStoreAndPrepareDB(0xc000eb03b8?)
[2022-07-27T11:13:11.067Z]  privilege/privileges/privileges_test.go:2311 +0x33
[2022-07-27T11:13:11.067Z] privilege/privileges/privileges_test_test.TestUseDB(0x151cc19?)
[2022-07-27T11:13:11.067Z]  privilege/privileges/privileges_test.go:1039 +0x47
[2022-07-27T11:13:11.067Z] testing.tRunner(0xc000f58340, 0x413d998)
[2022-07-27T11:13:11.067Z]  GOROOT/src/testing/testing.go:1439 +0x102
[2022-07-27T11:13:11.067Z] created by testing.(*T).Run
[2022-07-27T11:13:11.067Z]  GOROOT/src/testing/testing.go:1486 +0x35f
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 58 [select]:
[2022-07-27T11:13:11.067Z] github.com/pingcap/badger.(*DB).updateSize(0x0?, 0xc000463398)
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/db.go:1039 +0x105
[2022-07-27T11:13:11.067Z] created by github.com/pingcap/badger.Open
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/db.go:328 +0xf5c
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 59 [select]:
[2022-07-27T11:13:11.067Z] github.com/pingcap/badger/epoch.(*ResourceManager).collectLoop(0x0?, 0xc0004633c8)
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/epoch/manager.go:101 +0xd3
[2022-07-27T11:13:11.067Z] created by github.com/pingcap/badger/epoch.NewResourceManager
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/epoch/manager.go:79 +0xda
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 60 [select]:
[2022-07-27T11:13:11.067Z] github.com/pingcap/badger.(*blobGCHandler).run(0xc000f128c0, 0xc00000e4c8)
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/blob.go:468 +0xd6
[2022-07-27T11:13:11.067Z] created by github.com/pingcap/badger.(*blobManager).Open
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/blob.go:292 +0x5b0
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 61 [select]:
[2022-07-27T11:13:11.067Z] github.com/pingcap/badger.(*levelsController).runWorker(0xc000f82000, 0xc00000e558, 0x0)
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/levels.go:212 +0x1fb
[2022-07-27T11:13:11.067Z] created by github.com/pingcap/badger.(*levelsController).startCompact
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/levels.go:180 +0x65
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 62 [chan receive]:
[2022-07-27T11:13:11.067Z] github.com/pingcap/badger.(*DB).runFlushMemTable(0xc000d5e900, 0x0?)
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/db.go:920 +0xdd
[2022-07-27T11:13:11.067Z] created by github.com/pingcap/badger.Open
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/db.go:361 +0x158b
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 116 [chan receive]:
[2022-07-27T11:13:11.067Z] github.com/pingcap/badger.(*writeWorker).runMergeLSM(0xc000cc81a0?, 0xc00000e618?)
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/writer.go:154 +0x9b
[2022-07-27T11:13:11.067Z] created by github.com/pingcap/badger.startWriteWorker
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/writer.go:66 +0x2a5
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 115 [chan receive, locked to thread]:
[2022-07-27T11:13:11.067Z] github.com/pingcap/badger.(*writeWorker).runWriteLSM(0xc000cc8260, 0x413d998?)
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/writer.go:141 +0x15e
[2022-07-27T11:13:11.067Z] created by github.com/pingcap/badger.startWriteWorker
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/writer.go:65 +0x24d
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 114 [select]:
[2022-07-27T11:13:11.067Z] github.com/pingcap/badger.(*writeWorker).runWriteVLog(0xc000cc8260, 0xc00000e690)
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/writer.go:94 +0x137
[2022-07-27T11:13:11.067Z] created by github.com/pingcap/badger.startWriteWorker
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/writer.go:64 +0x1f9
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 117 [runnable]:
[2022-07-27T11:13:11.067Z] sync.runtime_Semacquire(0xc000b23c88?)
[2022-07-27T11:13:11.067Z]  GOROOT/src/runtime/sema.go:56 +0x25
[2022-07-27T11:13:11.067Z] sync.(*WaitGroup).Wait(0x5e7fbe0?)
[2022-07-27T11:13:11.067Z]  GOROOT/src/sync/waitgroup.go:136 +0x52
[2022-07-27T11:13:11.067Z] github.com/pingcap/badger.(*request).Wait(0xc000da4d00)
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/value.go:495 +0x2b
[2022-07-27T11:13:11.067Z] github.com/pingcap/badger.(*Txn).Commit(0xc000fcbb00)
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/transaction.go:517 +0x665
[2022-07-27T11:13:11.067Z] github.com/pingcap/badger.(*DB).Update(0xc0000346c0?, 0xc000b23eb0)
[2022-07-27T11:13:11.067Z]  external/com_github_pingcap_badger/transaction.go:596 +0xaf
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/store/mockstore/unistore/tikv.writeDBWorker.updateBatchGroup({0xc000cdd800?, 0xc000f129c0?}, {0xc0005da348?, 0x1, 0x0?})
[2022-07-27T11:13:11.067Z]  store/mockstore/unistore/tikv/write.go:113 +0x72
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/store/mockstore/unistore/tikv.writeDBWorker.run({0xc000cdd800?, 0xc000f129c0?})
[2022-07-27T11:13:11.067Z]  store/mockstore/unistore/tikv/write.go:107 +0x225
[2022-07-27T11:13:11.067Z] created by github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*dbWriter).Open
[2022-07-27T11:13:11.067Z]  store/mockstore/unistore/tikv/write.go:199 +0xca
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 118 [select]:
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/store/mockstore/unistore/tikv.writeLockWorker.run({0xc000cdd860?, 0xc000f129c0?})
[2022-07-27T11:13:11.067Z]  store/mockstore/unistore/tikv/write.go:148 +0x168
[2022-07-27T11:13:11.067Z] created by github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*dbWriter).Open
[2022-07-27T11:13:11.067Z]  store/mockstore/unistore/tikv/write.go:206 +0x159
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 119 [select]:
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*MVCCStore).runUpdateSafePointLoop(0xc000eb8180)
[2022-07-27T11:13:11.067Z]  store/mockstore/unistore/tikv/mvcc.go:1706 +0x252
[2022-07-27T11:13:11.067Z] created by github.com/pingcap/tidb/store/mockstore/unistore/tikv.NewMVCCStore
[2022-07-27T11:13:11.067Z]  store/mockstore/unistore/tikv/mvcc.go:87 +0x34c
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 120 [select]:
[2022-07-27T11:13:11.067Z] github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*MVCCStore).StartDeadlockDetection.func1()
[2022-07-27T11:13:11.067Z]  store/mockstore/unistore/tikv/mvcc.go:1379 +0x8f
[2022-07-27T11:13:11.067Z] created by github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*MVCCStore).StartDeadlockDetection
[2022-07-27T11:13:11.067Z]  store/mockstore/unistore/tikv/mvcc.go:1377 +0xcd
[2022-07-27T11:13:11.067Z] 
[2022-07-27T11:13:11.067Z] goroutine 121 [select]:
[2022-07-27T11:13:11.068Z] github.com/tikv/client-go/v2/oracle/oracles.(*pdOracle).updateTS(0xc000edb030, {0x45bf4a8, 0xc00005a0a8}, 0xc00000e618?)
[2022-07-27T11:13:11.068Z]  external/com_github_tikv_client_go_v2/oracle/oracles/pd.go:229 +0x125
[2022-07-27T11:13:11.068Z] created by github.com/tikv/client-go/v2/oracle/oracles.NewPdOracle
[2022-07-27T11:13:11.068Z]  external/com_github_tikv_client_go_v2/oracle/oracles/pd.go:77 +0x119
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 122 [select]:
[2022-07-27T11:13:11.068Z] github.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop(0xc000eb6f70, 0x413d998?)
[2022-07-27T11:13:11.068Z]  external/com_github_tikv_client_go_v2/internal/locate/region_cache.go:449 +0xea
[2022-07-27T11:13:11.068Z] created by github.com/tikv/client-go/v2/internal/locate.NewRegionCache
[2022-07-27T11:13:11.068Z]  external/com_github_tikv_client_go_v2/internal/locate/region_cache.go:420 +0x2ba
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 123 [select]:
[2022-07-27T11:13:11.068Z] github.com/tikv/client-go/v2/tikv.(*KVStore).runSafePointChecker(0xc000cbc000)
[2022-07-27T11:13:11.068Z]  external/com_github_tikv_client_go_v2/tikv/kv.go:234 +0x12b
[2022-07-27T11:13:11.068Z] created by github.com/tikv/client-go/v2/tikv.NewKVStore
[2022-07-27T11:13:11.068Z]  external/com_github_tikv_client_go_v2/tikv/kv.go:189 +0x40a
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 124 [select]:
[2022-07-27T11:13:11.068Z] github.com/tikv/client-go/v2/tikv.(*KVStore).safeTSUpdater(0xc000cbc000)
[2022-07-27T11:13:11.068Z]  external/com_github_tikv_client_go_v2/tikv/kv.go:530 +0x1ed
[2022-07-27T11:13:11.068Z] created by github.com/tikv/client-go/v2/tikv.NewKVStore
[2022-07-27T11:13:11.068Z]  external/com_github_tikv_client_go_v2/tikv/kv.go:190 +0x44a
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 125 [select]:
[2022-07-27T11:13:11.068Z] github.com/dgraph-io/ristretto.(*lfuPolicy).processItems(0xc000f12e80)
[2022-07-27T11:13:11.068Z]  external/com_github_dgraph_io_ristretto/policy.go:67 +0x91
[2022-07-27T11:13:11.068Z] created by github.com/dgraph-io/ristretto.newPolicy
[2022-07-27T11:13:11.068Z]  external/com_github_dgraph_io_ristretto/policy.go:51 +0x156
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 126 [select]:
[2022-07-27T11:13:11.068Z] github.com/dgraph-io/ristretto.(*Cache).processItems(0xc000edb110)
[2022-07-27T11:13:11.068Z]  external/com_github_dgraph_io_ristretto/cache.go:476 +0x18a
[2022-07-27T11:13:11.068Z] created by github.com/dgraph-io/ristretto.NewCache
[2022-07-27T11:13:11.068Z]  external/com_github_dgraph_io_ristretto/cache.go:213 +0x756
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 91 [select]:
[2022-07-27T11:13:11.068Z] github.com/pingcap/tidb/ddl.(*delRange).startEmulator(0xc000fb82a0)
[2022-07-27T11:13:11.068Z]  ddl/delete_range.go:160 +0x145
[2022-07-27T11:13:11.068Z] created by github.com/pingcap/tidb/ddl.(*delRange).start
[2022-07-27T11:13:11.068Z]  ddl/delete_range.go:142 +0x8a
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 92 [select]:
[2022-07-27T11:13:11.068Z] github.com/pingcap/tidb/ddl.(*ddl).startDispatchLoop(0xc0009dc480)
[2022-07-27T11:13:11.068Z]  ddl/job_table.go:175 +0x52a
[2022-07-27T11:13:11.068Z] github.com/pingcap/tidb/util.(*WaitGroupWrapper).Run.func1()
[2022-07-27T11:13:11.068Z]  util/wait_group_wrapper.go:33 +0x6b
[2022-07-27T11:13:11.068Z] created by github.com/pingcap/tidb/util.(*WaitGroupWrapper).Run
[2022-07-27T11:13:11.068Z]  util/wait_group_wrapper.go:31 +0x8d
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 93 [select]:
[2022-07-27T11:13:11.068Z] github.com/pingcap/tidb/ddl.(*worker).start(0xc000eb8b00, 0xc00042e600)
[2022-07-27T11:13:11.068Z]  ddl/ddl_worker.go:194 +0x365
[2022-07-27T11:13:11.068Z] created by github.com/pingcap/tidb/ddl.(*ddl).prepareWorkers4legacyDDL
[2022-07-27T11:13:11.068Z]  ddl/ddl.go:633 +0x26d
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 94 [select]:
[2022-07-27T11:13:11.068Z] github.com/pingcap/tidb/ddl.(*worker).start(0xc000eb8b80, 0xc00042e600)
[2022-07-27T11:13:11.068Z]  ddl/ddl_worker.go:194 +0x365
[2022-07-27T11:13:11.068Z] created by github.com/pingcap/tidb/ddl.(*ddl).prepareWorkers4legacyDDL
[2022-07-27T11:13:11.068Z]  ddl/ddl.go:633 +0x26d
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 96 [select]:
[2022-07-27T11:13:11.068Z] github.com/pingcap/tidb/ddl.(*ddl).PollTiFlashRoutine(0xc0009dc480)
[2022-07-27T11:13:11.068Z]  ddl/ddl_tiflash_api.go:576 +0x2aa
[2022-07-27T11:13:11.068Z] github.com/pingcap/tidb/util.(*WaitGroupWrapper).Run.func1()
[2022-07-27T11:13:11.068Z]  util/wait_group_wrapper.go:33 +0x6b
[2022-07-27T11:13:11.068Z] created by github.com/pingcap/tidb/util.(*WaitGroupWrapper).Run
[2022-07-27T11:13:11.068Z]  util/wait_group_wrapper.go:31 +0x8d
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 97 [select]:
[2022-07-27T11:13:11.068Z] github.com/pingcap/tidb/domain.(*Domain).topNSlowQueryLoop(0xc000517040)
[2022-07-27T11:13:11.068Z]  domain/domain.go:504 +0x188
[2022-07-27T11:13:11.068Z] created by github.com/pingcap/tidb/domain.(*Domain).Init
[2022-07-27T11:13:11.068Z]  domain/domain.go:890 +0x10dd
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 146 [select]:
[2022-07-27T11:13:11.068Z] github.com/pingcap/tidb/domain.(*Domain).infoSyncerKeeper(0xc000517040)
[2022-07-27T11:13:11.068Z]  domain/domain.go:536 +0x19e
[2022-07-27T11:13:11.068Z] created by github.com/pingcap/tidb/domain.(*Domain).Init
[2022-07-27T11:13:11.068Z]  domain/domain.go:891 +0x1125
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 147 [select]:
[2022-07-27T11:13:11.068Z] github.com/pingcap/tidb/domain.(*Domain).globalConfigSyncerKeeper(0xc000517040)
[2022-07-27T11:13:11.068Z]  domain/domain.go:559 +0x113
[2022-07-27T11:13:11.068Z] created by github.com/pingcap/tidb/domain.(*Domain).Init
[2022-07-27T11:13:11.068Z]  domain/domain.go:892 +0x116a
[2022-07-27T11:13:11.068Z] 
[2022-07-27T11:13:11.068Z] goroutine 148 [select]:
[2022-07-27T11:13:11.068Z] github.com/pingcap/tidb/domain.(*Domain).topologySyncerKeeper(0xc000517040)
[2022-07-27T11:13:11.068Z]  domain/domain.go:582 +0x198
[2022-07-27T11:13:11.068Z] created by github.com/pingcap/tidb/domain.(*Domain).Init
[2022-07-27T11:13:11.068Z]  domain/domain.go:895 +0x11d5

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

tiancaiamao commented 2 years ago

Lock and panic, without releasing the lock?

hawkingrei commented 2 years ago

Lock and panic, without releasing the lock?

Maybe. It is so terrible.

tiancaiamao commented 2 years ago

There is a panic(maybe from the user side), and when handling the panic, the Go runtime found some wrong and panic again https://github.com/golang/go/blob/f32519e5fbcf1b12f9654a6175e5e72b09ae8f3a/src/runtime/lock_futex.go#L127

This process repeat again and again, you can see the final stack.

I have no idea how the first panic is triggered. And I can't reproduce the bug:

cd privilege/privileges;
go test -run TestUseDB
hawkingrei commented 2 years ago

@tiancaiamao it happens again #36934

Defined2014 commented 1 year ago

close this, it's a golang runtime bug.

ref https://github.com/pingcap/tidb/pull/38805