search

pingcap / tidb

TiDB - the open-source, cloud-native, distributed SQL database designed for modern applications.
https://pingcap.com
Apache License 2.0
37.19k stars 5.84k forks source link

data race in `TestFastAnalyze` #25782

Closed zhouqiang-cl closed 2 years ago

zhouqiang-cl commented 3 years ago

Bug Report

[2021-06-27T06:16:11.915Z] ==================
[2021-06-27T06:16:11.915Z] WARNING: DATA RACE
[2021-06-27T06:16:11.915Z] Write at 0x000007ac5880 by goroutine 724:
[2021-06-27T06:16:11.915Z]   github.com/pingcap/tidb/executor_test.(*testFastAnalyze).TestFastAnalyze()
[2021-06-27T06:16:11.915Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/executor/analyze_test.go:418 +0x3f1
[2021-06-27T06:16:11.915Z]   runtime.call16()
[2021-06-27T06:16:11.915Z]       /usr/local/go/src/runtime/asm_amd64.s:550 +0x3d
[2021-06-27T06:16:11.915Z]   reflect.Value.Call()
[2021-06-27T06:16:11.915Z]       /usr/local/go/src/reflect/value.go:337 +0xd8
[2021-06-27T06:16:11.915Z]   github.com/pingcap/check.(*suiteRunner).forkTest.func1()
[2021-06-27T06:16:11.915Z]       /nfs/cache/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:850 +0xb3b
[2021-06-27T06:16:11.915Z]   github.com/pingcap/check.(*suiteRunner).forkCall.func1()
[2021-06-27T06:16:11.915Z]       /nfs/cache/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:739 +0x11d
[2021-06-27T06:16:11.915Z] 
[2021-06-27T06:16:11.915Z] Previous read at 0x000007ac5880 by goroutine 908:
[2021-06-27T06:16:11.915Z]   github.com/pingcap/tidb/executor.(*AnalyzeFastExec).buildStats()
[2021-06-27T06:16:11.915Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/executor/analyze.go:2044 +0x54
[2021-06-27T06:16:11.916Z]   github.com/pingcap/tidb/executor.analyzeFastExec()
[2021-06-27T06:16:11.916Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/executor/analyze.go:1505 +0x55
[2021-06-27T06:16:11.916Z]   github.com/pingcap/tidb/executor.(*AnalyzeExec).analyzeWorker()
[2021-06-27T06:16:11.916Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/executor/analyze.go:272 +0x34c
[2021-06-27T06:16:11.916Z] 
[2021-06-27T06:16:11.916Z] Goroutine 724 (running) created at:
[2021-06-27T06:16:11.916Z]   github.com/pingcap/check.(*suiteRunner).forkCall()
[2021-06-27T06:16:11.916Z]       /nfs/cache/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:734 +0x4cc
[2021-06-27T06:16:11.916Z]   github.com/pingcap/check.(*suiteRunner).forkTest()
[2021-06-27T06:16:11.916Z]       /nfs/cache/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:832 +0x1c4
[2021-06-27T06:16:11.916Z]   github.com/pingcap/check.(*suiteRunner).doRun()
[2021-06-27T06:16:11.916Z]       /nfs/cache/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:666 +0x144
[2021-06-27T06:16:11.916Z]   github.com/pingcap/check.(*suiteRunner).asyncRun.func1()
[2021-06-27T06:16:11.916Z]       /nfs/cache/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:650 +0x105
[2021-06-27T06:16:11.916Z] 
[2021-06-27T06:16:11.916Z] Goroutine 908 (running) created at:
[2021-06-27T06:16:11.916Z]   github.com/pingcap/tidb/executor.(*AnalyzeExec).Next()
[2021-06-27T06:16:11.916Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/executor/analyze.go:94 +0x1c4
[2021-06-27T06:16:11.916Z]   github.com/pingcap/tidb/executor.Next()
[2021-06-27T06:16:11.916Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/executor/executor.go:286 +0x2a8
[2021-06-27T06:16:11.916Z]   github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor()
[2021-06-27T06:16:11.916Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/executor/adapter.go:589 +0x3bc
[2021-06-27T06:16:11.916Z]   github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay()
[2021-06-27T06:16:11.916Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/executor/adapter.go:470 +0x276
[2021-06-27T06:16:11.916Z]   github.com/pingcap/tidb/executor.(*ExecStmt).Exec()
[2021-06-27T06:16:11.916Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/executor/adapter.go:419 +0x63a
[2021-06-27T06:16:11.916Z]   github.com/pingcap/tidb/session.runStmt()
[2021-06-27T06:16:11.916Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/session/session.go:1635 +0x323
[2021-06-27T06:16:11.916Z]   github.com/pingcap/tidb/session.(*session).ExecuteStmt()
[2021-06-27T06:16:11.916Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/session/session.go:1530 +0xc37
[2021-06-27T06:16:11.916Z]   github.com/pingcap/tidb/util/testkit.(*TestKit).Exec()
[2021-06-27T06:16:11.916Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/util/testkit/testkit.go:170 +0x2fc
[2021-06-27T06:16:11.916Z]   github.com/pingcap/tidb/util/testkit.(*TestKit).MustExec()
[2021-06-27T06:16:11.916Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/util/testkit/testkit.go:216 +0x96
[2021-06-27T06:16:11.916Z]   github.com/pingcap/tidb/executor_test.(*testSuite1).TestAnalyzeParameters()
[2021-06-27T06:16:11.916Z]       /home/jenkins/agent/workspace/tidb-unit-test-nightly/go/src/github.com/pingcap/tidb/executor/analyze_test.go:181 +0x2e9
[2021-06-27T06:16:11.916Z]   runtime.call16()
[2021-06-27T06:16:11.916Z]       /usr/local/go/src/runtime/asm_amd64.s:550 +0x3d
[2021-06-27T06:16:11.916Z]   reflect.Value.Call()
[2021-06-27T06:16:11.916Z]       /usr/local/go/src/reflect/value.go:337 +0xd8
[2021-06-27T06:16:11.916Z]   github.com/pingcap/check.(*suiteRunner).forkTest.func1()
[2021-06-27T06:16:11.916Z]       /nfs/cache/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:850 +0xb3b
[2021-06-27T06:16:11.916Z]   github.com/pingcap/check.(*suiteRunner).forkCall.func1()
[2021-06-27T06:16:11.916Z]       /nfs/cache/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:739 +0x11d
[2021-06-27T06:16:11.916Z] ==================

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

in ci https://ci.pingcap.net/blue/rest/organizations/jenkins/pipelines/tidb-unit-test-nightly/runs/2527/nodes/104/steps/336/log/?start=0

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

master

lzmhhh123 commented 3 years ago

/cc @winoros

zhouqiang-cl commented 3 years ago

happend again https://ci.pingcap.net/blue/rest/organizations/jenkins/pipelines/tidb-unit-test-nightly/runs/5390/nodes/104/steps/398/log/?start=0

bb7133 commented 3 years ago

I believe the race is caused by:

https://github.com/pingcap/tidb/blob/f710f3d9a3f6b4bbaa63afb8761ccafd66ad6113/executor/analyze.go#L2042

and

https://github.com/pingcap/tidb/blob/e5359ed45b96847ed3b9de7eea04341a239123eb/executor/analyze_test.go#L419

Please try to fix it @lzmhhh123 @winoros

github-actions[bot] commented 2 years ago

Please check whether the issue should be labeled with 'affects-x.y' or 'fixes-x.y.z', and then remove 'needs-more-info' label.