search

pingcap / tidb

TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
https://pingcap.com
Apache License 2.0
36.62k stars 5.77k forks source link

flaky test TestCTEIterationMemTracker #54379

Closed hawkingrei closed 1 week ago

hawkingrei commented 2 weeks ago

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

    testkit.go:274: 
            Error Trace:    pkg/testkit/testkit.go:274
                                        pkg/testkit/testkit.go:199
                                        pkg/testkit/testkit.go:174
                                        pkg/executor/cte_test.go:227
            Error:          Received unexpected error:
                            assert row container spill disk failed
                            github.com/pingcap/tidb/pkg/util.GetRecoverError
                                pkg/util/util.go:305
                            github.com/pingcap/tidb/pkg/executor.(*cteProducer).computeRecursivePart.func1
                                pkg/executor/cte.go:416
                            runtime.gopanic
                                GOROOT/src/runtime/panic.go:914
                            github.com/pingcap/tidb/pkg/executor.(*cteProducer).computeRecursivePart
                                pkg/executor/cte.go:450
                            github.com/pingcap/tidb/pkg/executor.(*cteProducer).produce
                                pkg/executor/cte.go:366
                            github.com/pingcap/tidb/pkg/executor.(*CTEExec).Next
                                pkg/executor/cte.go:113
                            github.com/pingcap/tidb/pkg/executor/internal/exec.Next
                                pkg/executor/internal/exec/executor.go:410
                            github.com/pingcap/tidb/pkg/executor.(*ExplainExec).executeAnalyzeExec
                                pkg/executor/explain.go:130
                            github.com/pingcap/tidb/pkg/executor.(*ExplainExec).generateExplainInfo
                                pkg/executor/explain.go:149
                            github.com/pingcap/tidb/pkg/executor.(*ExplainExec).Next
                                pkg/executor/explain.go:76
                            github.com/pingcap/tidb/pkg/executor/internal/exec.Next
                                pkg/executor/internal/exec/executor.go:410
                            github.com/pingcap/tidb/pkg/executor.(*ExecStmt).next
                                pkg/executor/adapter.go:1250
                            github.com/pingcap/tidb/pkg/executor.(*recordSet).Next
                                pkg/executor/adapter.go:175
                            github.com/pingcap/tidb/pkg/session.GetRows4Test
                                pkg/session/tidb.go:346
                            github.com/pingcap/tidb/pkg/session.ResultSetToStringSlice
                                pkg/session/tidb.go:364
                            github.com/pingcap/tidb/pkg/testkit.(*TestKit).ResultSetToResultWithCtx
                                pkg/testkit/testkit.go:273
                            github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQueryWithContext
                                pkg/testkit/testkit.go:199
                            github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQuery
                                pkg/testkit/testkit.go:174
                            github.com/pingcap/tidb/pkg/executor_test.TestCTEIterationMemTracker
                                pkg/executor/cte_test.go:227
                            testing.tRunner
                                GOROOT/src/testing/testing.go:1595
                            runtime.goexit
                                src/runtime/asm_amd64.s:1650
            Test:           TestCTEIterationMemTracker
            Messages:       sql:explain analyze with recursive cte1 as (select c1 from t1 union all select c1 + 1 c1 from cte1 where c1 < 5000) select * from cte1, args:[]

4. What is your TiDB version? (Required)

lance6716 commented 2 weeks ago

not fixed

https://do.pingcap.net/jenkins/blue/organizations/jenkins/pingcap%2Ftidb%2Fghpr_unit_test/detail/ghpr_unit_test/12842/pipeline

    testkit.go:274: 
            Error Trace:    pkg/testkit/testkit.go:274
                                        pkg/testkit/testkit.go:199
                                        pkg/testkit/testkit.go:174
                                        pkg/executor/test/cte/cte_test.go:227
            Error:          Received unexpected error:
                            assert row container spill disk failed
                            github.com/pingcap/tidb/pkg/util.GetRecoverError
                                pkg/util/util.go:305
                            github.com/pingcap/tidb/pkg/executor.(*cteProducer).computeRecursivePart.func1
                                pkg/executor/cte.go:417
                            runtime.gopanic
                                GOROOT/src/runtime/panic.go:914
                            github.com/pingcap/tidb/pkg/executor.(*cteProducer).computeRecursivePart
                                pkg/executor/cte.go:449
                            github.com/pingcap/tidb/pkg/executor.(*cteProducer).produce
                                pkg/executor/cte.go:367
                            github.com/pingcap/tidb/pkg/executor.(*CTEExec).Next
                                pkg/executor/cte.go:114
                            github.com/pingcap/tidb/pkg/executor/internal/exec.Next
                                pkg/executor/internal/exec/executor.go:410
                            github.com/pingcap/tidb/pkg/executor.(*ExplainExec).executeAnalyzeExec
                                pkg/executor/explain.go:130
                            github.com/pingcap/tidb/pkg/executor.(*ExplainExec).generateExplainInfo
                                pkg/executor/explain.go:149
                            github.com/pingcap/tidb/pkg/executor.(*ExplainExec).Next
                                pkg/executor/explain.go:76
                            github.com/pingcap/tidb/pkg/executor/internal/exec.Next
                                pkg/executor/internal/exec/executor.go:410
                            github.com/pingcap/tidb/pkg/executor.(*ExecStmt).next
                                pkg/executor/adapter.go:1241
                            github.com/pingcap/tidb/pkg/executor.(*recordSet).Next
                                pkg/executor/adapter.go:169
                            github.com/pingcap/tidb/pkg/session.GetRows4Test
                                pkg/session/tidb.go:346
                            github.com/pingcap/tidb/pkg/session.ResultSetToStringSlice
                                pkg/session/tidb.go:364
                            github.com/pingcap/tidb/pkg/testkit.(*TestKit).ResultSetToResultWithCtx
                                pkg/testkit/testkit.go:273
                            github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQueryWithContext
                                pkg/testkit/testkit.go:199
                            github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQuery
                                pkg/testkit/testkit.go:174
                            pkg/executor/test/cte/cte_test.TestCTEIterationMemTracker
                                pkg/executor/test/cte/cte_test.go:227
                            testing.tRunner
                                GOROOT/src/testing/testing.go:1595
                            runtime.goexit
                                src/runtime/asm_amd64.s:1650
            Test:           TestCTEIterationMemTracker
            Messages:       sql:explain analyze with recursive cte1 as (select c1 from t1 union all select c1 + 1 c1 from cte1 where c1 < 5000) select * from cte1, args:[]
Defined2014 commented 2 weeks ago

Another link https://do.pingcap.net/jenkins/blue/organizations/jenkins/pingcap%2Ftidb%2Fghpr_unit_test/detail/ghpr_unit_test/12872/pipeline

Defined2014 commented 2 weeks ago 
[2024/07/08 08:09:42.696 +00:00] [INFO] [row_container.go:443] ["memory exceeds quota, spill to disk now."] [consumed=380] [quota=128]
[2024/07/08 08:09:42.697 +00:00] [INFO] [cte.go:765] ["cte iteration info"] ["iterInTbl mem usage"=0] ["iterInTbl disk usage"=24000] ["iterOutTbl mem usage"=8245] ["iterOutTbl disk usage"=0] ["resTbl mem usage"=0] ["resTbl disk usage"=16800000] ["resTbl rows"=700000] ["iteration num"=700] []
    testkit.go:274: 
            Error Trace:    pkg/testkit/testkit.go:274

                                        pkg/testkit/testkit.go:199

                                        pkg/testkit/testkit.go:174

                                        pkg/executor/test/cte/cte_test.go:227

            Error:          Received unexpected error:

                            assert row container spill disk failed

                            github.com/pingcap/tidb/pkg/util.GetRecoverError

                                pkg/util/util.go:305

                            github.com/pingcap/tidb/pkg/executor.(*cteProducer).computeRecursivePart.func1

                                pkg/executor/cte.go:417

                            runtime.gopanic

                                GOROOT/src/runtime/panic.go:914

                            github.com/pingcap/tidb/pkg/executor.(*cteProducer).computeRecursivePart

                                pkg/executor/cte.go:449

                            github.com/pingcap/tidb/pkg/executor.(*cteProducer).produce

                                pkg/executor/cte.go:367

                            github.com/pingcap/tidb/pkg/executor.(*CTEExec).Next

                                pkg/executor/cte.go:114

                            github.com/pingcap/tidb/pkg/executor/internal/exec.Next

                                pkg/executor/internal/exec/executor.go:410

                            github.com/pingcap/tidb/pkg/executor.(*ExplainExec).executeAnalyzeExec

                                pkg/executor/explain.go:130

                            github.com/pingcap/tidb/pkg/executor.(*ExplainExec).generateExplainInfo

                                pkg/executor/explain.go:149

                            github.com/pingcap/tidb/pkg/executor.(*ExplainExec).Next

                                pkg/executor/explain.go:76

                            github.com/pingcap/tidb/pkg/executor/internal/exec.Next

                                pkg/executor/internal/exec/executor.go:410

                            github.com/pingcap/tidb/pkg/executor.(*ExecStmt).next

                                pkg/executor/adapter.go:1241

                            github.com/pingcap/tidb/pkg/executor.(*recordSet).Next

                                pkg/executor/adapter.go:169

                            github.com/pingcap/tidb/pkg/session.GetRows4Test

                                pkg/session/tidb.go:346

                            github.com/pingcap/tidb/pkg/session.ResultSetToStringSlice

                                pkg/session/tidb.go:364

                            github.com/pingcap/tidb/pkg/testkit.(*TestKit).ResultSetToResultWithCtx

                                pkg/testkit/testkit.go:273

                            github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQueryWithContext

                                pkg/testkit/testkit.go:199

                            github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQuery

                                pkg/testkit/testkit.go:174

                            pkg/executor/test/cte/cte_test.TestCTEIterationMemTracker

                                pkg/executor/test/cte/cte_test.go:227

                            testing.tRunner

                                GOROOT/src/testing/testing.go:1595

                            runtime.goexit

                                src/runtime/asm_amd64.s:1650

            Test:           TestCTEIterationMemTracker

            Messages:       sql:explain analyze with recursive cte1 as (select c1 from t1 union all select c1 + 1 c1 from cte1 where c1 < 5000) select * from cte1, args:[]
tiancaiamao commented 1 week ago

This fail seems quite frequent recently.