search

matrixorigin / matrixone

Hyperconverged cloud-edge native database
https://docs.matrixorigin.cn/en
Apache License 2.0
1.76k stars 274 forks source link

[Bug]: ut TestDedupSnapshot3 failed, data race #18899

Open sukki37 opened 5 hours ago

sukki37 commented 5 hours ago

Is there an existing issue for the same bug?

Branch Name

main

Commit ID

during one pr

Other Environment Information

- Hardware parameters:
- OS type:
- Others:

Actual Behavior

link: https://github.com/matrixorigin/matrixone/actions/runs/10774892448/job/30154935257?pr=18619

log: job-logs (1).txt.zip

024-09-15T01:27:25.3070992Z ==================
2024-09-15T01:27:25.3071252Z WARNING: DATA RACE
2024-09-15T01:27:25.3071563Z Read at 0x00c004dc5748 by goroutine 9768:
2024-09-15T01:27:25.3072251Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnimpl.(*txnImpl).GetStartTS()
2024-09-15T01:27:25.3072900Z       <autogenerated>:1 +0x84
2024-09-15T01:27:25.3073595Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnbase.(*TxnManager).MinTSForTest.func1()
2024-09-15T01:27:25.3074640Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/txn/txnbase/txnmgr.go:599 +0x87
2024-09-15T01:27:25.3075407Z   sync.(*Map).Range()
2024-09-15T01:27:25.3075918Z       /opt/hostedtoolcache/go/1.23.0/x64/src/sync/map.go:501 +0x1e5
2024-09-15T01:27:25.3076716Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnbase.(*TxnManager).MinTSForTest()
2024-09-15T01:27:25.3077721Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/txn/txnbase/txnmgr.go:597 +0x132
2024-09-15T01:27:25.3078611Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/testutil.InitTestDB.func1()
2024-09-15T01:27:25.3079576Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/testutil/engine.go:392 +0x52
2024-09-15T01:27:25.3080541Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/gc/v2.(*checkpointCleaner).checkExtras()
2024-09-15T01:27:25.3081582Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/gc/v2/checkpoint.go:953 +0x1de
2024-09-15T01:27:25.3082524Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/gc/v2.(*checkpointCleaner).Process()
2024-09-15T01:27:25.3083540Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/gc/v2/checkpoint.go:892 +0x34a
2024-09-15T01:27:25.3084448Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/gc/v2.(*DiskCleaner).process()
2024-09-15T01:27:25.3085428Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/gc/v2/diskcleaner.go:90 +0xe3
2024-09-15T01:27:25.3086402Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/gc/v2.(*DiskCleaner).process-fm()
2024-09-15T01:27:25.3087036Z       <autogenerated>:1 +0x51
2024-09-15T01:27:25.3087760Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/logstore/sm.(*safeQueue).Start.func1()
2024-09-15T01:27:25.3088888Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/logstore/sm/safeq.go:89 +0x3a1
2024-09-15T01:27:25.3089383Z 
2024-09-15T01:27:25.3089564Z Previous write at 0x00c004dc5748 by goroutine 9853:
2024-09-15T01:27:25.3090263Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnbase.(*TxnCtx).SetStartTS()
2024-09-15T01:27:25.3091239Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/txn/txnbase/txnctx.go:130 +0x84
2024-09-15T01:27:25.3092207Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnimpl.(*txnImpl).SetStartTS()
2024-09-15T01:27:25.3092855Z       <autogenerated>:1 +0x1f
2024-09-15T01:27:25.3093655Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/test.TestDedupSnapshot3.TestDedupSnapshot3.func1.func3()
2024-09-15T01:27:25.3094794Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/test/db_test.go:7721 +0x386
2024-09-15T01:27:25.3095497Z   github.com/panjf2000/ants/v2.(*goWorker).run.func1()
2024-09-15T01:27:25.3096223Z       /home/runner/go/pkg/mod/github.com/panjf2000/ants/v2@v2.7.4/worker.go:67 +0xdb
2024-09-15T01:27:25.3096663Z 
2024-09-15T01:27:25.3096792Z Goroutine 9768 (running) created at:
2024-09-15T01:27:25.3097413Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/logstore/sm.(*safeQueue).Start()
2024-09-15T01:27:25.3098367Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/logstore/sm/safeq.go:68 +0x1a4
2024-09-15T01:27:25.3099272Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db.Open.(*DiskCleaner).Start.func26()
2024-09-15T01:27:25.3100280Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/gc/v2/diskcleaner.go:96 +0x4a
2024-09-15T01:27:25.3100900Z   sync.(*Once).doSlow()
2024-09-15T01:27:25.3101397Z       /opt/hostedtoolcache/go/1.23.0/x64/src/sync/once.go:76 +0xe1
2024-09-15T01:27:25.3101850Z   sync.(*Once).Do()
2024-09-15T01:27:25.3102314Z       /opt/hostedtoolcache/go/1.23.0/x64/src/sync/once.go:67 +0x44
2024-09-15T01:27:25.3103037Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/gc/v2.(*DiskCleaner).Start()
2024-09-15T01:27:25.3104017Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/gc/v2/diskcleaner.go:95 +0x4c3c
2024-09-15T01:27:25.3104794Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db.Open()
2024-09-15T01:27:25.3105574Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/open.go:220 +0x4ba6
2024-09-15T01:27:25.3106366Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/testutil.InitTestDB()
2024-09-15T01:27:25.3107295Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/testutil/engine.go:388 +0x78
2024-09-15T01:27:25.3108158Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/testutil.NewTestEngine()
2024-09-15T01:27:25.3109091Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/testutil/engine.go:81 +0x67
2024-09-15T01:27:25.3109951Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/test.TestDedupSnapshot3()
2024-09-15T01:27:25.3110887Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/test/db_test.go:7689 +0x86
2024-09-15T01:27:25.3111474Z   testing.tRunner()
2024-09-15T01:27:25.3112009Z       /opt/hostedtoolcache/go/1.23.0/x64/src/testing/testing.go:1690 +0x226
2024-09-15T01:27:25.3112515Z   testing.(*T).Run.gowrap1()
2024-09-15T01:27:25.3113075Z       /opt/hostedtoolcache/go/1.23.0/x64/src/testing/testing.go:1743 +0x44
2024-09-15T01:27:25.3113449Z 
2024-09-15T01:27:25.3113583Z Goroutine 9853 (running) created at:
2024-09-15T01:27:25.3114004Z   github.com/panjf2000/ants/v2.(*goWorker).run()
2024-09-15T01:27:25.3114710Z       /home/runner/go/pkg/mod/github.com/panjf2000/ants/v2@v2.7.4/worker.go:48 +0xb1
2024-09-15T01:27:25.3115519Z   github.com/panjf2000/ants/v2.(*Pool).retrieveWorker.func1()
2024-09-15T01:27:25.3116273Z       /home/runner/go/pkg/mod/github.com/panjf2000/ants/v2@v2.7.4/pool.go:337 +0xa7
2024-09-15T01:27:25.3116484Z   github.com/panjf2000/ants/v2.(*Pool).retrieveWorker()
2024-09-15T01:27:25.3117151Z       /home/runner/go/pkg/mod/github.com/panjf2000/ants/v2@v2.7.4/pool.go:348 +0x198
2024-09-15T01:27:25.3117329Z   github.com/panjf2000/ants/v2.(*Pool).Submit()
2024-09-15T01:27:25.3117732Z       /home/runner/go/pkg/mod/github.com/panjf2000/ants/v2@v2.7.4/pool.go:222 +0x47
2024-09-15T01:27:25.3118110Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/test.TestDedupSnapshot3()
2024-09-15T01:27:25.3118595Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/test/db_test.go:7732 +0xa64
2024-09-15T01:27:25.3118772Z   testing.tRunner()
2024-09-15T01:27:25.3119141Z       /opt/hostedtoolcache/go/1.23.0/x64/src/testing/testing.go:1690 +0x226
2024-09-15T01:27:25.3119271Z   testing.(*T).Run.gowrap1()
2024-09-15T01:27:25.3119625Z       /opt/hostedtoolcache/go/1.23.0/x64/src/testing/testing.go:1743 +0x44
2024-09-15T01:27:25.3119721Z ==================

Expected Behavior

No response

Steps to Reproduce

run ut

Additional information

No response

XuPeng-SH commented 5 hours ago

it was fixed by #18861 and this ci fail was with very old main commit

image