search

matrixorigin / matrixone

Hyperconverged cloud-edge native database
https://docs.matrixorigin.cn/en
Apache License 2.0
1.79k stars 277 forks source link

[Bug]: TestTransferInMerge data race #18810

Closed reusee closed 1 month ago

reusee commented 2 months ago

Is there an existing issue for the same bug?

Branch Name

main

Commit ID

604e23210f543b6e22acf6e6597eabb46187c627

Other Environment Information

- Hardware parameters:
- OS type:
- Others:

Actual Behavior

2024-09-15T01:14:46.3335130Z ==================
2024-09-15T01:14:46.3335241Z WARNING: DATA RACE
2024-09-15T01:14:46.3335398Z Read at 0x00c0157dc270 by goroutine 12888:
2024-09-15T01:14:46.3336008Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/tables/txnentries.(*mergeObjectsEntry).collectDelsAndTransfer()
2024-09-15T01:14:46.3336643Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/tables/txnentries/mergeobjects.go:352 +0x671
2024-09-15T01:14:46.3337191Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/tables/txnentries.(*mergeObjectsEntry).PrepareCommit()
2024-09-15T01:14:46.3337816Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/tables/txnentries/mergeobjects.go:415 +0x7a4
2024-09-15T01:14:46.3338251Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnimpl.(*txnTable).PrepareCommit()
2024-09-15T01:14:46.3338764Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/txn/txnimpl/table.go:1254 +0x1ab
2024-09-15T01:14:46.3339188Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnimpl.(*txnDB).PrepareCommit()
2024-09-15T01:14:46.3339883Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/txn/txnimpl/txndb.go:497 +0x1d4
2024-09-15T01:14:46.3340383Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnimpl.(*txnStore).PrepareCommit()
2024-09-15T01:14:46.3340871Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/txn/txnimpl/store.go:771 +0x2f4
2024-09-15T01:14:46.3341382Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnbase.(*Txn).PrepareCommit()
2024-09-15T01:14:46.3341869Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/txn/txnbase/txn.go:327 +0x129
2024-09-15T01:14:46.3342339Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnbase.(*TxnManager).onPreparCommit()
2024-09-15T01:14:46.3342953Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/txn/txnbase/txnmgr.go:301 +0x37
2024-09-15T01:14:46.3343386Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnbase.(*TxnManager).onPrepare()
2024-09-15T01:14:46.3343923Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/txn/txnbase/txnmgr.go:355 +0x57
2024-09-15T01:14:46.3344379Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnbase.(*TxnManager).onPrepare1PC()
2024-09-15T01:14:46.3344868Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/txn/txnbase/txnmgr.go:380 +0x88
2024-09-15T01:14:46.3345347Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnbase.(*TxnManager).dequeuePreparing()
2024-09-15T01:14:46.3345846Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/txn/txnbase/txnmgr.go:474 +0x33b
2024-09-15T01:14:46.3346421Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnbase.(*TxnManager).dequeuePreparing-fm()
2024-09-15T01:14:46.3346587Z       <autogenerated>:1 +0x51
2024-09-15T01:14:46.3347029Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/logstore/sm.(*safeQueue).Start.func1()
2024-09-15T01:14:46.3347535Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/logstore/sm/safeq.go:89 +0x3a1
2024-09-15T01:14:46.3347547Z 
2024-09-15T01:14:46.3347742Z Previous write at 0x00c0157dc270 by goroutine 12903:
2024-09-15T01:14:46.3347871Z   runtime.mapclear()
2024-09-15T01:14:46.3348209Z       /opt/hostedtoolcache/go/1.23.0/x64/src/runtime/map.go:1075 +0x0
2024-09-15T01:14:46.3348621Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/model.(*TransferHashPage).Clear()
2024-09-15T01:14:46.3349083Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/model/pages.go:164 +0x1a4
2024-09-15T01:14:46.3349601Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/model.(*TransferTable[go.shape.*uint8]).executeTTL()
2024-09-15T01:14:46.3350155Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/model/transfer.go:97 +0xa1
2024-09-15T01:14:46.3351066Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/model.(*TransferTable[go.shape.*uint8]).RunTTL()
2024-09-15T01:14:46.3351820Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/model/transfer.go:113 +0xec
2024-09-15T01:14:46.3352148Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db.Open.func3()
2024-09-15T01:14:46.3352589Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/open.go:230 +0x109
2024-09-15T01:14:46.3352926Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/gc.(*Manager).process()
2024-09-15T01:14:46.3353373Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/gc/manager.go:94 +0x483
2024-09-15T01:14:46.3353816Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/gc.(*Manager).process-fm()
2024-09-15T01:14:46.3353976Z       <autogenerated>:1 +0x51
2024-09-15T01:14:46.3354403Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/logstore/sm.(*safeQueue).Start.func1()
2024-09-15T01:14:46.3354902Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/logstore/sm/safeq.go:89 +0x3a1
2024-09-15T01:14:46.3354914Z 
2024-09-15T01:14:46.3355050Z Goroutine 12888 (running) created at:
2024-09-15T01:14:46.3355447Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/logstore/sm.(*safeQueue).Start()
2024-09-15T01:14:46.3355939Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/logstore/sm/safeq.go:68 +0x1a4
2024-09-15T01:14:46.3356351Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/logstore/sm.(*stateMachine).Start()
2024-09-15T01:14:46.3356811Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/logstore/sm/sm.go:50 +0xa4
2024-09-15T01:14:46.3357339Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/txn/txnbase.(*TxnManager).Start()
2024-09-15T01:14:46.3357845Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/txn/txnbase/txnmgr.go:610 +0x77
2024-09-15T01:14:46.3358119Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db.Open()
2024-09-15T01:14:46.3358561Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/open.go:159 +0x2824
2024-09-15T01:14:46.3359024Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/testutil.InitTestDB()
2024-09-15T01:14:46.3359516Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/testutil/engine.go:390 +0x78
2024-09-15T01:14:46.3359939Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/testutil.NewTestEngine()
2024-09-15T01:14:46.3360431Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/testutil/engine.go:81 +0x67
2024-09-15T01:14:46.3360811Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/test.TestTransferInMerge()
2024-09-15T01:14:46.3361297Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/test/db_test.go:9623 +0x26e
2024-09-15T01:14:46.3361413Z   testing.tRunner()
2024-09-15T01:14:46.3361782Z       /opt/hostedtoolcache/go/1.23.0/x64/src/testing/testing.go:1690 +0x226
2024-09-15T01:14:46.3361915Z   testing.(*T).Run.gowrap1()
2024-09-15T01:14:46.3362271Z       /opt/hostedtoolcache/go/1.23.0/x64/src/testing/testing.go:1743 +0x44
2024-09-15T01:14:46.3362281Z 
2024-09-15T01:14:46.3362416Z Goroutine 12903 (running) created at:
2024-09-15T01:14:46.3362813Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/logstore/sm.(*safeQueue).Start()
2024-09-15T01:14:46.3363305Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/logstore/sm/safeq.go:68 +0x1a4
2024-09-15T01:14:46.3363700Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db.Open.(*Manager).Start.func35()
2024-09-15T01:14:46.3364137Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/gc/manager.go:160 +0x4a
2024-09-15T01:14:46.3364265Z   sync.(*Once).doSlow()
2024-09-15T01:14:46.3364588Z       /opt/hostedtoolcache/go/1.23.0/x64/src/sync/once.go:76 +0xe1
2024-09-15T01:14:46.3364699Z   sync.(*Once).Do()
2024-09-15T01:14:46.3365002Z       /opt/hostedtoolcache/go/1.23.0/x64/src/sync/once.go:67 +0x44
2024-09-15T01:14:46.3365327Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/gc.(*Manager).Start()
2024-09-15T01:14:46.3365784Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/gc/manager.go:159 +0x5b8e
2024-09-15T01:14:46.3366045Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db.Open()
2024-09-15T01:14:46.3366650Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/open.go:303 +0x5af3
2024-09-15T01:14:46.3367001Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/testutil.InitTestDB()
2024-09-15T01:14:46.3367500Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/testutil/engine.go:390 +0x78
2024-09-15T01:14:46.3367863Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/testutil.NewTestEngine()
2024-09-15T01:14:46.3368358Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/testutil/engine.go:81 +0x67
2024-09-15T01:14:46.3368735Z   github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db/test.TestTransferInMerge()
2024-09-15T01:14:46.3369218Z       /home/runner/work/matrixone/matrixone/pkg/vm/engine/tae/db/test/db_test.go:9623 +0x26e
2024-09-15T01:14:46.3369334Z   testing.tRunner()
2024-09-15T01:14:46.3369712Z       /opt/hostedtoolcache/go/1.23.0/x64/src/testing/testing.go:1690 +0x226
2024-09-15T01:14:46.3369842Z   testing.(*T).Run.gowrap1()
2024-09-15T01:14:46.3370200Z       /opt/hostedtoolcache/go/1.23.0/x64/src/testing/testing.go:1743 +0x44
2024-09-15T01:14:46.3370296Z ==================

Expected Behavior

no data race

Steps to Reproduce

https://github.com/matrixorigin/matrixone/actions/runs/10863914233/job/30154924981?pr=18778

Additional information

No response

XuPeng-SH commented 2 months ago

fixed

aressu1985 commented 2 months ago

@reusee 乐声看下

reusee commented 2 months ago

@reusee 乐声看下

已经修复，可以关闭的了