search

pingcap / dumpling

Dumpling is a fast, easy-to-use tool written by Go for dumping data from the database(MySQL, TiDB...) to local/cloud(S3, GCP...) in multifarious formats(SQL, CSV...).
Apache License 2.0
281 stars 85 forks source link

export: don't +1 TSO when set GC safepoint (#290) #316

Closed ti-chi-bot closed 3 years ago

ti-chi-bot commented 3 years ago

This is an automated cherry-pick of #290

What problem does this PR solve?

this error is hard to reproduce, so I added a 10 minutes sleep before any dump indeed occurs. The error may be slightly different but I think the root cause is same . Here's the log

[2021/06/03 19:20:30.262 +08:00] [INFO] [gc_worker.go:1531] ["[gc worker] sent safe point to PD"] [uuid=5e7451a0ff80001] ["safe point"=425385803120640001]
...
[2021/06/03 19:22:21.578 +08:00] [INFO] [set.go:312] ["load snapshot info schema"] [conn=5813] [SnapshotTS=425385803120640000]
...
[2021/06/03 19:22:21.579 +08:00] [INFO] [conn.go:812] ["command dispatched failed"] [conn=5813] [connInfo="id:5813, addr:127.0.0.1:62687 status:10, collation:utf8mb4_general_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="SET tidb_snapshot='2021-06-03 19:01:00'"] [txn_mode=PESSIMISTIC] [err="[tikv:9006]GC life time is shorter than transaction duration, transaction starts at 2021-06-03 19:01:00 +0800 CST, GC safe point is 2021-06-03 19:01:00 +0800 CST\ngithub.com/pingcap/errors.AddStack\n\t/Users/pingcap/gopkg/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/Users/pingcap/gopkg/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20201126102027-b0a155152ca3/normalize.go:156\ngithub.com/pingcap/tidb/store/tikv.(*KVStore).CheckVisibility\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/store/tikv/kv.go:104\ngithub.com/pingcap/tidb/store/tikv.(*tikvSnapshot).Get\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/store/tikv/snapshot.go:387\ngithub.com/pingcap/tidb/structure.(*TxStructure).Get\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/structure/string.go:36\ngithub.com/pingcap/tidb/structure.(*TxStructure).GetInt64\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/structure/string.go:45\ngithub.com/pingcap/tidb/meta.(*Meta).GetSchemaVersion\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/meta/meta.go:260\ngithub.com/pingcap/tidb/domain.(*Domain).loadInfoSchema\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/domain/domain.go:102\ngithub.com/pingcap/tidb/domain.(*Domain).GetSnapshotInfoSchema\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/domain/domain.go:316\ngithub.com/pingcap/tidb/executor.(*SetExecutor).loadSnapshotInfoSchemaIfNeeded\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/set.go:314\ngithub.com/pingcap/tidb/executor.(*SetExecutor).setSysVariable\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/set.go:195\ngithub.com/pingcap/tidb/executor.(*SetExecutor).Next\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/set.go:108\ngithub.com/pingcap/tidb/executor.Next\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/executor.go:277\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:537\ngithub.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:418\ngithub.com/pingcap/tidb/executor.(*ExecStmt).Exec\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/executor/adapter.go:368\ngithub.com/pingcap/tidb/session.runStmt\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/session/session.go:1531\ngithub.com/pingcap/tidb/session.(*session).ExecuteStmt\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/session/session.go:1426\ngithub.com/pingcap/tidb/server.(*TiDBContext).ExecuteStmt\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/server/driver_tidb.go:218\ngithub.com/pingcap/tidb/server.(*clientConn).handleStmt\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/server/conn.go:1630\ngithub.com/pingcap/tidb/server.(*clientConn).handleQuery\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/server/conn.go:1503\ngithub.com/pingcap/tidb/server.(*clientConn).dispatch\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/server/conn.go:1037\ngithub.com/pingcap/tidb/server.(*clientConn).Run\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/server/conn.go:795\ngithub.com/pingcap/tidb/server.(*Server).onConn\n\t/Users/pingcap/workspace/optimization-build-tidb-darwin-amd/go/src/github.com/pingcap/tidb/server/server.go:477\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"]

What is changed and how it works?

as title

Check List

Tests

Side effects

Related changes

Release note

lichunzhu commented 3 years ago

/merge

ti-chi-bot commented 3 years ago

This pull request has been accepted and is ready to merge.

Commit hash: b0b93bff9e91b7d6ca8e70dd5bb38cafe9becb40