pingcap / tidb

TiDB - the open-source, cloud-native, distributed SQL database designed for modern applications.
https://pingcap.com
Apache License 2.0
37.29k stars 5.85k forks source link

br backup txn or br restore txn,error:GetMinResolvedTSByStoresIDs 404 Not Found #53402

Open fancy-lee opened 6 months ago

fancy-lee commented 6 months ago

tidb-version: 5.3.0 br-version: Self-compiled latest version

When executing the br backup txnor br restore txn commands, a large number of log errors are output:

[2024/05/16 08:46:31.444 +08:00] [ERROR] [client.go:226] ["[pd] request failed with a non-200 status"] [source=tikv-driver] [name=GetMinResolvedTSByStoresIDs] [url=http://idc02-jfs01-tikv-backup-003.hogpu.cc:2379/pd/api/v1/min-resolved-ts] [method=GET] [caller-id=pd-http-client] [status="404 Not Found"] [body="404 page not found\n"] [stack="github.com/tikv/pd/client/http.(clientInner).doRequest\n\t/home/go21path/pkg/mod/github.com/tikv/pd/client@v0.0.0-20240322051414-fb9e2d561b6e/http/client.go:226\ngithub.com/tikv/pd/client/http.(clientInner).requestWithRetry.func1\n\t/home/go21path/pkg/mod/github.com/tikv/pd/client@v0.0.0-20240322051414-fb9e2d561b6e/http/client.go:134\ngithub.com/tikv/pd/client/http.(clientInner).requestWithRetry\n\t/home/go21path/pkg/mod/github.com/tikv/pd/client@v0.0.0-20240322051414-fb9e2d561b6e/http/client.go:144\ngithub.com/tikv/pd/client/http.(client).request\n\t/home/go21path/pkg/mod/github.com/tikv/pd/client@v0.0.0-20240322051414-fb9e2d561b6e/http/client.go:363\ngithub.com/tikv/pd/client/http.(client).GetMinResolvedTSByStoresIDs\n\t/home/go21path/pkg/mod/github.com/tikv/pd/client@v0.0.0-20240322051414-fb9e2d561b6e/http/interface.go:864\ngithub.com/tikv/client-go/v2/tikv.(KVStore).getMinResolvedTSByStoresIDs\n\t/home/go21path/pkg/mod/github.com/tikv/client-go/v2@v2.0.8-0.20240322070737-05aaba6cc6f7/tikv/kv.go:751\ngithub.com/tikv/client-go/v2/tikv.(KVStore).updateGlobalTxnScopeTSFromPD\n\t/home/go21path/pkg/mod/github.com/tikv/client-go/v2@v2.0.8-0.20240322070737-05aaba6cc6f7/tikv/kv.go:785\ngithub.com/tikv/client-go/v2/tikv.(KVStore).updateSafeTS\n\t/home/go21path/pkg/mod/github.com/tikv/client-go/v2@v2.0.8-0.20240322070737-05aaba6cc6f7/tikv/kv.go:654\ngithub.com/tikv/client-go/v2/tikv.(KVStore).safeTSUpdater\n\t/home/go21path/pkg/mod/github.com/tikv/client-go/v2@v2.0.8-0.20240322070737-05aaba6cc6f7/tikv/kv.go:647"] [2024/05/16 08:46:31.444 +08:00] [ERROR] [client.go:226] ["[pd] request failed with a non-200 status"] [source=tikv-driver] [name=GetMinResolvedTSByStoresIDs] [url=http://idc02-jfs01-tikv-backup-003.hogpu.cc:2379/pd/api/v1/min-resolved-ts] [method=GET] [caller-id=pd-http-client] [status="404 Not Found"] [body="404 page not found\n"] [stack="github.com/tikv/pd/client/http.(clientInner).doRequest\n\t/home/go21path/pkg/mod/github.com/tikv/pd/client@v0.0.0-20240322051414-fb9e2d561b6e/http/client.go:226\ngithub.com/tikv/pd/client/http.(clientInner).requestWithRetry.func1\n\t/home/go21path/pkg/mod/github.com/tikv/pd/client@v0.0.0-20240322051414-fb9e2d561b6e/http/client.go:134\ngithub.com/tikv/pd/client/http.(clientInner).requestWithRetry\n\t/home/go21path/pkg/mod/github.com/tikv/pd/client@v0.0.0-20240322051414-fb9e2d561b6e/http/client.go:144\ngithub.com/tikv/pd/client/http.(client).request\n\t/home/go21path/pkg/mod/github.com/tikv/pd/client@v0.0.0-20240322051414-fb9e2d561b6e/http/client.go:363\ngithub.com/tikv/pd/client/http.(client).GetMinResolvedTSByStoresIDs\n\t/home/go21path/pkg/mod/github.com/tikv/pd/client@v0.0.0-20240322051414-fb9e2d561b6e/http/interface.go:864\ngithub.com/tikv/client-go/v2/tikv.(KVStore).getMinResolvedTSByStoresIDs\n\t/home/go21path/pkg/mod/github.com/tikv/client-go/v2@v2.0.8-0.20240322070737-05aaba6cc6f7/tikv/kv.go:751\ngithub.com/tikv/client-go/v2/tikv.(KVStore).updateSafeTS\n\t/home/go21path/pkg/mod/github.com/tikv/client-go/v2@v2.0.8-0.20240322070737-05aaba6cc6f7/tikv/kv.go:673\ngithub.com/tikv/client-go/v2/tikv.(*KVStore).safeTSUpdater\n\t/home/go21path/pkg/mod/github.com/tikv/client-go/v2@v2.0.8-0.20240322070737-05aaba6cc6f7/tikv/kv.go:647"]

What is the cause? Is there any impact on backup and restore?

ArthurChiao commented 1 month ago

Noticed that 6.x has this API, but 5.x doesn't: https://github.com/tikv/pd/blob/v6.0.0/server/api/min_resolved_ts.go and according to our experience (br backup/restore txn), this error can be ignored.