search

pingcap / tidb

TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
https://pingcap.com
Apache License 2.0
36.86k stars 5.8k forks source link

Setting up and deploy TiDB: using docker-compose in MacOS #11287

Closed thuyenptr closed 5 years ago

thuyenptr commented 5 years ago

Bug Report

Hi,

  1. What did you do?

Setting up and deploy TiDB: using docker-compose in MacOS Following this tutorial: https://pingcap.com/blog/how_to_spin_up_an_htap_database_in_5_minutes_with_tidb_tispark/ when I start importing sample data with command

mysql -h 127.0.0.1 -P 4000 -u root < dss.ddl

it doesn't work and tidb has repeated error show in logs but it works well in Linux.

  1. What did you expect to see?
  2. What did you see instead? Error logs:
    • tidb logs:

[2019/07/17 04:07:11.161 +00:00] [INFO] [printer.go:40] ["Welcome to TiDB."] ["Release Version"=v3.0.0-rc.1-347-g23d4c97e2] ["Git Commit Hash"=23d4c97e2213e7f63f66f6c2094f728c9321200a] ["Git Branch"=master] ["UTC Build Time"="2019-07-17 01:29:23"] [GoVersion="go version go1.12 linux/amd64"] ["Race Enabled"=false] ["Check Table Before Drop"=false] ["TiKV Min Version"=2.1.0-alpha.1-ff3dd160846b7d1aed9079c389fc188f7f5ea13e] [2019/07/17 04:07:11.166 +00:00] [INFO] [printer.go:53] ["loaded config"] [config="{\"host\":\"0.0.0.0\",\"advertise-address\":\"\",\"port\":4000,\"cors\":\"\",\"store\":\"tikv\",\"path\":\"pd0:2379,pd1:2379,pd2:2379\",\"socket\":\"\",\"lease\":\"0\",\"run-ddl\":true,\"split-table\":true,\"token-limit\":1000,\"oom-action\":\"log\",\"mem-quota-query\":34359738368,\"enable-streaming\":false,\"txn-local-latches\":{\"enabled\":true,\"capacity\":2048000},\"lower-case-table-names\":2,\"log\":{\"level\":\"error\",\"format\":\"text\",\"disable-timestamp\":false,\"file\":{\"filename\":\"/logs/tidb.log\",\"log-rotate\":true,\"max-size\":300,\"max-days\":0,\"max-backups\":0},\"slow-query-file\":\"\",\"slow-threshold\":300,\"expensive-threshold\":10000,\"query-log-max-len\":2048},\"security\":{\"skip-grant-table\":false,\"ssl-ca\":\"\",\"ssl-cert\":\"\",\"ssl-key\":\"\",\"cluster-ssl-ca\":\"\",\"cluster-ssl-cert\":\"\",\"cluster-ssl-key\":\"\"},\"status\":{\"report-status\":true,\"status-host\":\"0.0.0.0\",\"status-port\":10080,\"metrics-addr\":\"pushgateway:9091\",\"metrics-interval\":15,\"record-db-qps\":false},\"performance\":{\"max-procs\":0,\"max-memory\":0,\"tcp-keep-alive\":true,\"cross-join\":true,\"stats-lease\":\"3s\",\"run-auto-analyze\":true,\"stmt-count-limit\":5000,\"feedback-probability\":0,\"query-feedback-limit\":1024,\"pseudo-estimate-ratio\":0.7,\"force-priority\":\"NO_PRIORITY\",\"bind-info-lease\":\"3s\",\"txn-entry-count-limit\":300000,\"txn-total-size-limit\":104857600},\"prepared-plan-cache\":{\"enabled\":false,\"capacity\":100,\"memory-guard-ratio\":0.1},\"opentracing\":{\"enable\":false,\"sampler\":{\"type\":\"const\",\"param\":1,\"sampling-server-url\":\"\",\"max-operations\":0,\"sampling-refresh-interval\":0},\"reporter\":{\"queue-size\":0,\"buffer-flush-interval\":0,\"log-spans\":false,\"local-agent-host-port\":\"\"},\"rpc-metrics\":false},\"proxy-protocol\":{\"networks\":\"\",\"header-timeout\":5},\"tikv-client\":{\"grpc-connection-count\":16,\"grpc-keepalive-time\":10,\"grpc-keepalive-timeout\":3,\"commit-timeout\":\"41s\",\"max-txn-time-use\":590,\"max-batch-size\":128,\"overload-threshold\":200,\"max-batch-wait-time\":0,\"batch-wait-size\":8},\"binlog\":{\"enable\":false,\"write-timeout\":\"15s\",\"ignore-error\":false,\"binlog-socket\":\"\",\"strategy\":\"range\"},\"compatible-kill-query\":false,\"plugin\":{\"dir\":\"\",\"load\":\"\"},\"pessimistic-txn\":{\"enable\":false,\"default\":false,\"max-retry-count\":256,\"ttl\":\"30s\"},\"check-mb4-value-in-utf8\":true,\"treat-old-version-utf8-as-utf8mb4\":true,\"enable-table-lock\":false,\"delay-clean-table-lock\":0}"] [2019/07/17 04:12:11.296 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.301 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.306 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.307 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.308 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.319 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.308 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.308 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.308 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.326 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.308 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.308 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.312 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.312 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.312 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.312 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.312 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.312 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.312 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.312 +00:00] [ERROR] [client_batch.go:257] ["batchRecvLoop error when receive"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = transport is closing"] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:257"] [2019/07/17 04:12:11.295 +00:00] [ERROR] [client.go:448] ["[pd] getTS error"] [error="rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader"] [errorVerbose="rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader\ngithub.com/pingcap/pd/client.(*client).processTSORequests\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:484\ngithub.com/pingcap/pd/client.(*client).tsLoop\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:433\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337"] [stack="github.com/pingcap/log.Error\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/log@v0.0.0-20190307075452-bd41d9273596/global.go:42\ngithub.com/pingcap/pd/client.(*client).tsLoop\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:448"] [2019/07/17 04:12:11.317 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.324 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.325 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.329 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.330 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.331 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.333 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.334 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.334 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.336 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.344 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.345 +00:00] [ERROR] [client_batch.go:229] ["batchRecvLoop re-create streaming fail"] [target=tikv0:20160] [error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 172.18.0.10:20160: connect: connection refused\""] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).reCreateStreamingClient\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:229\ngithub.com/pingcap/tidb/store/tikv.(*batchCommandsClient).batchRecvLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:263"] [2019/07/17 04:12:11.354 +00:00] [ERROR] [client.go:448] ["[pd] getTS error"] [error="rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader"] [errorVerbose="rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader\ngithub.com/pingcap/pd/client.(*client).processTSORequests\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:484\ngithub.com/pingcap/pd/client.(*client).tsLoop\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:433\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337"] [stack="github.com/pingcap/log.Error\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/log@v0.0.0-20190307075452-bd41d9273596/global.go:42\ngithub.com/pingcap/pd/client.(*client).tsLoop\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:448"] [2019/07/17 04:12:11.355 +00:00] [ERROR] [pd.go:129] ["updateTS error"] [error="rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader"] [errorVerbose="rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader\ngithub.com/pingcap/pd/client.(*client).processTSORequests\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:484\ngithub.com/pingcap/pd/client.(*client).tsLoop\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:433\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337\ngithub.com/pingcap/pd/client.(*tsoRequest).Wait\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:602\ngithub.com/pingcap/pd/client.(*client).GetTS\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:620\ngithub.com/pingcap/tidb/store/tikv/oracle/oracles.(*pdOracle).getTimestamp\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/oracle/oracles/pd.go:103\ngithub.com/pingcap/tidb/store/tikv/oracle/oracles.(*pdOracle).updateTS\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/oracle/oracles/pd.go:127\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337\ngithub.com/pingcap/errors.AddStack\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/errors@v0.11.4/errors.go:174\ngithub.com/pingcap/errors.Trace\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/errors@v0.11.4/juju_adaptor.go:15\ngithub.com/pingcap/tidb/store/tikv/oracle/oracles.(*pdOracle).getTimestamp\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/oracle/oracles/pd.go:105\ngithub.com/pingcap/tidb/store/tikv/oracle/oracles.(*pdOracle).updateTS\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/oracle/oracles/pd.go:127\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337"] [stack="github.com/pingcap/tidb/store/tikv/oracle/oracles.(*pdOracle).updateTS\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/oracle/oracles/pd.go:129"] [2019/07/17 04:12:11.360 +00:00] [ERROR] [client.go:448] ["[pd] getTS error"] [error="rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader"] [errorVerbose="rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader\ngithub.com/pingcap/pd/client.(*client).processTSORequests\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:484\ngithub.com/pingcap/pd/client.(*client).tsLoop\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:433\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337"] [stack="github.com/pingcap/log.Error\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/log@v0.0.0-20190307075452-bd41d9273596/global.go:42\ngithub.com/pingcap/pd/client.(*client).tsLoop\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:448"] [2019/07/17 04:12:11.362 +00:00] [ERROR] [client.go:448] ["[pd] getTS error"] [error="rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader"] [errorVerbose="rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader\ngithub.com/pingcap/pd/client.(*client).processTSORequests\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:484\ngithub.com/pingcap/pd/client.(*client).tsLoop\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:433\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337"] [stack="github.com/pingcap/log.Error\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/log@v0.0.0-20190307075452-bd41d9273596/global.go:42\ngithub.com/pingcap/pd/client.(*client).tsLoop\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:448"] [2019/07/17 04:12:11.363 +00:00] [ERROR] [pd.go:129] ["updateTS error"] [error="rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader"] [errorVerbose="rpc error: code = Unknown desc = rpc error: code = Unavailable desc = not leader\ngithub.com/pingcap/pd/client.(*client).processTSORequests\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:484\ngithub.com/pingcap/pd/client.(*client).tsLoop\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:433\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337\ngithub.com/pingcap/pd/client.(*tsoRequest).Wait\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:602\ngithub.com/pingcap/pd/client.(*client).GetTS\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/pd@v0.0.0-20190712044914-75a1f9f3062b/client/client.go:620\ngithub.com/pingcap/tidb/store/tikv/oracle/oracles.(*pdOracle).getTimestamp\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/oracle/oracles/pd.go:103\ngithub.com/pingcap/tidb/store/tikv/oracle/oracles.(*pdOracle).updateTS\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/oracle/oracles/pd.go:127\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337\ngithub.com/pingcap/errors.AddStack\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/errors@v0.11.4/errors.go:174\ngithub.com/pingcap/errors.Trace\n\t/home/jenkins/workspace/build_tidb_master/go/pkg/mod/github.com/pingcap/errors@v0.11.4/juju_adaptor.go:15\ngithub.com/pingcap/tidb/store/tikv/oracle/oracles.(*pdOracle).getTimestamp\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/oracle/oracles/pd.go:105\ngithub.com/pingcap/tidb/store/tikv/oracle/oracles.(*pdOracle).updateTS\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/oracle/oracles/pd.go:127\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337"] [stack="github.com/pingcap/tidb/store/tikv/oracle/oracles.(*pdOracle).updateTS\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/oracle/oracles/pd.go:129"] [2019/07/17 04:12:11.380 +00:00] [ERROR] [client_batch.go:182] ["batch commands send error"] [target=tikv0:20160] [error=EOF] [stack="github.com/pingcap/tidb/store/tikv.(*batchCommandsClient).send\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:182\ngithub.com/pingcap/tidb/store/tikv.(*batchConn).batchSendLoop\n\t/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/client_batch.go:379"]

[2019/07/17 04:12:11.223 +00:00] [ERROR] [redirector.go:52] ["redirect but server is not leader"] [from=pd1] [server=pd0] [2019/07/17 04:12:11.243 +00:00] [ERROR] [redirector.go:52] ["redirect but server is not leader"] [from=pd2] [server=pd0] [2019/07/17 04:12:51.127 +00:00] [ERROR] [redirector.go:97] ["request failed"] [error="Get http://pd1:2379/pd/api/v1/trend: context canceled"] [2019/07/17 04:12:59.801 +00:00] [ERROR] [grpclog.go:75] ["transport: loopyWriter.run returning. Err: write tcp 172.18.0.4:2379->172.18.0.10:45548: write: broken pipe"] [2019/07/17 04:13:00.616 +00:00] [ERROR] [grpclog.go:75] ["transport: loopyWriter.run returning. Err: write tcp 172.18.0.4:2379->172.18.0.11:58084: write: broken pipe"] [2019/07/17 04:13:02.078 +00:00] [ERROR] [grpclog.go:75] ["transport: loopyWriter.run returning. Err: write tcp 172.18.0.4:2379->172.18.0.10:45580: write: broken pipe"] [2019/07/17 04:13:08.626 +00:00] [ERROR] [heartbeat_streams.go:121] ["send keepalive message fail"] [target-store-id=1] [error=EOF] [2019/07/17 04:13:08.885 +00:00] [ERROR] [heartbeat_streams.go:121] ["send keepalive message fail"] [target-store-id=5] [error=EOF] [2019/07/17 04:13:08.995 +00:00] [ERROR] [heartbeat_streams.go:121] ["send keepalive message fail"] [target-store-id=4] [error=EOF] [2019/07/17 04:13:23.346 +00:00] [ERROR] [client.go:115] ["region sync with leader meet error"] [error="rpc error: code = Canceled desc = context canceled"] [2019/07/17 04:13:50.477 +00:00] [ERROR] [leader.go:123] ["campaign leader meet error"] [error="context deadline exceeded"] [2019/07/17 04:14:47.942 +00:00] [ERROR] [etcdutil.go:107] ["load from etcd meet error"] [error="context deadline exceeded"] [2019/07/17 04:14:47.960 +00:00] [ERROR] [leader.go:92] ["get leader meet error"] [error="context deadline exceeded"] [2019/07/17 04:14:48.187 +00:00] [ERROR] [leader.go:85] ["no etcd leader, check leader later"] [2019/07/17 04:14:48.230 +00:00] [ERROR] [metricutil.go:70] ["could not push metrics to Prometheus Pushgateway"] [error="Put http://pushgateway:9091/metrics/job/pd0/instance/ecd23976e2f3: dial tcp: lookup pushgateway on 127.0.0.11:53: no such host"] [2019/07/17 04:14:48.405 +00:00] [ERROR] [leader.go:85] ["no etcd leader, check leader later"]

[2019/07/17 04:07:11.615 +00:00] [ERROR] [client.rs:345] ["failed to send heartbeat"] [err="Grpc(RpcFinished(Some(RpcStatus { status: Ok, details: None })))"] [2019/07/17 04:07:11.618 +00:00] [ERROR] [util.rs:287] ["request failed, retry"] [err="Grpc(RpcFinished(Some(RpcStatus { status: Ok, details: None })))"] [2019/07/17 04:07:11.620 +00:00] [ERROR] [util.rs:287] ["request failed, retry"] [err="Other(SendError(\"...\"))"] [2019/07/17 04:07:11.621 +00:00] [ERROR] [util.rs:287] ["request failed, retry"] [err="Other(SendError(\"...\"))"] [2019/07/17 04:07:15.050 +00:00] [ERROR] [endpoint.rs:454] [error-response] [err="region message: \"region epoch is not match\" epoch_not_match { current_regions { id: 2 start_key: 7480000000000000FF2700000000000000F8 region_epoch { conf_ver: 1 version: 19 } peers { id: 3 store_id: 1 } } current_regions { id: 40 start_key: 7480000000000000FF2500000000000000F8 end_key: 7480000000000000FF2700000000000000F8 region_epoch { conf_ver: 1 version: 19 } peers { id: 41 store_id: 1 } } }"] [2019/07/17 04:07:42.062 +00:00] [ERROR] [endpoint.rs:454] [error-response] [err="region message: \"peer is not leader\" not_leader { region_id: 16 leader { id: 47 store_id: 5 } }"] [2019/07/17 04:08:15.145 +00:00] [ERROR] [endpoint.rs:454] [error-response] [err="region message: \"peer is not leader\" not_leader { region_id: 18 leader { id: 48 store_id: 4 } }"] [2019/07/17 04:12:04.419 +00:00] [ERROR] [process.rs:175] ["get snapshot failed"] [err="Request(message: \"region epoch is not match\" epoch_not_match { current_regions { id: 2 start_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 3 store_id: 1 } peers { id: 76 store_id: 4 } peers { id: 79 store_id: 5 } } current_regions { id: 108 start_key: 7480000000000000FF3700000000000000F8 end_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 109 store_id: 1 } peers { id: 110 store_id: 4 } peers { id: 111 store_id: 5 } } })"] [cid=324] [2019/07/17 04:12:04.424 +00:00] [ERROR] [process.rs:175] ["get snapshot failed"] [err="Request(message: \"region epoch is not match\" epoch_not_match { current_regions { id: 2 start_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 3 store_id: 1 } peers { id: 76 store_id: 4 } peers { id: 79 store_id: 5 } } current_regions { id: 108 start_key: 7480000000000000FF3700000000000000F8 end_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 109 store_id: 1 } peers { id: 110 store_id: 4 } peers { id: 111 store_id: 5 } } })"] [cid=325] [2019/07/17 04:12:04.424 +00:00] [ERROR] [process.rs:175] ["get snapshot failed"] [err="Request(message: \"region epoch is not match\" epoch_not_match { current_regions { id: 2 start_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 3 store_id: 1 } peers { id: 76 store_id: 4 } peers { id: 79 store_id: 5 } } current_regions { id: 108 start_key: 7480000000000000FF3700000000000000F8 end_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 109 store_id: 1 } peers { id: 110 store_id: 4 } peers { id: 111 store_id: 5 } } })"] [cid=326] [2019/07/17 04:12:04.425 +00:00] [ERROR] [process.rs:175] ["get snapshot failed"] [err="Request(message: \"region epoch is not match\" epoch_not_match { current_regions { id: 2 start_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 3 store_id: 1 } peers { id: 76 store_id: 4 } peers { id: 79 store_id: 5 } } current_regions { id: 108 start_key: 7480000000000000FF3700000000000000F8 end_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 109 store_id: 1 } peers { id: 110 store_id: 4 } peers { id: 111 store_id: 5 } } })"] [cid=327] [2019/07/17 04:12:04.425 +00:00] [ERROR] [process.rs:175] ["get snapshot failed"] [err="Request(message: \"region epoch is not match\" epoch_not_match { current_regions { id: 2 start_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 3 store_id: 1 } peers { id: 76 store_id: 4 } peers { id: 79 store_id: 5 } } current_regions { id: 108 start_key: 7480000000000000FF3700000000000000F8 end_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 109 store_id: 1 } peers { id: 110 store_id: 4 } peers { id: 111 store_id: 5 } } })"] [cid=328] [2019/07/17 04:12:04.426 +00:00] [ERROR] [process.rs:175] ["get snapshot failed"] [err="Request(message: \"region epoch is not match\" epoch_not_match { current_regions { id: 2 start_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 3 store_id: 1 } peers { id: 76 store_id: 4 } peers { id: 79 store_id: 5 } } current_regions { id: 108 start_key: 7480000000000000FF3700000000000000F8 end_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 109 store_id: 1 } peers { id: 110 store_id: 4 } peers { id: 111 store_id: 5 } } })"] [cid=329] [2019/07/17 04:12:04.427 +00:00] [ERROR] [process.rs:175] ["get snapshot failed"] [err="Request(message: \"region epoch is not match\" epoch_not_match { current_regions { id: 2 start_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 3 store_id: 1 } peers { id: 76 store_id: 4 } peers { id: 79 store_id: 5 } } current_regions { id: 108 start_key: 7480000000000000FF3700000000000000F8 end_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 109 store_id: 1 } peers { id: 110 store_id: 4 } peers { id: 111 store_id: 5 } } })"] [cid=330] [2019/07/17 04:12:04.427 +00:00] [ERROR] [process.rs:175] ["get snapshot failed"] [err="Request(message: \"region epoch is not match\" epoch_not_match { current_regions { id: 2 start_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 3 store_id: 1 } peers { id: 76 store_id: 4 } peers { id: 79 store_id: 5 } } current_regions { id: 108 start_key: 7480000000000000FF3700000000000000F8 end_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 109 store_id: 1 } peers { id: 110 store_id: 4 } peers { id: 111 store_id: 5 } } })"] [cid=331] [2019/07/17 04:12:04.427 +00:00] [ERROR] [process.rs:175] ["get snapshot failed"] [err="Request(message: \"region epoch is not match\" epoch_not_match { current_regions { id: 2 start_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 3 store_id: 1 } peers { id: 76 store_id: 4 } peers { id: 79 store_id: 5 } } current_regions { id: 108 start_key: 7480000000000000FF3700000000000000F8 end_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 109 store_id: 1 } peers { id: 110 store_id: 4 } peers { id: 111 store_id: 5 } } })"] [cid=332] [2019/07/17 04:12:04.428 +00:00] [ERROR] [process.rs:175] ["get snapshot failed"] [err="Request(message: \"region epoch is not match\" epoch_not_match { current_regions { id: 2 start_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 3 store_id: 1 } peers { id: 76 store_id: 4 } peers { id: 79 store_id: 5 } } current_regions { id: 108 start_key: 7480000000000000FF3700000000000000F8 end_key: 7480000000000000FF3900000000000000F8 region_epoch { conf_ver: 5 version: 27 } peers { id: 109 store_id: 1 } peers { id: 110 store_id: 4 } peers { id: 111 store_id: 5 } } })"] [cid=333]

  1. What version of TiDB are you using (tidb-server -V or run select tidb_version(); on TiDB)? | Release Version: v3.0.0-rc.1-347-g23d4c97e2 Git Commit Hash: 23d4c97e2213e7f63f66f6c2094f728c9321200a Git Branch: master UTC Build Time: 2019-07-17 01:29:23 GoVersion: go version go1.12 linux/amd64 Race Enabled: false TiKV Min Version: 2.1.0-alpha.1-ff3dd160846b7d1aed9079c389fc188f7f5ea13e Check Table Before Drop: false |

Thank you.

thuyenptr commented 5 years ago