IANTHEREAL
commented
7 years ago @chenbobaoge 请提供一下 tidb 启动命令参数以及配置文件
Closed chenbobaoge closed 4 years ago
IANTHEREAL
commented
7 years ago @chenbobaoge 请提供一下 tidb 启动命令参数以及配置文件
chenbobaoge
commented
7 years ago @GregoryIan tidb没有通过配置文件启动的。直接用命令参数启动。命令如下： nohup tidb-latest-linux-amd64/bin/tidb-server --store=tikv --path="192.168.2.194:2379" --log-file=tidb.log --binlog-socket=unix:///tmp/pump.sock &
IANTHEREAL
commented
7 years ago @chenbobaoge 那我们在这里讨论吧。 tidb 和 pump 是在同一台机器上面吗，部署 pump 后重启过 tidb 吗？
chenbobaoge
commented
7 years ago @GregoryIan
好，就在这里讨论吧
1 在一台机器上，我的架构是，1台放pd和db 以及pump和drainer。3台放kv。
2部署完成后 重启过tidb。请问:tidb,pump,drainer的启动顺序有讲究吗？
我再把tidb的日志贴全一点吧：我加粗斜体的部分您看下是不是有问题
我奇怪的是：我配置的drainer的文件中，下游mysql数据库的ip是 2.86 端口是：3306，但这里的日志显示在这个ip上new 2044端口的连接，而且之后提示 [warning] parse attrs error:EOF
2017/09/30 16:42:50.146 printer.go:33: [info] Welcome to TiDB.
2017/09/30 16:42:50.146 printer.go:34: [info] Release Version: 0.9.0
2017/09/30 16:42:50.146 printer.go:35: [info] Git Commit Hash: fe31f4b5974a68a3e5698f2b3473101691217d18
2017/09/30 16:42:50.146 printer.go:36: [info] Git Branch: master
2017/09/30 16:42:50.146 printer.go:37: [info] UTC Build Time: 2017-09-20 01:56:02
2017/09/30 16:42:50.146 client.go:99: [info] [pd] create pd client with endpoints [192.168.2.194:2379]
2017/09/30 16:42:50.149 client.go:189: [info] [pd] leader switches to: http://192.168.2.194:2379, previous:
2017/09/30 16:42:50.149 client.go:117: [info] [pd] init cluster id 6470369233434023289
2017/09/30 16:42:50.152 tidb.go:69: [info] store tikv-6470369233434023289 new domain, ddl lease 10s, stats lease 3000000000
2017/09/30 16:42:50.153 ddl.go:304: [info] [ddl] start delRangeManager OK, with emulator: false
2017/09/30 16:42:50.153 ddl.go:311: [info] [ddl] start DDL:4e472f52-5136-4121-9ddb-2ac871686025
2017/09/30 16:42:50.211 domain.go:110: [info] [ddl] full load InfoSchema from version 0 to 78, in 50.198346ms
2017/09/30 16:42:50.219 main.go:169: [info] created binlog client at unix:///tmp/pump.sock
2017/09/30 16:42:50.219 gc_worker.go:106: [info] [gc worker] 57b47cf3f5c0009 start.
2017/09/30 16:42:50.219 server.go:183: [warning] Secure connection is NOT ENABLED
2017/09/30 16:42:50.219 server.go:162: [info] Server is running MySQL Protocol at [0.0.0.0:4000]
2017/09/30 16:42:50.219 main.go:178: [info] disable Prometheus push client
2017/09/30 16:42:50.219 systime_mon.go:24: [info] start system time monitor
2017/09/30 16:42:50.219 http_status.go:58: [info] Listening on :10080 for status and metrics report.
2017/09/30 16:42:50.226 simple.go:127: [info] [0] execute rollback statement
2017/09/30 16:42:50.226 txn.go:174: [info] [kv] Rollback txn 394988347771846666
2017/09/30 16:42:57.166 server.go:118: [info] [1] new connection 192.168.2.68:2044
2017/09/30 16:42:57.167 conn.go:273: [warning] parse attrs error:EOF
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/server/conn.go:290:
2017/09/30 16:42:57.187 set.go:145: [info] [1] set system variable autocommit = 1
2017/09/30 16:42:57.205 set.go:145: [info] [1] set system variable net_write_timeout = 600
2017/09/30 16:42:57.205 set.go:145: [info] [1] set system variable sql_select_limit = 502
2017/09/30 16:42:57.206 metrics.go:355: [warning] [EXPENSIVE_QUERY] / ApplicationName=DataGrip 2017.2.2 / select * from t1
2017/09/30 16:42:57.217 set.go:145: [info] [1] set system variable sql_select_limit = 18446744073709551615
2017/09/30 16:43:06.355 set.go:145: [info] [1] set system variable net_write_timeout = 600
2017/09/30 16:43:06.386 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:07.386 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:08.387 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:09.387 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:10.387 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:11.387 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:12.388 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:13.388 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:14.388 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:15.388 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:16.389 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:17.389 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:18.389 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:19.389 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:20.390 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:21.390 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:22.390 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:23.390 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:24.391 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:25.391 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:26.391 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:43:26.391 session.go:309: [warning] [1] finished txn:
IANTHEREAL
commented
7 years ago 没问题，粗斜体是你客户端到 tidb 的 connection. 启动顺序一般是 pump -> tidb -> drainer. 按照你的部署和配置理论上应该不会出错，你可以尝试删除 /tmp/pump.sock 然后按照顺序重启一遍，如果不 ok 我们可以远程一下
chenbobaoge
commented
7 years ago @GregoryIan
1 /tmp/pump.sock 已经尝试过了删除，因为不删除，启动drainer会提示pump.sock已存在。 2 关于启动顺序，我回头再按你说的顺序试一下，我之前的启动顺序是tidb->pump->drainer 不知道是不是这个原因导致的
chenbobaoge
commented
7 years ago @GregoryIan 按您说的启动顺序启动，还是没有成功，随后我把我的社交联系方式发到你的ArGregoryIan@gmail.com邮箱，你方便的时候帮我远程看一下，谢谢
chenbobaoge
commented
6 years ago @GregoryIan 你好， 1.上回这个问题，我重新按Ansible方式安装tidb之后 ，没有再有卡死的情况发生，不知道什么原因 2.另外现在有另外一个问题，就是使用binlog把tidb同步到mysql的过程中，发现insert语句，没有问题，可以正常同步，但create语句却没有反应，不知道为什么，是不支持吗？ 我的drainer.toml如下
# drainer Configuration.
# drainer 提供服务的地址(默认 "127.0.0.1:8249")
addr = "192.168.11.140:8249"
# 向 pd 查询在线 pump 的时间间隔 (默认 10，单位 秒)
detect-interval = 10
# drainer 数据存储位置路径 (默认 "data.drainer")
data-dir = "drainerdata"
# pd 集群节点的地址 (默认 "http://127.0.0.1:2379")
pd-urls = "http://192.168.11.142:2379,http://192.168.11.143:2379,http://192.168.11.144:2379"
# log 文件路径
log-file = "drainer.log"
# syncer Configuration.
[syncer]
## db 过滤列表 (默认 "INFORMATION_SCHEMA,PERFORMANCE_SCHEMA,mysql,test"),
## 不支持对 ignore schemas 的 table 进行 rename DDL 操作
ignore-schemas = "INFORMATION_SCHEMA,PERFORMANCE_SCHEMA,mysql"
# 输出到下游数据库一个事务的 sql 数量 (default 1)
txn-batch = 1
# 同步下游的并发数，该值设置越高同步的吞吐性能越好 (default 1)
worker-count = 5
# 是否禁用拆分单个 binlog 的 sqls 的功能，如果设置为 true，则按照每个 binlog
# 顺序依次还原成单个事务进行同步( 下游服务类型为 mysql, 该项设置为 False )
disable-dispatch = false
# drainer 下游服务类型 (默认为 mysql)
# 参数有效值为 "mysql", "pb"
db-type = "mysql"
# replicate-do-db priority over replicate-do-table if have same db name
# and we support regex expression ,
# 以 '~' 开始声明使用正则表达式
#
#replicate-do-db = ["firsttidb"]
#[[syncer.replicate-do-table]]
#db-name ="test"
#tbl-name = "log"
#[[syncer.replicate-do-table]]
#db-name ="test"
#tbl-name = "~^a.*"
# db-type 设置为 mysql 时，下游数据库服务器参数
[syncer.to]
host = "192.168.11.143"
user = "root"
password = "123456"
port = 3306
# db-type 设置为 pb 时,存放 binlog 文件的目录
# [syncer.to]
# dir = "data.drainer"
buggithubs
commented
6 years ago @chenbobaoge
chenbobaoge
commented
6 years ago @buggithubs 辛苦你了，这大礼拜天你也反应这么快 tidb-server -V Release Version: v1.0.1-1-gaee0b39 Git Commit Hash: aee0b39af29c87742811f02c657dcf2b0fe1d60c Git Commit Branch: release-1.0 UTC Build Time: 2017-11-01 01:51:09
pump -V pump Version: 2.0.0+git Git Commit Hash: 5893e217b104f025db924c877616baa9234a6712 Build TS: 2017-10-27 07:47:47 Go Version: go1.8 Go OS/Arch: linuxamd64
meyu44
commented
6 years ago @chenbobaoge 多谢关注，如果不介意，可以给 info@pingcap.com 发邮件，我们通过社交软件更高效便捷沟通
ayzmkk
commented
6 years ago 不知道楼主解决了吗？ 我也遇到类似的问题了，下游是mysql ， create的时候tidb卡死。是按照Ansible的方式部署的。
yikeke
你好 想通过TiDB-Binlog 连接mysql做实时备份。 启动各服务正常，进程正常，日志也没有错误，select执行正常。 可是一执行insert 语句， tidb进程就卡死，只有重启tidb服务才能恢复正常
tidb.log内容, [global:3]critical error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:26.117 session.go:670: [warning] [1] session error:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/sessionctx/binloginfo/binloginfo.go:91: [global:3]critical error rpc error: code = Unavailable desc = grpc: the connection is unavailable
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/2pc.go:634:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/2pc.go:560:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/txn.go:157:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/session.go:286:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/session.go:310:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/tidb.go:173:
{
"currDBName": "test2",
"id": 1,
"status": 2,
"strictMode": true,
"user": {
"Username": "root",
"Hostname": "192.168.2.68"
}
}
2017/09/30 16:54:26.117 conn.go:417: [error] [1] critical error, stop the server listener /home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/sessionctx/binloginfo/binloginfo.go:91: [global:3]critical error rpc error: code = Unavailable desc = grpc: the connection is unavailable
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/2pc.go:634:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/2pc.go:560:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/store/tikv/txn.go:157:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/session.go:286:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/session.go:310:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/tidb.go:173:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/session.go:672:
/home/jenkins/workspace/build_tidb_master/go/src/github.com/pingcap/tidb/server/conn.go:730:
2017/09/30 16:54:26.117 server.go:277: [info] [1] close connection
2017/09/30 16:54:26.121 server.go:248: [error] listener stopped, waiting for manual kill.
2017/09/30 16:54:26.129 2pc.go:548: [info] 2PC clean up done, tid: 394988524946063365
2017/09/30 16:54:27.117 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:28.117 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:29.118 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:30.118 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:31.118 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:32.118 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:33.118 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:34.119 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:35.119 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:36.119 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:37.119 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:38.120 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:39.120 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:40.120 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:41.121 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:42.121 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:43.121 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:44.121 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:45.121 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:54:46.122 2pc.go:649: [error] failed to write binlog: [global:3]critical error rpc error: code = Unavailable desc = grpc: the connection is unavailable
2017/09/30 16:55:26.121 server.go:248: [error] listener stopped, waiting for manual kill.
2017/09/30 16:56:26.121 server.go:248: [error] listener stopped, waiting for manual kill.
2017/09/30 16:57:26.122 server.go:248: [error] listener stopped, waiting for manual kill.
2017/09/30 16:54:06.112 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:07.112 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:08.112 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:09.113 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:10.113 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:11.113 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:12.113 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:13.114 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:14.114 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:15.114 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:16.115 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:17.115 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:18.115 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:19.115 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:20.115 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:21.116 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:22.116 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:23.116 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:24.116 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:25.116 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:26.117 binloginfo.go:88: [error] write binlog error rpc error: code = Unavailable desc = grpc: the connection is unavailable 2017/09/30 16:54:26.117 session.go:309: [warning] [1] finished txn:
pump.toml配置文件 addr = "127.0.0.1:8250" advertise-addr = "" gc = 7 data-dir = "data.pump" log-file = "pump.log" heartbeat-interval = 3 pd-urls = "http://192.168.2.194:2379" socket = "unix:///tmp/pump.sock"
drainer.toml配置文件 addr = "127.0.0.1:8249" detect-interval = 10 data-dir = "data.drainer" pd-urls = "http://192.168.2.194:2379" log-file = "drainer.log" [syncer] ignore-schemas = "INFORMATION_SCHEMA,PERFORMANCE_SCHEMA,mysql" txn-batch = 1 worker-count = 1 disable-dispatch = false db-type = "mysql" replicate-do-db =["test2"] [syncer.to] host = "192.168.2.68" user = "root" password = "123456" port = 3306
mysql 是在192.168.2.68上。 希望指点迷津
版本信息 Release Version: 0.9.0 Git Commit Hash: fe31f4b5974a68a3e5698f2b3473101691217d18 Git Commit Branch: master UTC Build Time: 2017-09-20 01:56:02