search

pingcap / tidb

TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
https://pingcap.com
Apache License 2.0
37.11k stars 5.83k forks source link

Something wrong with binlog #6239

Closed lixiangdude closed 6 years ago

lixiangdude commented 6 years ago

I installed and tested the pump and drainer in March, and it works. Then I shut down the drainer. Today, I started the drainer discovery log to report "[warning] [stream] node erp-web-other03:8250, pos {Suffix:9 Offset:0}, error RPC error: code = Unknown desc = binlogger: file not found". I guess it may have been caused by the removal of the pump data after it expired. How can I solve this problem?

WangXiangUSTC commented 6 years ago

could you please provide the Git Commit Hash and the error log in detail, you can find Commit Hash in drainer's log.

IANTHEREAL commented 6 years ago

it seems u use old version binlog, it's a old bug. there are two suggestions:

  1. suggest u upgrade to have new version binlog that using kafka, u can use tidb-ansible
  2. u need to show use the binlog files in host erp-web-other03, then we help u to modify your save point file. [binlog files are under data dir configuration item of pump]
lixiangdude commented 6 years ago

@WangXiangUSTC Well,I can't find Commit Hash in drainer's log(if the drainer's log means drainer.log file). There is only a group of words that are recycled: 2018/04/09 16:51:23 collector.go:225: [info] erp-web-other03:8250 last CommitTS 0 2018/04/09 16:51:24 pump.go:385: [warning] [stream] node erp-web-other03:8250, pos {Suffix:9 Offset:0}, error rpc error: code = Unknown desc = binlogger: file not found 2018/04/09 16:51:27 pump.go:385: [warning] [stream] node erp-web-other03:8250, pos {Suffix:9 Offset:0}, error rpc error: code = Unknown desc = binlogger: file not found 2018/04/09 16:51:30 pump.go:385: [warning] [stream] node erp-web-other03:8250, pos {Suffix:9 Offset:0}, error rpc error: code = Unknown desc = binlogger: file not found

lixiangdude commented 6 years ago

@GregoryIan Ok,I'll try to use the new version. Thank you~

WangXiangUSTC commented 6 years ago

@Lee-Think you can execute ./bin/drainer -V to get commit version

lixiangdude commented 6 years ago

@WangXiangUSTC Git Commit Hash: 90b25b725d5ae2aa0b9959825dffeb7b415ba829

WangXiangUSTC commented 6 years ago

do you remove some file by yourself?

lixiangdude commented 6 years ago

No, it should have been removed by pump

WangXiangUSTC commented 6 years ago

pump will gc expired file, and the default gc value is 7 days.

lixiangdude commented 6 years ago

I've understood, thanks~~