pingcap / tidb-binlog

A tool used to collect and merge tidb's binlog for real-time data backup and synchronization.
Apache License 2.0
292 stars 131 forks source link

drainer crashes on VIEW ddl #521

Closed kolbe closed 5 years ago

kolbe commented 5 years ago

If the drainer encounters a CREATE VIEW statement, it crashes:

2019/04/05 14:24:59 schema.go:235: [debug] handle ddl job id(1109): {"id":1109,"type":21,"schema_id":1062,"table_id":1108,"state":6,"err":null,"err_count":0,"
row_count":0,"raw_args":null,"schema_state":5,"snapshot_ver":0,"start_ts":407500477872209971,"dependency_id":0,"query":"CREATE VIEW v1 AS SELECT JSON_TYPE(JSO
N_OBJECT());","binlog":{"SchemaVersion":1441,"DBInfo":null,"TableInfo":{"id":1108,"name":{"O":"v1","L":"v1"},"charset":"utf8","collate":"utf8_general_ci","col
s":[{"id":1,"name":{"O":"JSON_TYPE(JSON_OBJECT())","L":"json_type(json_object())"},"offset":0,"origin_default":null,"default":null,"default_bit":null,"generat
ed_expr_string":"","generated_stored":false,"dependences":null,"type":{"Tp":0,"Flag":0,"Flen":0,"Decimal":0,"Charset":"","Collate":"","Elems":null},"state":5,
"comment":""}],"index_info":null,"fk_info":null,"state":5,"pk_is_handle":false,"comment":"","auto_inc_id":0,"max_col_id":1,"max_idx_id":0,"update_timestamp":4
07500477885317127,"ShardRowIDBits":0,"partition":null,"compression":""},"FinishedTS":407500477898948611},"version":1,"reorg_meta":null,"priority":0}
2019/04/05 14:24:59 syncer.go:638: [debug] receive publish binlog item: {startTS: 407502545135861761, commitTS: 407502545135861764, node: seastar.local:8250}
2019/04/05 14:24:59 server.go:225: [error] syncer exited, error table v1(1108) not found
github.com/pingcap/errors.NotFoundf
        /Users/kolbe/Devel/go/pkg/mod/github.com/pingcap/errors@v0.11.0/juju_adaptor.go:72
github.com/pingcap/tidb-binlog/drainer.(*Schema).ReplaceTable
        /Users/kolbe/Devel/go/src/github.com/pingcap/tidb-binlog/drainer/schema.go:183
github.com/pingcap/tidb-binlog/drainer.(*Schema).handleDDL
        /Users/kolbe/Devel/go/src/github.com/pingcap/tidb-binlog/drainer/schema.go:397
github.com/pingcap/tidb-binlog/drainer.(*Schema).handlePreviousDDLJobIfNeed
        /Users/kolbe/Devel/go/src/github.com/pingcap/tidb-binlog/drainer/schema.go:238
github.com/pingcap/tidb-binlog/drainer.(*Syncer).run
        /Users/kolbe/Devel/go/src/github.com/pingcap/tidb-binlog/drainer/syncer.go:496
github.com/pingcap/tidb-binlog/drainer.(*Syncer).Start
        /Users/kolbe/Devel/go/src/github.com/pingcap/tidb-binlog/drainer/syncer.go:96
github.com/pingcap/tidb-binlog/drainer.(*Server).StartSyncer.func1
        /Users/kolbe/Devel/go/src/github.com/pingcap/tidb-binlog/drainer/server.go:223
runtime.goexit
        /usr/local/Cellar/go/1.12.1/libexec/src/runtime/asm_amd64.s:1337
kolbe commented 5 years ago

The issue is that there's no case for model.ActionCreateView in the switch inside drainer/schema.go:handleDDL(). The fix is pretty simple:


index dd84b5b..d67cf6b 100644
--- a/drainer/schema.go
+++ b/drainer/schema.go
@@ -315,7 +315,7 @@ func (s *Schema) handleDDL(job *model.Job) (schemaName string, tableName string,
                schemaName = schema.Name.O
                tableName = table.Name.O

-       case model.ActionCreateTable:
+       case model.ActionCreateTable, model.ActionCreateView:
                table := job.BinlogInfo.TableInfo
                if table == nil {
                        return "", "", "", errors.NotFoundf("table %d", job.TableID)
@@ -336,7 +336,7 @@ func (s *Schema) handleDDL(job *model.Job) (schemaName string, tableName string,
                schemaName = schema.Name.O
                tableName = table.Name.O

-       case model.ActionDropTable:
+       case model.ActionDropTable, model.ActionDropView:
                schema, ok := s.SchemaByID(job.SchemaID)
                if !ok {
                        return "", "", "", errors.NotFoundf("schema %d", job.SchemaID)```

... however, that relies on commit 20bc24aef58addbd906513ec13424e5755e6d97a to github.com/pingcap/parser, which adds ActionDropView to the parser model.

For some reason commit 20bc24aef58addbd906513ec13424e5755e6d97a was never pulled into the release-2.1 branch. I cherry-picked it locally and everything built fine, so I'll file a PR for that in github.com/pingcap/parser.
kolbe commented 5 years ago

This is the PR to get 20bc24aef58addbd906513ec13424e5755e6d97a cherry-picked to github.com/pingcap/parser: https://github.com/pingcap/parser/pull/276