search

actiontech / dtle

Distributed Data Transfer Service for MySQL
https://actiontech.github.io/dtle-docs-cn
Mozilla Public License 2.0
551 stars 132 forks source link

Panic occured on dtle when execute `create table XXX` in 'scenario of reciprocating migration on the same mysql instance' #413

Closed Dinosauria closed 5 years ago

Dinosauria commented 5 years ago

Description

Panic occured on dtle when execute create table XXX in 'scenario of reciprocating migration on the same mysql instance'

Steps to reproduce the issue:

  1. step 7 in test case DTLE-191

Describe the results you received:

Panic on dtle, exit dtle pid.

Describe the results you expected:

Output of ./dtle version:

Dtle 9.9.9.9 (git: master 5a894a13e03f87c03dd583d66ec9cabf96ae8999)

Additional information you deem important (e.g. issue happens only occasionally):

Happened in recently dtle versions. (The code change maybe after 06 Mar 2019 at 08:00:00 )

Additional details (log, config, job config etc):

  1. dtle.log: 
    
2019/03/10 18:56:33.346200 [DEBUG] [jobid:326d9a4d-53f6-b40d-8b6b-a8bf2cf9f996] applier. incr. ack-recv. nEntries: 1 
2019/03/10 18:56:33.346230 [DEBUG] [jobid:326d9a4d-53f6-b40d-8b6b-a8bf2cf9f996] mysql.applier: a binlogEntry. remaining: 0. gno: 3204, lc: 3203, seq: 3204 
2019/03/10 18:56:33.346240 [DEBUG] [jobid:326d9a4d-53f6-b40d-8b6b-a8bf2cf9f996] mysql.applier: skipping a dtle tx. osid: 1052fab4-4323-11e9-8ecc-0242ac640901 
2019/03/10 18:56:33.346421 [DEBUG] [jobid:326d9a4d-53f6-b40d-8b6b-a8bf2cf9f996] mysql.extractor: send acked gno: 3204, n: 1 
2019/03/10 18:56:33.354225 [DEBUG] [jobid:326d9a4d-53f6-b40d-8b6b-a8bf2cf9f996] mysql.reader: query event: schema: db3, query: CREATE TABLE sbtest1 (
id INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
k INTEGER UNSIGNED DEFAULT '0' NOT NULL,
c CHAR(120) DEFAULT '' NOT NULL,
pad CHAR(60) DEFAULT '' NOT NULL,
PRIMARY KEY (id)
) /*! ENGINE = innodb MAX_ROWS = 1000000 */ 
2019/03/10 18:56:33.354785 [DEBUG] [jobid:326d9a4d-53f6-b40d-8b6b-a8bf2cf9f996] mysql.reader: ddl is create table 
2019/03/10 18:56:33.354807 [WARN] [jobid:326d9a4d-53f6-b40d-8b6b-a8bf2cf9f996] error handle create table in binlog: GetTableColumns: table does not exists in sqle context. table: %!v(MISSING) 
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x873bd6]

goroutine 8666 [running]: github.com/actiontech/dtle/internal/config/mysql.(ColumnList).Names(0x0, 0xc420d8a3f0, 0x70, 0xc420d8a3f0) /universe/src/github.com/actiontech/dtle/internal/config/mysql/types.go:192 +0x26 github.com/actiontech/dtle/internal/config/mysql.(ColumnList).String(0x0, 0x1677ac4, 0x38) /universe/src/github.com/actiontech/dtle/internal/config/mysql/types.go:240 +0x2b github.com/actiontech/dtle/internal/client/driver/mysql/binlog.(BinlogReader).handleEvent.func1(0xc42132aa00, 0x164fb38) /universe/src/github.com/actiontech/dtle/internal/client/driver/mysql/binlog/binlog_reader.go:403 +0xe9 github.com/actiontech/dtle/internal/client/driver/mysql/binlog.(BinlogReader).handleEvent(0xc42163a300, 0xc42172d920, 0xc421349020, 0x0, 0x0) /universe/src/github.com/actiontech/dtle/internal/client/driver/mysql/binlog/binlog_reader.go:436 +0xb03 github.com/actiontech/dtle/internal/client/driver/mysql/binlog.(BinlogReader).DataStreamEvents(0xc42163a300, 0xc421349020, 0xc4202298c0, 0xc420a162a0) /universe/src/github.com/actiontech/dtle/internal/client/driver/mysql/binlog/binlog_reader.go:658 +0x213 github.com/actiontech/dtle/internal/client/driver/mysql.(Extractor).StreamEvents(0xc4202298c0, 0x1655068, 0x24) /universe/src/github.com/actiontech/dtle/internal/client/driver/mysql/extractor.go:804 +0x80 github.com/actiontech/dtle/internal/client/driver/mysql.(Extractor).initiateStreaming.func1(0xc4202298c0) /universe/src/github.com/actiontech/dtle/internal/client/driver/mysql/extractor.go:412 +0x66 created by github.com/actiontech/dtle/internal/client/driver/mysql.(Extractor).initiateStreaming /universe/src/github.com/actiontech/dtle/internal/client/driver/mysql/extractor.go:410 +0x3f

Dinosauria commented 5 years ago

Same thing happened on step7 of DTLE-7 log: dtle_panic_log.txt

Dinosauria commented 5 years ago

Same thing happened on step11 of DTLE-130 log: dtle_panic_log.txt

ghost commented 5 years ago

Simplified

  1. create a job with "ReplicateDoDb": []
  2. src: create schema a; use a; create table ...
  3. it crashes
Dinosauria commented 5 years ago

Do not panic again in test case DTLE-191 , DTLE-7 and DTLE-130