search

actiontech / dtle

Distributed Data Transfer Service for MySQL
https://actiontech.github.io/dtle-docs-cn
Mozilla Public License 2.0
550 stars 132 forks source link

mapping mysql-mysql job: drop db cause panic #928

Closed asiroliu closed 2 years ago

asiroliu commented 2 years ago

Description

mapping mysql-mysql job: drop db cause panic

Steps to reproduce the issue

  1. create db on src MySQL 
    mysql> create database action_db_1;
  2. create mysql-mysql mapping job 
    {
"job_id": "mapping_rename_schema_1_db",
"is_password_encrypted": false,
"task_step_name": "all",
"failover": true,
"retry": 2,
"src_task": {
"task_name": "src",
"node_id": "412688f9-ab46-8692-4fcc-3d1102c2594a",
"mysql_src_task_config": {
  "gtid": "",
  "binlog_relay": false
},
"drop_table_if_exists": true,
"skip_create_db_table": false,
"repl_chan_buffer_size": 120,
"chunk_size": 2000,
"group_max_size": 1,
"group_timeout": 100,
"connection_config": {
  "database_type": "MySQL",
  "host": "172.100.9.3",
  "port": 3306,
  "user": "test_src",
  "password": "test_src"
},
"replicate_do_db": [
  {
    "table_schema": "action_db_1",
    "table_schema_rename": "action_db_1_rename"
  }
]
},
"dest_task": {
"task_name": "dest",
"node_id": "e032432d-8ca0-85b0-a5b7-7ec86b88e8e7",
"mysql_dest_task_config": {
  "use_my_sql_dependency": false,
  "dependency_history_size": 2500,
  "parallel_workers": 32
},
"connection_config": {
  "database_type": "MySQL",
  "host": "172.100.9.2",
  "port": 3306,
  "user": "test_dest",
  "password": "test_dest"
}
}
}
  3. drop database on src MySQ 
    mysql> drop database action_db_1;
  4. dest dtle panic 
    2022-03-29T16:24:00.621+0800 [DEBUG] client.driver_mgr.dtle: query event: driver=dtle job=mapping_rename_schema_1_db-migration query="DROP DATABASE IF EXISTS `action_db_1`" schema=action_db_1 @module=reader timestamp=2022-03-29T16:24:00.620+0800
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle: panic: runtime error: invalid memory address or nil pointer dereference: driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle: [signal SIGSEGV: segmentation violation code=0x1 addr=0x58 pc=0x2631f36]: driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle: : driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle: goroutine 4284 [running]:: driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle: github.com/actiontech/dtle/drivers/mysql/mysql/binlog.(*BinlogReader).handleQueryEvent(0xc003cefc80, 0xc003e02930, 0xc000a18420, 0xc002441f40, 0xc00394a800): driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle:    /universe/src/github.com/actiontech/dtle/drivers/mysql/mysql/binlog/binlog_reader.go:680 +0x15b6: driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle: github.com/actiontech/dtle/drivers/mysql/mysql/binlog.(*BinlogReader).handleEvent(0xc003cefc80, 0xc003e02930, 0xc000a18420, 0x0, 0x0): driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle:    /universe/src/github.com/actiontech/dtle/drivers/mysql/mysql/binlog/binlog_reader.go:453 +0x54d: driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle: github.com/actiontech/dtle/drivers/mysql/mysql/binlog.(*BinlogReader).DataStreamEvents(0xc003cefc80, 0xc000a18420, 0xc000fed6c0, 0x10000c000000003): driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle:    /universe/src/github.com/actiontech/dtle/drivers/mysql/mysql/binlog/binlog_reader.go:961 +0x68a: driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle: github.com/actiontech/dtle/drivers/mysql/mysql.(*Extractor).StreamEvents(0xc000fed6c0, 0x30b6e91, 0x13): driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle:    /universe/src/github.com/actiontech/dtle/drivers/mysql/mysql/extractor.go:1065 +0x88: driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle: github.com/actiontech/dtle/drivers/mysql/mysql.(*Extractor).initiateStreaming.func1(0xc000fed6c0): driver=dtle
2022-03-29T16:24:00.623+0800 [DEBUG] client.driver_mgr.dtle:    /universe/src/github.com/actiontech/dtle/drivers/mysql/mysql/extractor.go:684 +0x90: driver=dtle
2022-03-29T16:24:00.624+0800 [DEBUG] client.driver_mgr.dtle: created by github.com/actiontech/dtle/drivers/mysql/mysql.(*Extractor).initiateStreaming: driver=dtle
2022-03-29T16:24:00.624+0800 [DEBUG] client.driver_mgr.dtle:    /universe/src/github.com/actiontech/dtle/drivers/mysql/mysql/extractor.go:681 +0x67: driver=dtle
2022-03-29T16:24:00.634+0800 [DEBUG] client.driver_mgr.stdio: received EOF, stopping recv loop: driver=dtle err="rpc error: code = Unavailable desc = transport is closing"
2022-03-29T16:24:00.634+0800 [DEBUG] client.driver_mgr: plugin process exited: driver=dtle path=/opt/dtle/usr/share/dtle/nomad-plugin/dtle pid=325 error="exit status 2"
2022-03-29T16:24:00.634+0800 [ERROR] client.driver_mgr: error receiving stream from TaskStats driver RPC, closing stream: driver=dtle error="rpc error: code = Unavailable desc = transport is closing"
2022-03-29T16:24:00.634+0800 [DEBUG] client.alloc_runner.task_runner: driver plugin has shutdown; attempting to recover task: alloc_id=bda16db7-5b67-a54e-67db-cf61f106946b task=src driver=dtle
2022-03-29T16:24:00.634+0800 [WARN]  client.driver_mgr: failed to reattach to plugin, starting new instance: driver=dtle err="singleton plugin exited"
2022-03-29T16:24:00.634+0800 [WARN]  client.driver_mgr: received fingerprint error from driver: driver=dtle error="plugin is shut down"

Output of ./dtle version:**

9.9.9.9-master-1e50c92
asiroliu commented 2 years ago

verify version： 9.9.9.9-master-4b3624c