polardb / polardbx-sql

PolarDB-X is a cloud native distributed SQL Database designed for high concurrency, massive storage, complex querying scenarios.
Apache License 2.0
1.54k stars 325 forks source link

mysqlbinlog基于cdc全局binlog恢复报Not supported variable for now 'pseudo_thread_id' #64

Open wuxuguang2022 opened 2 years ago

wuxuguang2022 commented 2 years ago

用mysqlbinlog基于全局binlog恢复时报错,之后的insert 表数据sql未执行,数据不能恢复,示例: mysql -f -hxxx -Pxxx -upolardbx_root -pxxx testdb<testdb.sql ERROR 1193 (HY000) at line 14: [147a79b9c8003000][192.168.110.21:9540][testdb]Not supported variable for now 'pseudo_thread_id' ERROR 3009 (HY000) at line 18: [147a79b9cac03000][192.168.110.21:9540][testdb]ERR-CODE: [PXC-4500][ERR_PARSER] statement com.alibaba.polardbx.druid.sql.dialect.mysql.ast.statement.MySqlHintStatement not supported

testdb.sql截取后的前部份内容如下(mysqlbinlog -v --base64-output=decode-rows --skip-gtids=true -d testdb --start-position=2907258 --stop-position=3243742 /tmp/binlog.000001 > testdb.sql): /!50616 SET @@SESSION.GTID_NEXT='AUTOMATIC'//!/;

at 2906355

220615 13:09:49 server id 1842087586 end_log_pos 2906999 CRC32 0x5afb9f3c Query thread_id=1 exec_time=1655432947 error_code=0

use testdb/!/; SET TIMESTAMP=1655269789/!/; SET @@session.pseudo_thread_id=1/!/; SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/!/; SET @@session.sql_mode=1142947872/!/; SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/!/; /!\C utf8mb3 //!/; SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=33/!/; SET @@session.time_zone='SYSTEM'/!/; SET @@session.lc_time_names=0/!/; SET @@session.collation_database=DEFAULT/!/;

POLARX_ORIGIN_SQL=CREATE TABLE t1 ( id bigint(11) NOT NULL AUTO_INCREMENT, name varchar(255) DEFAULT NULL, score bigint(11) DEFAULT NULL, PRIMA

RY KEY (id) ) ENGINE = InnoDB DEFAULT CHARSET = utf8 PARTITION BY HASH (id) PARTITIONS 9# POLARX_TSO=694270470023177836814737699134400184320000000000000000 create table t1 ( id bigint(11) not null auto_increment, name varchar(255) default null, score bigint(11) default null, primary key (id) ) engi ne = innodb default charset = utf8 default character set = utf8 default collate = utf8_general_ci/!/;

at 2906999

at 2907082

220615 13:09:49 server id 1842087586 end_log_pos 2907137 CRC32 0x4e1bf385 Query thread_id=0 exec_time=0 error_code=0

SET TIMESTAMP=1655269789/!/; BEGIN /!/;

at 2907137

at 2907207

220615 13:09:49 server id 1842087586 end_log_pos 2907258 CRC32 0x010dbfa8 Table_map: testdb.t1 mapped to number 5

at 2907258

220615 13:09:49 server id 1842087586 end_log_pos 2907315 CRC32 0x8eae71f8 Write_rows: table id 5 flags: STMT_END_F

INSERT INTO testdb.t1

"testdb.sql" 4358L, 65328C

agapple commented 2 years ago

mysqlbinlog工具解析一半的binlog数据文件,目前暂不支持直接做source导入

需要先用binlog to sql的工具,先转成完整的sql语句后导入polardb-x

lulu2panpan commented 2 years ago

mysqlbinlog解析出的内容,需要单独进行协议和语法层面的适配,暂时不支持类似原生mysql的直接导入能力,参见https://dev.mysql.com/doc/refman/8.0/en/binlog.html

lulu2panpan commented 2 years ago

分析了一下,--base64-output=decode-rows参数还是不能加的,通过这个参数decode出来的sql都是带注释的,进行导入的时候会被忽略掉,相当于啥都没做。原生导入能力正在规划中,暂时只能靠工具进行sql转换,再导入了

wuxuguang2022 commented 2 years ago

那如果不加--base64-output=decode-rows参数呢

------------------ 原始邮件 ------------------ 发件人: "ApsaraDB/galaxysql" @.>; 发送时间: 2022年6月21日(星期二) 中午12:57 @.>; @.**@.>; 主题: Re: [ApsaraDB/galaxysql] mysqlbinlog基于cdc全局binlog恢复报Not supported variable for now 'pseudo_thread_id' (Issue #64)

分析了一下,--base64-output=decode-rows参数还是不能加的,通过这个参数decode出来的sql都是带注释的,进行导入的时候会被忽略掉,相当于啥都没做。原生导入能力正在规划中,暂时只能靠工具进行sql转换,再导入了

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

lulu2panpan commented 2 years ago

见上面回复内容,不加的话,解析出来的内容,需要内核尽心独立识别和适配,暂时还不支持