fastsql 2.0.0_preview_644在解析ddl报错

[spccold]

ddl:

create table test(id int signed)

在mysql上执行时没问题的

exception:

Exception in thread "main" com.alibaba.fastsql.sql.parser.ParserException: syntax error, error in :' int signed)', expect ), actual null, pos 31, line 1, column 26, token IDENTIFIER signed
    at com.alibaba.fastsql.sql.parser.SQLParser.printError(SQLParser.java:363)
    at com.alibaba.fastsql.sql.parser.SQLParser.accept(SQLParser.java:371)
    at com.alibaba.fastsql.sql.dialect.mysql.parser.MySqlCreateTableParser.parseCreateTable(MySqlCreateTableParser.java:418)
    at com.alibaba.fastsql.sql.dialect.mysql.parser.MySqlStatementParser.parseCreate(MySqlStatementParser.java:265)
    at com.alibaba.fastsql.sql.parser.SQLStatementParser.parseStatementList(SQLStatementParser.java:224)
    at com.alibaba.fastsql.sql.SQLUtils.parseStatements(SQLUtils.java:525)
    at com.alibaba.fastsql.sql.repository.SchemaRepository.console(SchemaRepository.java:439)

[spccold]

DruidDdlParser.parse("ALTER TABLE test2 DROP INDEX name_indx, DEFAULT CHARACTER SET DEFAULT", "test2");

报错如下:

18:50:54.476 [main] ERROR c.a.o.c.p.i.mysql.ddl.DruidDdlParser - parse query event failed, queryString:ALTER TABLE `test2` DROP INDEX `name_indx`, DEFAULT CHARACTER SET DEFAULT
com.alibaba.fastsql.sql.parser.ParserException: syntax error, error in :' SET DEFAULT', expect =, actual DEFAULT, pos 73, line 1, column 67, token DEFAULT
    at com.alibaba.fastsql.sql.parser.SQLParser.printError(SQLParser.java:363) ~[fastsql-2.0.0_preview_644.jar:2.0.0_preview_644]
    at com.alibaba.fastsql.sql.parser.SQLParser.accept(SQLParser.java:371) ~[fastsql-2.0.0_preview_644.jar:2.0.0_preview_644]
    at com.alibaba.fastsql.sql.dialect.mysql.parser.MySqlStatementParser.alterTableCharacter(MySqlStatementParser.java:5415) ~[fastsql-2.0.0_preview_644.jar:2.0.0_preview_644]
    at com.alibaba.fastsql.sql.dialect.mysql.parser.MySqlStatementParser.parseAlterTable(MySqlStatementParser.java:4782) ~[fastsql-2.0.0_preview_644.jar:2.0.0_preview_644]
    at com.alibaba.fastsql.sql.dialect.mysql.parser.MySqlStatementParser.parseAlter(MySqlStatementParser.java:3894) ~[fastsql-2.0.0_preview_644.jar:2.0.0_preview_644]
    at com.alibaba.fastsql.sql.parser.SQLStatementParser.parseStatementList(SQLStatementParser.java:271) ~[fastsql-2.0.0_preview_644.jar:2.0.0_preview_644]
    at com.alibaba.fastsql.sql.parser.SQLStatementParser.parseStatementList(SQLStatementParser.java:93) ~[fastsql-2.0.0_preview_644.jar:2.0.0_preview_644]
    at com.alibaba.fastsql.sql.SQLUtils.parseStatements(SQLUtils.java:534) ~[fastsql-2.0.0_preview_644.jar:2.0.0_preview_644]
    at com.alibaba.otter.canal.parse.inbound.mysql.ddl.DruidDdlParser.parse(DruidDdlParser.java:55) ~[classes/:na]
    at canal.DdlParserApp.main(DdlParserApp.java:12) [classes/:na]

@agapple 这个已经影响了tsdb功能, 帮忙看看

[agapple]

估计是后半段的SQL

[spccold]

是的，这个是我们日常环境遇到的ddl，mysql本身执行没啥问题

[spccold]

表结构如下:

create table test(
    id int primary key auto_increment,
    name varchar(20),
    key name_index(name)
)

执行如下ddl

1. alter table test ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
2. alter table test ROW_FORMAT=compact KEY_BLOCK_SIZE=0

DruidDdlParser在解析上面两个ddl均会失败, 但是目前也没太大的负面影响, 错误如下

com.alibaba.fastsql.sql.parser.ParserException: syntax error, error in :'SSED KEY_BLOCK_SIZE=8, pos 62, line 1, column 63, token IDENTIFIER KEY_BLOCK
_SIZE
        at com.alibaba.fastsql.sql.parser.SQLParser.printError(SQLParser.java:363)
        at com.alibaba.fastsql.sql.parser.SQLStatementParser.parseStatementList(SQLStatementParser.java:534)
        at com.alibaba.fastsql.sql.parser.SQLStatementParser.parseStatementList(SQLStatementParser.java:93)
        at com.alibaba.fastsql.sql.SQLUtils.parseStatements(SQLUtils.java:534)
        at com.alibaba.otter.canal.parse.inbound.mysql.ddl.DruidDdlParser.parse(DruidDdlParser.java:55)
        at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.parseQueryEvent(LogEventConvert.java:163)
        at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.parse(LogEventConvert.java:107)
        at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.parse(LogEventConvert.java:67)
        at com.alibaba.otter.canal.parse.inbound.AbstractEventParser.parseAndProfilingIfNecessary(AbstractEventParser.java:348)
        at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$3$1.sink(AbstractEventParser.java:195)
        at com.alibaba.otter.canal.parse.inbound.mysql.MysqlConnection.dump(MysqlConnection.java:134)
        at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$3.run(AbstractEventParser.java:231)

com.alibaba.fastsql.sql.parser.ParserException: syntax error, error in :'pact KEY_BLOCK_SIZE=0, pos 59, line 1, column 60, token IDENTIFIER KEY_BLOCK_SIZE
        at com.alibaba.fastsql.sql.parser.SQLParser.printError(SQLParser.java:363)
        at com.alibaba.fastsql.sql.parser.SQLStatementParser.parseStatementList(SQLStatementParser.java:534)
        at com.alibaba.fastsql.sql.parser.SQLStatementParser.parseStatementList(SQLStatementParser.java:93)
        at com.alibaba.fastsql.sql.SQLUtils.parseStatements(SQLUtils.java:534)
        at com.alibaba.otter.canal.parse.inbound.mysql.ddl.DruidDdlParser.parse(DruidDdlParser.java:55)
        at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.parseQueryEvent(LogEventConvert.java:163)
        at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.parse(LogEventConvert.java:107)
        at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.parse(LogEventConvert.java:67)
        at com.alibaba.otter.canal.parse.inbound.AbstractEventParser.parseAndProfilingIfNecessary(AbstractEventParser.java:348)
        at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$3$1.sink(AbstractEventParser.java:195)
        at com.alibaba.otter.canal.parse.inbound.mysql.MysqlConnection.dump(MysqlConnection.java:134)
        at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$3.run(AbstractEventParser.java:231)

此时show create table test结果如下:

CREATE TABLE `test` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `name` varchar(20) DEFAULT NULL,
  PRIMARY KEY (`id`) KEY_BLOCK_SIZE=8,
  KEY `name_index` (`name`) KEY_BLOCK_SIZE=8
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=COMPACT

DatabaseTableMeta#compareTableMetaDbAndMemory 会执行show create table test并把得到的建表语句执行MemoryTableMeta#apply, 但是fastsql无法识别上面的ddl，依然报错，会继续导致NPE， 错误如下

com.alibaba.fastsql.sql.parser.ParserException: syntax error, error in :'PRIMARY KEY (`id`) KEY_BLOCK_SIZE=8,
  KEY `idx_r', expect ), actual null, pos 2411, line 42, column 23, token IDENTIFIER KEY_BLOCK_SIZE
        at com.alibaba.fastsql.sql.parser.SQLParser.printError(SQLParser.java:363)
        at com.alibaba.fastsql.sql.parser.SQLParser.accept(SQLParser.java:371)
        at com.alibaba.fastsql.sql.dialect.mysql.parser.MySqlCreateTableParser.parseCreateTable(MySqlCreateTableParser.java:418)
        at com.alibaba.fastsql.sql.dialect.mysql.parser.MySqlStatementParser.parseCreate(MySqlStatementParser.java:265)
        at com.alibaba.fastsql.sql.parser.SQLStatementParser.parseStatementList(SQLStatementParser.java:224)
        at com.alibaba.fastsql.sql.SQLUtils.parseStatements(SQLUtils.java:525)
        at com.alibaba.fastsql.sql.repository.SchemaRepository.console(SchemaRepository.java:439)
        at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta.apply(MemoryTableMeta.java:78)
        at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.TableMetaCache.parseTableMeta(TableMetaCache.java:101)
        at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.compareTableMetaDbAndMemory(DatabaseTableMeta.java:352)
        at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.applySnapshotToDB(DatabaseTableMeta.java:294)
        at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.access$100(DatabaseTableMeta.java:46)
        at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta$2.run(DatabaseTableMeta.java:91)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
11:52:09.697 [[scheduler-table-meta-snapshot]] ERROR c.a.o.c.p.i.m.t.DatabaseTableMeta - scheudle applySnapshotToDB faield
java.lang.NullPointerException: null
        at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.TableMetaCache.parseTableMeta(TableMetaCache.java:103)
        at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.compareTableMetaDbAndMemory(DatabaseTableMeta.java:352)
        at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.applySnapshotToDB(DatabaseTableMeta.java:294)
        at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.access$100(DatabaseTableMeta.java:46)
        at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta$2.run(DatabaseTableMeta.java:91)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

总结起来看，就是目前fastsql目前貌似无法识别KEY_BLOCK_SIZE

[agapple]

ALTER TABLE test2 DROP INDEX name_indx, DEFAULT CHARACTER SET = DEFAULT，这样的语法可以解析通过

[agapple]

1.1.4 当前主干已修复大部分DDL.

create table test(id int signed)

这个在MySQL DDL语法文档里也没描述，你们线上有这用法？

[cvz6]

com.alibaba.fastsql.sql.parser.ParserException: syntax error, error in :'ort` COLLATE 'utf8mb4_general_ci', pos 42, line 1, column 44, token IDENTIFIER utf8mb4_general_ci