search

zendesk / maxwell

Maxwell's daemon, a mysql-to-json kafka producer
https://maxwells-daemon.io/
Other
3.95k stars 996 forks source link

Add `tablespace` token to accept backtick'd names #2076

Closed a-bates closed 4 months ago

a-bates commented 5 months ago

Versions:

On the connected mysql server:

CREATE TABLE f (i int not null, primary key (i)) TABLESPACE `innodb_system`;

Results in the following error in Maxwell:

2024-01-25 15:00:27 ERROR MysqlParserListener - (creation_tablespace TABLESPACE `innodb_system`)
2024-01-25 15:00:27 ERROR SchemaChange - Error parsing SQL: 'CREATE TABLE f (i int not null, primary key (i)) TABLESPACE `innodb_system`'
2024-01-25 15:00:27 ERROR AbstractSchemaStore - Error on bin log position Position[BinlogPosition[mysql-bin.000028:3010267], lastHeartbeat=1706220019712]
com.zendesk.maxwell.schema.ddl.MaxwellSQLSyntaxError: `innodb_system`
    at com.zendesk.maxwell.schema.ddl.MysqlParserListener.visitErrorNode(MysqlParserListener.java:93)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:17)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
    at com.zendesk.maxwell.schema.ddl.SchemaChange.parseSQL(SchemaChange.java:101)
    at com.zendesk.maxwell.schema.ddl.SchemaChange.parse(SchemaChange.java:115)
    at com.zendesk.maxwell.schema.AbstractSchemaStore.resolveSQL(AbstractSchemaStore.java:49)
    at com.zendesk.maxwell.schema.MysqlSchemaStore.processSQL(MysqlSchemaStore.java:102)
    at com.zendesk.maxwell.replication.BinlogConnectorReplicator.processQueryEvent(BinlogConnectorReplicator.java:385)
    at com.zendesk.maxwell.replication.BinlogConnectorReplicator.processQueryEvent(BinlogConnectorReplicator.java:407)
    at com.zendesk.maxwell.replication.BinlogConnectorReplicator.getRow(BinlogConnectorReplicator.java:738)
    at com.zendesk.maxwell.replication.BinlogConnectorReplicator.work(BinlogConnectorReplicator.java:235)
    at com.zendesk.maxwell.util.RunLoopProcess.runLoop(RunLoopProcess.java:34)
    at com.zendesk.maxwell.Maxwell.startInner(Maxwell.java:302)
    at com.zendesk.maxwell.Maxwell.start(Maxwell.java:227)
    at com.zendesk.maxwell.Maxwell.main(Maxwell.java:337)

Tablespace names, like table names, can be wrapped in backticks. The parser wasn't properly handling allowing backtick'd names.

We found that using MySQL 8 and Percona-Toolkit's pt-online-schema-change to perform table migrations, the CREATE TABLE statements that are generated for the _table_new table include /*!50100 TABLESPACE `innodb_system` */ which will throw the above error.

This could to be related to this MySQL bug where the above will be inserted when the innodb_file_per_table=0 is set and the table has received any alterations after creation. Without this option set, running pt-online-schema-change does not insert the failing clause.

osheroff commented 4 months ago

thx!