search

zendesk / maxwell

Maxwell's daemon, a mysql-to-json kafka producer
https://maxwells-daemon.io/
Other
4.05k stars 1.01k forks source link

Un-Graceful exit when encountering TRUNCATE statements #64

Closed akshayi1 closed 9 years ago

akshayi1 commented 9 years ago

I issued a truncate table statement and Maxwell didn't like it. To the extent that it crashed, and failed every time I tried to run it. I had to use the purge binary logs before now() and delete from maxwell.positions commands to get it running again. Here is the log:

TRUNCATE table error:

23:12:55,915 DEBUG SchemaChange - SQL_PARSE <- "truncate table max_test3"
line 1:0 no viable alternative at input 'truncate'
23:12:55,941 ERROR MysqlParserListener - (statement truncate table max_test3)
23:12:55,942 ERROR SchemaChange - Error parsing SQL: 'truncate table max_test3'
com.zendesk.maxwell.schema.ddl.MaxwellSQLSyntaxError: truncate
at com.zendesk.maxwell.schema.ddl.MysqlParserListener.visitErrorNode(MysqlParserListener.java:85)
at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:41)
at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
at com.zendesk.maxwell.schema.ddl.SchemaChange.parse(SchemaChange.java:65)
at com.zendesk.maxwell.MaxwellParser.processQueryEvent(MaxwellParser.java:206)
at com.zendesk.maxwell.MaxwellParser.getEvent(MaxwellParser.java:192)
at com.zendesk.maxwell.MaxwellParser.getEvent(MaxwellParser.java:198)
at com.zendesk.maxwell.MaxwellParser.run(MaxwellParser.java:98)
at com.zendesk.maxwell.Maxwell.run(Maxwell.java:69)
at com.zendesk.maxwell.Maxwell.main(Maxwell.java:75)
23:12:55,953 DEBUG SchemaPosition - Writing binlog position to maxwell.positions: BinlogPosition[mysql-logs.000003:6424]

Maxwell does not recover after truncate statement:

bin/maxwell --user='root' --password='root' --host='127.0.0.1' --producer=stdout
Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar 
23:17:15,992 INFO  Maxwell - Maxwell is booting, starting at BinlogPosition[mysql-logs.000003:6424]
23:17:15,993 DEBUG SchemaStore - looking to restore schema at target position BinlogPosition[mysql-logs.000003:6424]
23:17:15,995 INFO  SchemaStore - Restoring schema id 6 (last modified at BinlogPosition[mysql-logs.000003:330])
23:17:16,057 INFO  TransportImpl - connecting to host: 127.0.0.1, port: 3306
23:17:16,088 INFO  TransportImpl - connected to host: 127.0.0.1, port: 3306, context: AbstractTransport.Context[threadId=18,scramble=EDcI$uB*Gi[m4V5E>E}h,protocolVersion=10,serverHost=127.0.0.1,serverPort=3306,serverStatus=2,serverCollation=8,serverVersion=5.6.24-0ubuntu2-log,serverCapabilities=63487]
23:17:16,088 INFO  AuthenticatorImpl - start to login, user: root, host: 127.0.0.1, port: 3306
23:17:16,093 INFO  AuthenticatorImpl - login successfully, user: root, detail: OKPacket[packetMarker=0,affectedRows=0,insertId=0,serverStatus=2,warningCount=0,message=<null>]
23:17:16,226 DEBUG SchemaChange - SQL_PARSE <- "truncate table max_test3"
line 1:0 no viable alternative at input 'truncate'
23:17:16,274 ERROR MysqlParserListener - (statement truncate table max_test3)
23:17:16,283 ERROR SchemaChange - Error parsing SQL: 'truncate table max_test3'
com.zendesk.maxwell.schema.ddl.MaxwellSQLSyntaxError: truncate
at com.zendesk.maxwell.schema.ddl.MysqlParserListener.visitErrorNode(MysqlParserListener.java:85)
at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:41)
at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
at com.zendesk.maxwell.schema.ddl.SchemaChange.parse(SchemaChange.java:65)
at com.zendesk.maxwell.MaxwellParser.processQueryEvent(MaxwellParser.java:206)
at com.zendesk.maxwell.MaxwellParser.getEvent(MaxwellParser.java:192)
at com.zendesk.maxwell.MaxwellParser.getEvent(MaxwellParser.java:198)
at com.zendesk.maxwell.MaxwellParser.run(MaxwellParser.java:98)
at com.zendesk.maxwell.Maxwell.run(Maxwell.java:69)
at com.zendesk.maxwell.Maxwell.main(Maxwell.java:75)

Please add support for TRUNCATE statements, or at least ignore them (maybe not a great idea :)). But this kind of failure is no good. Thanks!

akshayi1 commented 9 years ago

68 fixes this. Thanks!