search

eselyavka / liquibase-impala

Liquibase extension to add Impala Database support
Apache License 2.0
24 stars 18 forks source link

Error truncating DATABASECHANGELOGLOCK table in Hive, when running as a maven plugin #2

Closed turu closed 7 years ago

turu commented 7 years ago

Hi,

First of all I would like to say well done to you for creating this extension - it's a great step forward in automating db management in Hadoop warehousing world.

I've been experimenting a bit with liquibase-impala on our testing cluster and I bumped into a following issue:

When running liquibase for Hive as a maven plugin - via mvn liquibase:do_sth (which may or may not have something to do with my issue), liquibase fails to execute liquibase:update goal with locking turned on, due to issuing a malformed TRUNCATE sql statement on databasechangeloglock table. TABLE keyword is missing after TRUNCATE. The following message appears: [Cloudera][HiveJDBCDriver](500051) ERROR processing query/statement. Error Code: 40000, SQL state: TStatus(statusCode:ERROR_STATUS, infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Error while compiling statement: FAILED: ParseException line 1:9 missing TABLE at 'DATABASECHANGELOGLOCK' near '<EOF>':17:16,

This is due to the fact that TruncateGenerator is selected instead of HiveTruncateGenerator, despite both of them supporting that particular call.

Closer inspection reveals that indeed the TABLE keyword is missing from TruncateGenerator. See https://github.com/eselyavka/liquibase-impala/blob/99c46ef487a4714f5cc02d7df6a4851b5efe3d83/src/main/java/liquibase/ext/metastore/sqlgenerator/TruncateGenerator.java#L39

A quick fix could be to add the missing keyword to TruncateGenerator. It shouldn't break anything since the extended syntax of 'TRUNCATE TABLE' is supported by both Hive and Impala. However, perhaps the real fix would be to adjust priorities of these generators or even remove HiveTruncateGenerator altogether after applying that SQL syntax fix? I can submit a pr, when we agree on the right way to resolve this issue.

What do you think @eselyavka ?

eselyavka commented 7 years ago

Hi, Thanks to finding my plugin useful, feel free to open any PR which you think improve the plugin itself. I am totally agree to delete HiveTruncateGenerator and add missing keyword to TruncateGenerator.