"hms-mirror" is a utility used to bridge the gap between two clusters and migrate hive metadata.
Apache License 2.0
13
stars
8
forks
source link
partitionDiscovery configuration setting "initMSCK: true" is not generating/running msck repair table scripts for partitioned tables when using -d SCHEMA_ONLY #99
We have are migrating the data between legacy CDH and CDP cluster using BDR.
We are using hms-mirror to migrate the HMS metadata to change the default external tablespace location from /user/hive/warehouse to /warehouse/tablespace/external/hive.
During testing of external table metadata migration using the following cli arguments we noticed that msck repair table was not running:
hms-mirror -d SCHEMA_ONLY -db ".*" --hadoop-classpath --reset-to-default-location --external-warehouse-directory /warehouse/tablespace/external/hive --warehouse-directory /warehouse/tablespace/managed/hive --distcp --accept --execute
Perhaps this is due to the fact that we are using -d SCHEMA_ONLY.
For our example, we'd like to have the msck repair table scripts produced and executed if running --execute.
Happy for this to be an option if it isn't deemed to fit with the -d SCHEMA_ONLY migration strategy.
dstreev
commented
8 months ago
We have are migrating the data between legacy CDH and CDP cluster using BDR. We are using hms-mirror to migrate the HMS metadata to change the default external tablespace location from /user/hive/warehouse to /warehouse/tablespace/external/hive. During testing of external table metadata migration using the following cli arguments we noticed that msck repair table was not running: hms-mirror -d SCHEMA_ONLY -db ".*" --hadoop-classpath --reset-to-default-location --external-warehouse-directory /warehouse/tablespace/external/hive --warehouse-directory /warehouse/tablespace/managed/hive --distcp --accept --execute
Perhaps this is due to the fact that we are using -d SCHEMA_ONLY. For our example, we'd like to have the msck repair table scripts produced and executed if running --execute. Happy for this to be an option if it isn't deemed to fit with the -d SCHEMA_ONLY migration strategy.