cloudera-labs / hms-mirror

"hms-mirror" is a utility used to bridge the gap between two clusters and migrate hive metadata.
Apache License 2.0
13 stars 8 forks source link

Extend Non-Standard Locations to the STORAGE_MIGRATION strategy. Ref Issue #61 #64

Closed dstreev closed 1 year ago

dstreev commented 1 year ago

The default STORAGE_MIGRATION process uses SQL to rename, create new, and move data. Since the -epl process was originally done for NON data movement strategies because the movement of data with Hive would reorganize the data into standard locations, we can't move data under this scenario.

The process will need to rename the original table, create the new one in the new location, and build out the ALTER PARTITION STATEMENTS to align. Then build the distcp plan for the transfer. We need to account for the location in the partition locations too.