search

cloudera-labs / hms-mirror

"hms-mirror" is a utility used to bridge the gap between two clusters and migrate hive metadata.
Apache License 2.0
13 stars 8 forks source link

DISTCP options when there is only 1 source path in source file #105

Closed dstreev closed 3 months ago

dstreev commented 7 months ago

This doesn't append the last path and just merges in this to the root target. Which misses the 'table' level directory.

dstreev commented 3 months ago

Adjustments made here and an added option to control the level of distcp folder reduction. The new property: consolidate-tables-for-distcp (defaults to false), allows further directory reduction when 'true'. But that may include more directories than intended. Hence the 'false' default and behavioral change.