cloudera-labs / hms-mirror

"hms-mirror" is a utility used to bridge the gap between two clusters and migrate hive metadata.
Apache License 2.0
13 stars 8 forks source link

right table location is not getting updated with arguments db-prefix and db-rename #39

Closed hpasumarthi closed 1 year ago

hpasumarthi commented 1 year ago

Hi David, We have been testing db-prefix and db-rename options and right table location do not have the new db name.

our sample command is

hms-mirror -cfg uat-dev.yaml -d SQL -db phoenix_raw -tf fx_house_rates --db-rename 'uat_phoenix_raw' -o /tmp/hms-mirror-dist-latest/working-dir/sql_real_test1/ -slc -sql

hms-mirror -cfg uat-dev.yaml -d SQL -db phoenix_raw -tf fx_house_rates --db-prefix 'uat_' -o /tmp/hms-mirror-dist-latest/working-dir/sql_real_test/ -slc -sql Can we get this fixed ?

dstreev commented 1 year ago

In each of these cases, the database new is created with the appropriate 'new/altered' name. I think you're referring to the location element of the table create.

hms-mirror uses a literal translation (minus the namespace) when doing the transfers. And since we can't rely on the 'db' name in the path being accurate, we can't make a assumption to make an adjustment here.

But there is an option. Use the -rdl|--reset-default-location along with -ewd|--external-warehouse-directory and -wd|--warehouse-directory settings to control where the new data goes.

For example: hms-mirror -cfg uat-dev.yaml -d SQL -db phoenix_raw -tf fx_house_rates --db-prefix 'uat_' -o /tmp/hms-mirror-dist-latest/working-dir/sql_real_test/ -slc -rdl -wd hdfs://new_namespace/new_root_mngd_location -ewd hdfs://new_namespace/new_root_ext_location

Note: The -sql option has been deprecated.