search

cloudera-labs / hms-mirror

"hms-mirror" is a utility used to bridge the gap between two clusters and migrate hive metadata.
Apache License 2.0
13 stars 8 forks source link

issue with alter table command which has invalid location i.e multiple s3a's #41

Closed hpasumarthi closed 1 year ago

hpasumarthi commented 1 year ago

Hello Team, Alter table command along with create database is having invalid locations. Logs are attached to the case. 954685

Sample job is here

hms-mirror -cfg uat-dev.yaml -d SQL -db phoenix_raw -tf fx_house_rates --db-prefix 'uat_' -o /tmp/hms-mirror-dist-latest/working-dir/sql_real_test_$(date +"%Y%m%d%H%M")/ -slc -rdl -wd s3a://S3bucket/data/warehouse/tablespace/managed/hive -ewd s3a://S3bucket/data/warehouse/tablespace/external/hive -e

RIGHT_execute.sql file comes with invalid location multiple times s3a

ALTER DATABASE uat_phoenix_raw SET LOCATION "s3a://S3bucket//s3a://S3bucket/data/warehouse/tablespace/external/hive/uat_phoenix_raw.db";

ALTER DATABASE uat_phoenix_raw SET MANAGEDLOCATION "s3a://S3bucket//s3a://S3bucket/data/warehouse/tablespace/managed/hive/uat_phoenix_raw.db";

Can we get this fixed. Regards, Hemanth

dstreev commented 1 year ago

When you specify the -ewd and -wd options, it prefixes the value with the 'namespace' element used in the cluster (RIGHT) configuration. Remove the prefix from the -ewd and -wd values.