mrchristine / db-migration

Databricks Migration Tools
Other
43 stars 27 forks source link

databases migrated failed to inherit path / location from source workspace #56

Closed gobiviswanath closed 3 years ago

gobiviswanath commented 3 years ago

Source : image

destination

image

mrchristine commented 3 years ago

Confirmed that this works now on my test environment. We now export the database properties into a custom log file, and during replay check if this deviates from the default dbfs:/user/hive/warehouse/ path, if so then re-create with the custom path.

gobiviswanath commented 3 years ago

receive error:

Get: https://adb-5985677774820244.4.azuredatabricks.net/api/1.2/commands/status Database: default Database: deltadb2 Database: deltadb3 post: https://adb-5985677774820244.4.azuredatabricks.net/api/1.2/commands/execute Get: https://adb-5985677774820244.4.azuredatabricks.net/api/1.2/commands/status ERROR: IndentationError: unexpected indent {'resultType': 'error', 'summary': 'IndentationError: unexpected indent', 'cause': 'File "", line 2\n import json\n ^\nIndentationError: unexpected indent'} post: https://adb-5985677774820244.4.azuredatabricks.net/api/1.2/commands/execute Get: https://adb-5985677774820244.4.azuredatabricks.net/api/1.2/commands/status ERROR: NameError: name 'get_db_json' is not defined {"resultType": "error", "summary": "<span class=\"ansi-red-fg\">NameError: name 'get_db_json' is not defined", "cause": "---------------------------------------------------------------------------\nNameError Traceback (most recent call last)\n in \n----> 1 print(get_db_json(\"default\"))\n\nNameError: name 'get_db_json' is not defined"}

Traceback (most recent call last): File "/Users/gobinath/migrate/export_db.py", line 219, in main() File "/Users/gobinath/migrate/export_db.py", line 172, in main hive_c.export_hive_metastore(cluster_name=args.cluster_name, has_unicode=args.metastore_unicode) File "/Users/gobinath/migrate/dbclient/HiveClient.py", line 141, in export_hive_metastore db_json = self.get_desc_database_details(db_name, cid, ec_id) File "/Users/gobinath/migrate/dbclient/HiveClient.py", line 96, in get_desc_database_details raise ValueError("Desc database extended failure") ValueError: Desc database extended failure (base) C02Y77B9JG5H:migrate gobinath$