search

databrickslabs / migrate

Old scripts for one-off ST-to-E2 migrations. Use "terraform exporter" linked in the readme.
Other
186 stars 128 forks source link

empty ACL file trying to export metastore_table_acls #299

Closed snowch closed 7 months ago

snowch commented 7 months ago

I'm trying to migrate metastore_table_acls on Azure Databricks workspace as per the following, but the exported ACL file is empty:

xxxx@xxxx:~/migrate$ python3 ./migration_pipeline.py --export-pipeline --profile oldWS --azure --set-export-dir /home/bear/export --skip-tasks 'instance_profiles' 'users' 'groups' 'workspace_item_log' 'workspace_acls' 'notebooks' 'secrets' 'clusters' 'instance_pools' 'jobs' 'metastore' 'metastore_table_acls' 'mlflow_experiments' 'mlflow_runs' --keep-tasks 'metastore_table_acls'
Using the session id: M20240329131637
2024-03-29,13:16:37;INFO;Start export_instance_profiles
2024-03-29,13:16:37;INFO;export_instance_profiles Skipped.
2024-03-29,13:16:37;INFO;Start export_users
2024-03-29,13:16:37;INFO;export_users Skipped.
2024-03-29,13:16:37;INFO;Start export_groups
2024-03-29,13:16:37;INFO;export_groups Skipped.
2024-03-29,13:16:37;INFO;Start export_workspace_items_log
2024-03-29,13:16:37;INFO;export_workspace_items_log Skipped.
2024-03-29,13:16:37;INFO;Start export_workspace_acls
2024-03-29,13:16:37;INFO;export_workspace_acls Skipped.
2024-03-29,13:16:37;INFO;Start export_notebooks
2024-03-29,13:16:37;INFO;export_notebooks Skipped.
2024-03-29,13:16:37;INFO;Start export_secrets
2024-03-29,13:16:37;INFO;export_secrets Skipped.
2024-03-29,13:16:37;INFO;Start export_clusters
2024-03-29,13:16:37;INFO;export_clusters Skipped.
2024-03-29,13:16:37;INFO;Start export_instance_pools
2024-03-29,13:16:37;INFO;export_instance_pools Skipped.
2024-03-29,13:16:37;INFO;Start export_jobs
2024-03-29,13:16:37;INFO;export_jobs Skipped.
2024-03-29,13:16:37;INFO;Start export_metastore
2024-03-29,13:16:37;INFO;export_metastore Skipped.
2024-03-29,13:16:37;INFO;Start export_metastore_table_acls
Starting API_Table_ACL_Work_Leave_Me_Alone with id xxxx
Cluster state: PENDING
Cluster state: PENDING
Cluster state: PENDING
Cluster state: PENDING
Cluster state: RUNNING
{
    "run_id": 26866
}
We leave the cluster running, in case you needed again: cluster_id: xxxx
ERROR : Notebook run failed: https://adb-xxxx.azuredatabricks.net/?o=xxxx#job/xxxx/run/26866
        ... http_status code 200 
        ... notebook execution result_state:  FAILED
Internal Notebook error, while executing  ACL Export
2024-03-29,13:20:50;INFO;export_metastore_table_acls Completed. Total time taken: 0:04:13.669964
2024-03-29,13:20:50;INFO;Start finish_export
Export finished successfully. Session Id: M20240329131637
2024-03-29,13:20:50;INFO;finish_export Completed. Total time taken: 0:00:00.000115

Exported ACL file is Empty.

mrchristine commented 7 months ago

You'll need to investigate the job failure at your job run https://adb-xxxx.azuredatabricks.net/?o=xxxx#job/xxxx/run/26866 This will indicate whether the default ACL cluster is configured correctly. If not, specify a cluster to export the ACLs yourself using the --cluster-name option.