search

databrickslabs / migrate

Old scripts for one-off ST-to-E2 migrations. Use "terraform exporter" linked in the readme.
Other
186 stars 128 forks source link

Error while importing into my new Azure databricks workspace #260

Closed kishoresmeda closed 1 year ago

kishoresmeda commented 1 year ago

I've done an export of a old workspace from Azure. While importing the same in the new account, i see this error. Not sure how source_info.txt gets created. Any help will be most appreciated. @gregwood-db

(migrate) @abcdef ➜ /workspaces/migrate (master) $ python migration_pipeline.py --profile newWS --azure --import-pipeline --use-checkpoint Using the session id: M20230413044012 Traceback (most recent call last): File "migration_pipeline.py", line 349, in main() File "migration_pipeline.py", line 344, in main pipeline = build_pipeline(args) File "migration_pipeline.py", line 75, in build_pipeline return build_import_pipeline(client_config, checkpoint_service, args) File "migration_pipeline.py", line 131, in build_import_pipeline with open(source_info_file, 'r') as f: FileNotFoundError: [Errno 2] No such file or directory: 'azure_logs/M20230413044012/source_info.txt'

kishoresmeda commented 1 year ago

Similar to updating AWS account id in readme section, is there anything to do with Azure account? If yes, please suggest the steps. Thanks in advance.

veenaramesh commented 1 year ago

source_info.txt is just a file that contains the original URL. So, in order to continue with import you can create a txt file that contains that URL. It should proceed with the import pipeline without an issue.

As for the reason this happened, did you use the same session/checkpoint to export everything via the migration_pipeline.py?

kishoresmeda commented 1 year ago

Hi @veenaramesh. Thanks for your response. I shall create the file and let u know how it goes.

For your second question, I've a session id that was created during the export process. i didn't use this session id for my import and just left it blank so got a new session id(import) created. Are you suggesting that i should use the same session id (from my export) when i'm doing the import?

I will try it anyway and let u know the result. Thanks Veena.

kishoresmeda commented 1 year ago

Hi @veenaramesh I think i got you right. I had to use the same session id from the export step during my import. It worked!

Maybe it should have been in the documentation (Readme file) to avoid this confusion. I now know it makes sense use the same session for both export and import but for someone just starting out, making this clear in the first place would be much helpful.

I can't thank you enough for your help in opening my eye.