search

databrickslabs / migrate

Old scripts for one-off ST-to-E2 migrations. Use "terraform exporter" linked in the readme.
Other
186 stars 127 forks source link

export a single users workspace fails with exception #119

Closed paulg-mscience closed 2 years ago

paulg-mscience commented 2 years ago

When I run a command similar to this:

python export_db.py --profile DEMO --export-home example@foobar.com

I get the following exception:

Traceback (most recent call last):
  File "export_db.py", line 308, in <module>
    main()
  File "export_db.py", line 262, in main
    ws_c.export_user_home(username, 'user_exports')
  File "/home/ec2-user/github/databricks-migrate/dbclient/WorkspaceClient.py", line 181, in export_user_home
    workspace_log_writer = ThreadSafeWriter(self.get_export_dir() + 'user_workspace.log', "a")
  File "/home/ec2-user/github/databricks-migrate/thread_safe_writer.py", line 17, in __init__
    self.filewriter = open(*args)
FileNotFoundError: [Errno 2] No such file or directory: 'logs/user_exports/example@foobar.com/user_workspace.log'
mrchristine commented 2 years ago

@kevinkim9264 @souravkhandelwal09 can we take a look at this?

kevinkim9264 commented 2 years ago

@paulg-mscience you can try this https://github.com/databrickslabs/migrate/pull/120

paulg-mscience commented 2 years ago

@kevinkim9264 You fixed the first exception. Thank you. Now I am encountering a new exception.

Start with empty or non-existing logs folder.

python export_db.py --profile DEMO --export-home example@foobar.com
Traceback (most recent call last):
  File "export_db.py", line 308, in <module>
    main()
  File "export_db.py", line 262, in main
    ws_c.export_user_home(username, 'user_exports')
  File "/home/ec2-user/github/databricks-migrate/dbclient/WorkspaceClient.py", line 195, in export_user_home
    num_of_nbs_dl = self.download_notebooks(ws_dir='user_artifacts/')
  File "/home/ec2-user/github/databricks-migrate/dbclient/WorkspaceClient.py", line 292, in download_notebooks
    notebook_error_logger = logging_utils.get_error_logger(
  File "/home/ec2-user/github/databricks-migrate/logging_utils.py", line 25, in get_error_logger
    error_handler = logging.FileHandler(failed_log_file, 'w+')
  File "/home/ec2-user/.pyenv/versions/3.8.7/lib/python3.8/logging/__init__.py", line 1143, in __init__
    StreamHandler.__init__(self, self._open())
  File "/home/ec2-user/.pyenv/versions/3.8.7/lib/python3.8/logging/__init__.py", line 1172, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding)
FileNotFoundError: [Errno 2] No such file or directory: '/home/ec2-user/github/databricks-migrate/logs/user_exports/example@foobar.com/app_logs/failed_export_notebooks.log'
paulg-mscience commented 2 years ago

Should I open a new issue or can you re-open this issue?

kevinkim9264 commented 2 years ago

@paulg-mscience thanks for reporting. let me look at it soon

kevinkim9264 commented 2 years ago

@paulg-mscience can you try this please https://github.com/databrickslabs/migrate/pull/123