search

databrickslabs / ucx

Your best companion for upgrading to Unity Catalog. UCX will guide you, the Databricks customer, through the process of upgrading your account, groups, workspaces, jobs etc. to Unity Catalog.
Other
193 stars 69 forks source link

[BUG]: Permission denied when running databricks labs ucx lint-local-code #1942

Closed jgarciaf106 closed 1 day ago

jgarciaf106 commented 6 days ago

Is there an existing issue for this?

Current Behavior

A permission denied error occurs when running the databricks labs ucx lint-local-code on a Windows PC. It doesn't matter which directory is run from, or when passing the directory as an option, it still throws an error.

Expected Behavior

It is expected that when running from the default directory or passing a specific directory or file to have the code lint using databricks labs ucx lint-local-code

Steps To Reproduce

  1. cd to the target directory where the databricks labs ucx lint-local-code is going scan the code.
  2. run databricks labs ucx lint-local-code
  3. prompt for to pass file or directory shows up
  4. press enter shows the error.

Cloud

Azure

Operating System

Windows

Version

latest via Databricks CLI

Relevant log output

C:\Users\dnissim\OneDrive - BDO Canada LLP\Desktop\Databricks-Test>databricks labs ucx lint-local-code --debug --profile adb-3386741836704280
10:29:00  INFO start pid=3432 version=0.221.1 args="databricks, labs, ucx, lint-local-code, --debug, --profile, adb-3386741836704280"
10:29:00 DEBUG Fetching latest releases for databrickslabs/ucx from GitHub API pid=3432
10:29:01 DEBUG Loading installed version info from: C:\Users\dnissim\.databricks\labs\ucx\state\version.json pid=3432
10:29:01  INFO Overriding login profile: adb-3386741836704280 pid=3432
10:29:01 DEBUG Loading adb-3386741836704280 profile from C:\Users\dnissim\.databrickscfg pid=3432 sdk=true
10:29:01 DEBUG Resolved login: Config: host=https://adb-3386741836704280.0.azuredatabricks.net/, profile=adb-3386741836704280, config_file=C:\Users\dnissim\.databrickscfg pid=3432 sdk=true
10:29:01 DEBUG Passing down environment variables: DATABRICKS_AUTH_TYPE, DATABRICKS_HOST pid=3432
10:29:01 DEBUG Forwarding subprocess: C:\Users\dnissim\.databricks\labs\ucx\state\venv\Scripts\python.exe C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\cli.py {"command":"lint-local-code","flags":{"log_level":"debug","path":""},"output_type":""} pid=3432
10:29:01 DEBUG starting: C:\Users\dnissim\.databricks\labs\ucx\state\venv\Scripts\python.exe C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\cli.py {"command":"lint-local-code","flags":{"log_level":"debug","path":""},"output_type":""} pid=3432
10:29:03 DEBUG [databricks.sdk] Loaded from environment
10:29:03 DEBUG [databricks.sdk] Ignoring pat auth, because databricks-cli is preferred
10:29:03 DEBUG [databricks.sdk] Ignoring basic auth, because databricks-cli is preferred
10:29:03 DEBUG [databricks.sdk] Ignoring metadata-service auth, because databricks-cli is preferred
10:29:03 DEBUG [databricks.sdk] Ignoring oauth-m2m auth, because databricks-cli is preferred
10:29:03 DEBUG [databricks.sdk] Ignoring azure-client-secret auth, because databricks-cli is preferred
10:29:03 DEBUG [databricks.sdk] Ignoring github-oidc-azure auth, because databricks-cli is preferred
10:29:03 DEBUG [databricks.sdk] Ignoring azure-cli auth, because databricks-cli is preferred
10:29:03 DEBUG [databricks.sdk] Ignoring external-browser auth, because databricks-cli is preferred
10:29:03 DEBUG [databricks.sdk] Attempting to configure auth: databricks-cli
10:29:03  INFO [databricks.sdk] Using Databricks CLI authentication
Which file or directory do you want to lint? (default: C:/Users/dnissim/OneDrive - BDO Canada LLP/Desktop/Databricks-Test):
10:29:07 DEBUG [d.l.u.s.linters.files] Resolving unknown import: azure.storage.blob
10:29:07 DEBUG [d.l.u.s.linters.files] Resolving unknown import: azure.storage.blob
10:29:13 DEBUG [d.l.u.s.linters.imports] Can't compute UninferableBase
10:29:13 DEBUG [d.l.u.s.linters.imports] Can't compute UninferableBase
10:29:13 ERROR [src/databricks/labs/ucx.lint-local-code] Failed to call lint-local-code: Traceback (most recent call last):
  File "C:\Users\dnissim\.databricks\labs\ucx\state\venv\Lib\site-packages\databricks\labs\blueprint\cli.py", line 109, in _route
    cmd.fn(**kwargs)
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\cli.py", line 489, in lint_local_code    
    ctx.local_code_linter.lint(prompts, None if path is None else Path(path))
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 110, in lint
    located_advices = list(self.lint_path(path))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 122, in lint_path
    problems = container.build_dependency_graph(graph)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 72, in build_dependency_graph
    return list(self._build_dependency_graph(parent))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 78, in _build_dependency_graph
    yield from parent.register_dependency(dependency).problems
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 88, in register_dependency
    problems = container.build_dependency_graph(child_graph)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 72, in build_dependency_graph
    return list(self._build_dependency_graph(parent))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 78, in _build_dependency_graph
    yield from parent.register_dependency(dependency).problems
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 88, in register_dependency
    problems = container.build_dependency_graph(child_graph)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 72, in build_dependency_graph
    return list(self._build_dependency_graph(parent))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 78, in _build_dependency_graph
    yield from parent.register_dependency(dependency).problems
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 88, in register_dependency
    problems = container.build_dependency_graph(child_graph)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 72, in build_dependency_graph
    return list(self._build_dependency_graph(parent))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 78, in _build_dependency_graph
    yield from parent.register_dependency(dependency).problems
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 88, in register_dependency
    problems = container.build_dependency_graph(child_graph)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 72, in build_dependency_graph
    return list(self._build_dependency_graph(parent))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 78, in _build_dependency_graph
    yield from parent.register_dependency(dependency).problems
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 88, in register_dependency
    problems = container.build_dependency_graph(child_graph)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 72, in build_dependency_graph
    return list(self._build_dependency_graph(parent))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 78, in _build_dependency_graph
    yield from parent.register_dependency(dependency).problems
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 88, in register_dependency
    problems = container.build_dependency_graph(child_graph)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\linters\files.py", line 43, in build_dependency_graph
    return parent.build_graph_from_python_source(self._original_code)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 189, in build_graph_from_python_source
    for problem in self._process_node(base_node):
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 204, in _process_node
    yield from self._register_notebook(base_node)
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 225, in _register_notebook
    yield from self.register_notebook(Path(path))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 59, in register_notebook
    maybe_graph = self.register_dependency(maybe.dependency)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 88, in register_dependency
    problems = container.build_dependency_graph(child_graph)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\notebooks\sources.py", line 72, in build_dependency_graph
    cell_problems = cell.build_dependency_graph(parent)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\notebooks\cells.py", line 167, in build_dependency_graph
    problems = parent.register_notebook(notebook_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 59, in register_notebook
    maybe_graph = self.register_dependency(maybe.dependency)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 83, in register_dependency
    container = dependency.load(self.path_lookup)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 263, in load 
    return self._loader.load_dependency(path_lookup, self)
  File "C:\Users\dnissim\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\notebooks\loaders.py", line 54, in load_dependency
    content = absolute_path.read_text("utf-8")
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\AppData\Local\Programs\Python\Python312-32\Lib\pathlib.py", line 1027, in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dnissim\AppData\Local\Programs\Python\Python312-32\Lib\pathlib.py", line 1013, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\dnissim\\.databricks\\labs\\ucx\\config'
10:29:13  INFO completed execution pid=3432 exit_code=0
JCZuurmond commented 5 days ago

@jgarciaf106 : Thanks for reporting this with the full stack trace! See linked PR