[BUG]: ModuleNotFoundError: No module named 'astroid'

arqgroup-charunathan commented 3 weeks ago

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior

Hi, While trying to run the table migration command I'm facing this issue. Also tried to upgrade the ucx and got the same issue databricks labs ucx create-table-mapping

Expected Behavior

No response

Steps To Reproduce

No response

Cloud

AWS

Operating System

Windows

Version

latest via Databricks CLI

Relevant log output

Trying to execute the command for table migration:
PS C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main> databricks labs ucx create-table-mapping
Traceback (most recent call last):
  File "C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main\src\databricks\labs\ucx\cli.py", line 13, in <module>
    from databricks.labs.ucx.contexts.account_cli import AccountContext
  File "C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main\src\databricks\labs\ucx\contexts\account_cli.py", line 7, in <module>
    from databricks.labs.ucx.account.aggregate import AccountAggregate
  File "C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main\src\databricks\labs\ucx\account\aggregate.py", line 14, in <module>
    from databricks.labs.ucx.contexts.workspace_cli import WorkspaceContext
  File "C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main\src\databricks\labs\ucx\contexts\workspace_cli.py", line 17, in <module>
    from databricks.labs.ucx.contexts.application import CliContext
  File "C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main\src\databricks\labs\ucx\contexts\application.py", line 18, in <module>
    from databricks.labs.ucx.source_code.python_libraries import PythonLibraryResolver
  File "C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main\src\databricks\labs\ucx\source_code\python_libraries.py", line 14, in <module>
    from databricks.labs.ucx.source_code.graph import (
  File "C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main\src\databricks\labs\ucx\source_code\graph.py", line 8, in <module>
    from astroid import ImportFrom  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'astroid'
Error: exit status 1
----------------------------------
Trying to upgrade to new ucx version: 
PS C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main> databricks labs upgrade ucx
Traceback (most recent call last):
  File "C:\Users\charunathan.sivanand\.databricks\labs\ucx\lib\src\databricks\labs\ucx\install.py", line 50, in <module>
    from databricks.labs.ucx.contexts.account_cli import AccountContext
  File "C:\Users\charunathan.sivanand\.databricks\labs\ucx\lib\src\databricks\labs\ucx\contexts\account_cli.py", line 7, in <module>
    from databricks.labs.ucx.account.aggregate import AccountAggregate
  File "C:\Users\charunathan.sivanand\.databricks\labs\ucx\lib\src\databricks\labs\ucx\account\aggregate.py", line 14, in <module>
    from databricks.labs.ucx.contexts.workspace_cli import WorkspaceContext
  File "C:\Users\charunathan.sivanand\.databricks\labs\ucx\lib\src\databricks\labs\ucx\contexts\workspace_cli.py", line 17, in <module>
    from databricks.labs.ucx.contexts.application import CliContext
  File "C:\Users\charunathan.sivanand\.databricks\labs\ucx\lib\src\databricks\labs\ucx\contexts\application.py", line 18, in <module>
    from databricks.labs.ucx.source_code.python_libraries import PythonLibraryResolver
  File "C:\Users\charunathan.sivanand\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\python_libraries.py", line 14, in <module>
    from databricks.labs.ucx.source_code.graph import (
  File "C:\Users\charunathan.sivanand\.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 8, in <module>
    from astroid import ImportFrom, NodeNG  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'astroid'
Error: installer: exit status 1

nfx commented 3 weeks ago

please update databricks cli to the latest version and reopen this issue if it still happens.

arqgroup-charunathan commented 3 weeks ago

I have updated the databricks cli to the latest version and still facing the same error.

PS C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main> python --version Python 3.12.4 PS C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main> winget upgrade Databricks.DatabricksCLI Found DatabricksCLI [Databricks.DatabricksCLI] Version 0.221.1 This application is licensed to you by its owner. Microsoft is not responsible for, nor does it grant any licenses to, third-party packages. Downloading https://github.com/databricks/cli/releases/download/v0.221.1/databricks_cli_0.221.1_windows_amd64-signed.zip ██████████████████████████████ 9.51 MB / 9.51 MB Successfully verified installer hash Extracting archive... Successfully extracted archive Starting package uninstall... Successfully uninstalled Starting package install... Command line alias added: "databricks" Successfully installed PS C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main> databricks version Databricks CLI v0.221.1 PS C:\Users\charunathan.sivanand\Downloads\ucx-main\ucx-main> databricks labs upgrade ucx Traceback (most recent call last): File "C:\Users\charunathan.sivanand.databricks\labs\ucx\lib\src\databricks\labs\ucx\install.py", line 50, in from databricks.labs.ucx.contexts.account_cli import AccountContext File "C:\Users\charunathan.sivanand.databricks\labs\ucx\lib\src\databricks\labs\ucx\contexts\account_cli.py", line 7, in from databricks.labs.ucx.account.aggregate import AccountAggregate File "C:\Users\charunathan.sivanand.databricks\labs\ucx\lib\src\databricks\labs\ucx\account\aggregate.py", line 14, in from databricks.labs.ucx.contexts.workspace_cli import WorkspaceContext File "C:\Users\charunathan.sivanand.databricks\labs\ucx\lib\src\databricks\labs\ucx\contexts\workspace_cli.py", line 17, in from databricks.labs.ucx.contexts.application import CliContext File "C:\Users\charunathan.sivanand.databricks\labs\ucx\lib\src\databricks\labs\ucx\contexts\application.py", line 18, in from databricks.labs.ucx.source_code.python_libraries import PythonLibraryResolver File "C:\Users\charunathan.sivanand.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\python_libraries.py", line 14, in from databricks.labs.ucx.source_code.graph import ( File "C:\Users\charunathan.sivanand.databricks\labs\ucx\lib\src\databricks\labs\ucx\source_code\graph.py", line 8, in from astroid import ImportFrom, NodeNG # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named 'astroid' Error: installer: exit status 1

arqgroup-charunathan commented 3 weeks ago

Hi @nfx, Could you please reopen this issue as I'm unable to reopen it. Thanks

arqgroup-charunathan commented 3 weeks ago

Hi @nfx while upgrade it is using the latest release version https://github.com/databrickslabs/ucx/releases/tag/v0.26.0 Python libraries was not updated as per the current git code in the main branch, and it is causing the error. Thanks

ralphkootker commented 3 weeks ago

I'm running into the same error what has been the solution to this?

pdbarrington commented 2 weeks ago

same.

JCZuurmond commented 1 week ago

Python libraries was not updated as per the current git code in the main branch, and it is causing the error.

When downloading the release, astroid is mentioned in the dependencies.

Could y'all try an uninstall and install? databricks labs uninstall ucx and databricks labs install ucx

arqgroup-charunathan commented 1 week ago

I have uninstalled both the ucx tool and Python latest version and then I have reinstalled the python 3.10 version and ucx tool again. Now it's working fine for me. When I tried with my latest python version, it didn't work for me. Thanks

JCZuurmond commented 1 week ago

@arqgroup-charunathan : Thanks for letting us now. What was your "latest python version" for which it did not work?

arqgroup-charunathan commented 1 week ago

@JCZuurmond: I used python-3.12.4 version which didn't work for me.

JCZuurmond commented 1 week ago

Logging here that someone ran into this issue with python 3.10.4 and was not able to uninstall

JCZuurmond commented 1 week ago

Hi everyone, in case your experiencing this issue, the recommend approach is to reinstall ucx completely:

First uninstall: databricks labs uninstall ucx, without deleting ucx inventory database (prompt during uninstall)
Then install: databricks labs install ucx

Now, some of you experience a lock in because astroid is not available during uninstall. In that case, manually install astroid by:

Locating the python executable that Databricks labs ucx uses: ~/.databricks/labs/ucx/state/venv/bin/python3.
Confriming that astroid is missing: ~/.databricks/labs/ucx/state/venv/bin/python3 -m astroid (raises ModuleNotFound)
Installing astroid using pip: ~/.databricks/labs/ucx/state/venv/bin/python3 -m pip install astroid
Confirming astroid is installed: ~/.databricks/labs/ucx/state/venv/bin/python3 -m astroid (should not raise ModuleNotFound)
Reinstall ucx as mentioned above: databricks labs uninstall ucx and databricks labs install ucx

I hope this helps! Please report below if this resolves your issue or if it does not resolve your issue

nfx commented 4 days ago

@arqgroup-charunathan @pdbarrington @ralphkootker please run databricks labs upgrade ucx --debug - we need to see if certain pip commands were executed under the hood

databrickslabs / ucx