"No module named 'unicodedata'" error on installing rdbms-connect extension

[mustafacagataytulun]

I get a "No module named 'unicodedata'" error while trying to install rdbms-connect extension on Azure CLI 2.43.0 with Windows operating system. I will add full stack trace at the end.

The command I use is this:

az extension add --name rdbms-connect --debug

To Reproduce

1. Have a Windows PC.
2. Install latest Azure CLI. (2.43.0 as I open this issue.)
3. Open a command prompt or PowerShell Core
4. Try to install rdbms-connect extension. The installation will fail.

Expected behavior The installation should finish successfully without any error.

Environment summary AZ CLI 2.43.0 (installed via MSI package) Windows 10/Windows Server 2022 (fails on both) Cmd.exe/PowerShell Core (fails on both)

Additional context I can confirm that this issue does not occur when using an Ubuntu 22.04 or Ubuntu 22.04 on WSL (I mean, via Bash terminal).

Another issue #24622 was opened for the same error and closed afterwards. It seems that there was no problem with 2.37.0 version.

Full Stack Trace

ERROR: Exception:
Traceback (most recent call last):
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\cli\base_command.py", line 160, in exc_logging_wrapper
    status = run_func(*args)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\cli\req_command.py", line 247, in wrapper
    return func(self, options, args)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\commands\install.py", line 400, in run
    requirement_set = resolver.resolve(
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 373, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 213, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 204, in _get_updated_criteria
    self._add_to_criteria(criteria, requirement, parent=candidate)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 172, in _add_to_criteria
    if not criterion.candidates:
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\resolvelib\structs.py", line 151, in __bool__
    return bool(self._sequence)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 155, in __bool__
    return any(self)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 47, in _iter_built
    candidate = func()
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 206, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 297, in __init__
    super().__init__(
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 162, in __init__
    self.dist = self._prepare()
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 231, in _prepare
    dist = self._prepare_distribution()
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 308, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\operations\prepare.py", line 491, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\operations\prepare.py", line 577, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\operations\prepare.py", line 69, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\distributions\sdist.py", line 48, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\distributions\sdist.py", line 118, in _install_build_reqs
    build_reqs = self._get_build_requires_wheel()
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\distributions\sdist.py", line 95, in _get_build_requires_wheel
    return backend.get_requires_for_build_wheel()
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\utils\misc.py", line 685, in get_requires_for_build_wheel
    return super().get_requires_for_build_wheel(config_settings=cs)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\pep517\wrappers.py", line 173, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\pep517\wrappers.py", line 319, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 77, in _build_backend
    obj = import_module(mod_path)
  File "importlib\__init__.py", line 126, in import_module
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\Users\beawolf\AppData\Local\Temp\pip-build-env-q8wqmk8r\overlay\Lib\site-packages\poetry\core\masonry\api.py", line 11, in <module>
    from poetry.core.factory import Factory
  File "C:\Users\beawolf\AppData\Local\Temp\pip-build-env-q8wqmk8r\overlay\Lib\site-packages\poetry\core\factory.py", line 16, in <module>
    from poetry.core.utils.helpers import combine_unicode
  File "C:\Users\beawolf\AppData\Local\Temp\pip-build-env-q8wqmk8r\overlay\Lib\site-packages\poetry\core\utils\helpers.py", line 7, in <module>
    import unicodedata
ModuleNotFoundError: No module named 'unicodedata'

[ghost]

Thank you for your feedback. This has been routed to the support team for assistance.

[yonzhan]

route to CXP team

[clementchane]

Hello, Same error here. Trying to run : az postgres flexible-server execute but cannot install dependency on rdbms-connect. I can only install module unicodedata2 with pip install.

pip show unicodedata
WARNING: Package(s) not found: unicodedata
pip install unicodedata
ERROR: Could not find a version that satisfies the requirement unicodedata (from versions: none)
ERROR: No matching distribution found for unicodedata

PS C:\WINDOWS\system32> pip show unicodedata2
Name: unicodedata2
Version: 15.0.0
....

[SaurabhSharma-MSFT]

@mustafacagataytulun I am able to reproduce this issue and redirecting this to respective team for the fix. Thanks.

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureSQLGitHub.

Issue Details

---

I get a "No module named 'unicodedata'" error while trying to install _rdbms-connect_ extension on Azure CLI 2.43.0 with Windows operating system. I will add full stack trace at the end. The command I use is this: ```ps az extension add --name rdbms-connect --debug ``` **To Reproduce** 1. Have a Windows PC. 2. Install latest Azure CLI. (2.43.0 as I open this issue.) 3. Open a command prompt or PowerShell Core 4. Try to install _rdbms-connect_ extension. The installation will fail. **Expected behavior** The installation should finish successfully without any error. **Environment summary** AZ CLI 2.43.0 (installed via MSI package) Windows 10/Windows Server 2022 (fails on both) Cmd.exe/PowerShell Core (fails on both) **Additional context** I can confirm that this issue does not occur when using an Ubuntu 22.04 or Ubuntu 22.04 on WSL (I mean, via Bash terminal). Another issue #24622 was opened for the same error and closed afterwards. It seems that there was no problem with 2.37.0 version. **Full Stack Trace** ```python ERROR: Exception: Traceback (most recent call last): File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\cli\base_command.py", line 160, in exc_logging_wrapper status = run_func(*args) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\cli\req_command.py", line 247, in wrapper return func(self, options, args) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\commands\install.py", line 400, in run requirement_set = resolver.resolve( File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 92, in resolve result = self._result = resolver.resolve( File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 481, in resolve state = resolution.resolve(requirements, max_rounds=max_rounds) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 373, in resolve failure_causes = self._attempt_to_pin_criterion(name) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 213, in _attempt_to_pin_criterion criteria = self._get_updated_criteria(candidate) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 204, in _get_updated_criteria self._add_to_criteria(criteria, requirement, parent=candidate) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 172, in _add_to_criteria if not criterion.candidates: File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\resolvelib\structs.py", line 151, in __bool__ return bool(self._sequence) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 155, in __bool__ return any(self) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 143, in return (c for c in iterator if id(c) not in self._incompatible_ids) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 47, in _iter_built candidate = func() File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 206, in _make_candidate_from_link self._link_candidate_cache[link] = LinkCandidate( File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 297, in __init__ super().__init__( File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 162, in __init__ self.dist = self._prepare() File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 231, in _prepare dist = self._prepare_distribution() File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 308, in _prepare_distribution return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\operations\prepare.py", line 491, in prepare_linked_requirement return self._prepare_linked_requirement(req, parallel_builds) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\operations\prepare.py", line 577, in _prepare_linked_requirement dist = _get_prepared_distribution( File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\operations\prepare.py", line 69, in _get_prepared_distribution abstract_dist.prepare_distribution_metadata( File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\distributions\sdist.py", line 48, in prepare_distribution_metadata self._install_build_reqs(finder) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\distributions\sdist.py", line 118, in _install_build_reqs build_reqs = self._get_build_requires_wheel() File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\distributions\sdist.py", line 95, in _get_build_requires_wheel return backend.get_requires_for_build_wheel() File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_internal\utils\misc.py", line 685, in get_requires_for_build_wheel return super().get_requires_for_build_wheel(config_settings=cs) File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\pep517\wrappers.py", line 173, in get_requires_for_build_wheel return self._call_hook('get_requires_for_build_wheel', { File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\pep517\wrappers.py", line 319, in _call_hook raise BackendUnavailable(data.get('traceback', '')) pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last): File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 77, in _build_backend obj = import_module(mod_path) File "importlib\__init__.py", line 126, in import_module File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "C:\Users\beawolf\AppData\Local\Temp\pip-build-env-q8wqmk8r\overlay\Lib\site-packages\poetry\core\masonry\api.py", line 11, in from poetry.core.factory import Factory File "C:\Users\beawolf\AppData\Local\Temp\pip-build-env-q8wqmk8r\overlay\Lib\site-packages\poetry\core\factory.py", line 16, in from poetry.core.utils.helpers import combine_unicode File "C:\Users\beawolf\AppData\Local\Temp\pip-build-env-q8wqmk8r\overlay\Lib\site-packages\poetry\core\utils\helpers.py", line 7, in import unicodedata ModuleNotFoundError: No module named 'unicodedata' ```

Author:	mustafacagataytulun
Assignees:	SaurabhSharma-MSFT
Labels:	`bug`, `SQL`, `Service Attention`, `customer-reported`, `Installation`, `Auto-Assign`
Milestone:	Backlog

[bebound]

This issue similar to https://github.com/Azure/azure-cli/issues/24781, it's embedded python or pip problem. pip fails to install rdbms-connect's dependency pendulum. I've open https://github.com/python/cpython/issues/100399, but haven't found any solution for this.

---

The cause is that DLL and pyd files are put into root folder in embedded python, but they are put into DLLs in standard Python. Root folder is excluded from sys.path in pip build env, thus packages like pyexpat and unicodedata fails to import.

I've created a PR to fix this for pip https://github.com/pypa/pip/pull/11724, but pip team thinks it's embedded Python bug and don't want to special case it.

[jiasli]

I am also able to repro.

When using the embeded Python, unicodedata and xml.parsers.expat libraries can be imported directly, but not during pip install:

> & "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe" -c "import unicodedata; print(unicodedata.__file__)"
C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\unicodedata.pyd

> & "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe" -c "import xml.parsers.expat; print(xml.parsers.expat.__file__)"
C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python310.zip\xml\parsers\expat.pyc

But when I use the system Python, I get:

> & "C:\Users\myname\AppData\Local\Programs\Python\Python310\python.exe" -c "import unicodedata; print(unicodedata.__file__)"
C:\Users\myname\AppData\Local\Programs\Python\Python310\DLLs\unicodedata.pyd

> & "C:\Users\myname\AppData\Local\Programs\Python\Python310\python.exe" -c "import xml.parsers.expat; print(xml.parsers.expat.__file__)"
C:\Users\myname\AppData\Local\Programs\Python\Python310\lib\xml\parsers\expat.py

[rlaveycal]

I also have this issue. I was able to install version 1.0.3 on Windows

{
  "azure-cli": "2.46.0",
  "azure-cli-core": "2.46.0",
  "azure-cli-telemetry": "1.0.8",
  "extensions": {
    "rdbms-connect": "1.0.3"
  }
}

[jerzy-wachowiak]

Same behavior as described above on Win10, 22H2 Python 3.11.3 and

{
  "azure-cli": "2.49.0",
  "azure-cli-core": "2.49.0",
  "azure-cli-telemetry": "1.0.8",
  "extensions": {
    "account": "0.2.2",
    "aks-preview": "0.5.140",
    "azure-devops": "0.23.0",
    "k8s-extension": "1.4.1",
    "rdbms-connect": "1.0.3"
  }
}

The workaround is

az extension add --name rdbms-connect --version 1.0.3