BlobServiceClient ImportError: cannot import name 'ParamSpec' from 'typing_extensions'

[svadali2]

• Package Name: azure.storage.blob
• Package Version: 12.10.0
• Operating System: Azure Databricks Cluster, DataBricks Runtime 10.3 ML
• Python Version: 3.8

Describe the bug An ImportError from typing_extensions package shows up when I run from azure.storage.blob import BlobServiceClient. I run the commands from a Notebook inside the Azure Databricks cluster.

To Reproduce Steps to reproduce the behavior:

1. start up a VM with Python3.8 and create a new notebook.
2. create a new cell and run ! pip3 install azure-storage-blob
3. create a new cell and run from azure.storage.blob import BlobServiceClient

Expected behavior Be able to import and use the package without any import errors from typing_extensions or any other dependent package.

Screenshots Adding some screenshots of the stack trace

Additional context I fixed the problem in one notebook by doing

%rm -r /databricks/python3/lib/python3.8/site-packages/typing_extensions-4.1.1.dist-info /databricks/python3/lib/python3.8/site-packages/typing_extensions.py

This did not work on a second notebook where I tried it.

[kristapratico]

Hi @svadali2 thanks for your issue. Are you able to share which version of azure-core is installed in the notebook?

[svadali2]

Sure, here is the azure-core version:

azure-core 1.23.0

[kristapratico]

@svadali2 Just wanted to provide an update -- I'm having trouble reproducing this. I recreated the environment with the same runtime and Python version and I'm not seeing the error raised on import:

Just to outline my steps, I added the storage library under the library tab here instead of doing the pip install.

We have seen a similar issue occur with the SDK, albeit with Synapse notebooks, and the workaround was to recreate the spark pool. Meanwhile I'll reach out to the service team responsible and get back to you.

[kristapratico]

@svadali2 the Databricks service team got back to me and said it would be best to file a support ticket for this issue so they can bring in the relevant folks to help. You can do this on your databricks resource in Azure Portal:

[ghost]

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

[svadali2]

@kristapratico thanks for the response, i found a workaround in the meantime. i will go ahead and file a support issue with Databricks!

[xiaoyongzhu]

I also found this issue, and can be reporduced by using Databricks 10.2 ML runtime.

[kristapratico]

@xiaoyongzhu moving the chat from the PR to this issue:

@kristapratico seems this PR introduces some import issue in databricks ML runtime (see this issue: linkedin/feathr#154). Also seems like ParamSpec is added in 3.10 (https://docs.python.org/3/library/typing.html#typing.ParamSpec) so maybe we should also have a way to relax this limitation?

And can be repoduced by this (in databricks ML 10.2): !pip install azure-core==1.23.1 from azure.core.tracing.decorator import distributed_trace

and will throw out this error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<command-3599966980702568> in <module>
----> 1 from azure.core.tracing.decorator import distributed_trace

/databricks/python_shell/dbruntime/PythonPackageImportsInstrumentation/__init__.py in import_patch(name, globals, locals, fromlist, level)
    160             # Import the desired module. If you’re seeing this while debugging a failed import,
    161             # look at preceding stack frames for relevant error information.
--> 162             original_result = python_builtin_import(name, globals, locals, fromlist, level)
    163 
    164             is_root_import = thread_local._nest_level == 1

/databricks/python/lib/python3.8/site-packages/azure/core/tracing/decorator.py in <module>
     29 
     30 from typing import Callable, Any, TypeVar, overload
---> 31 from typing_extensions import ParamSpec
     32 from .common import change_context, get_function_and_class_name
     33 from ..settings import settings

ImportError: cannot import name 'ParamSpec' from 'typing_extensions' (/databricks/python/lib/python3.8/site-packages/typing_extensions.py)

@xiaoyongzhu thanks for bringing to my attention, as it is still an issue I'm going to re-open this. Can you provide more details on how to set up your Databricks environment? I had previously tried to repro this error in a Databricks notebook with no luck (see above). Are you running this in a notebook? Also which version of typing-extensions is getting installed?

typing.ParamSpec was added in Python 3.10, but typing-extensions should backport it to as early as 3.6 (looks like you're using 3.8). While this import works outside of Databricks, I'll need to investigate why it's a problem in the Databricks runtime and follow up with the service team.

[xiaoyongzhu]

I'll ping you in teams and give an environment so you can repro @kristapratico

[kristapratico]

Some notes:

This issue repros in DataBricks ML Runtimes, specifically. Standard runtimes do not have the issue. The current workaround is to downgrade the azure-core version to 1.22.1:

!pip install azure-core==1.22.1

I've opened a support request with the azure databricks team and will ping updates to this thread as I get them.

[kristapratico]

Update: heard back from the service team. The issue breaks down to this -- typing-extensions is already loaded into the namespace for the ML runtime. Installing azure-core requires a newer version of typing-extensions which should upgrade from 3.7.4.3 to 4.X.X. However, since the module was already loaded in the notebook, it does not get reloaded. It's possible to manually trigger this using this code snippet:

import typing_extensions
from importlib import reload
reload(typing_extensions)

Note that this is just a workaround to use the latest azure-core version. While this is not an SDK bug, I'm following up with the service team to get an idea of when this bug will be fixed in azure databricks. So just to summarize, the current workarounds:

1. Re-load the typing-extensions module by pasting this code into a notebook cell:

   import typing_extensions
   from importlib import reload
   reload(typing_extensions)

2. Use an init script to install the client library and dependencies:

#!/bin/bash
/databricks/python/bin/pip install azure-core
/databricks/python/bin/pip uninstall typing-extensions -y
/databricks/python/bin/pip install 'typing-extensions==4.0.1'

Documentation on adding an init script to your cluster can be found here: https://docs.microsoft.com/azure/databricks/clusters/init-scripts#--configure-a-cluster-scoped-init-script

3. Downgrade the azure-core version to 1.22.1 (least recommended)

!pip install azure-core==1.22.1

[solidcloudio]

I think the underlying issue here is the version of python typing_extensions library that's being used. The latest azure core sdk utilizes ParamSpec type from the typing_extensions library. the dev requirements file says you need typing_extensions>=3.7.2

https://github.com/Azure/azure-sdk-for-python/blob/c8291ac6cb0dbd865da03a88dd2bcb9279e2c4a6/sdk/core/azure-mgmt-core/dev_requirements.txt

However, the ParamSpec type wasnt introduced into the typing_extensions library until v3.10.0 https://github.com/python/typing_extensions/commit/b697a12f2793655db99dde660bb47458ad36aa55

So, if you import azure-identity into python and you have typing_extensions >= 3.7.2 to < 3.10.0 then azure-identity will fail.

[kristapratico]

@solidcloudio, can you share the output of pip freeze for your environment and help me understand where you see the failure?

azure-core installs typing-extensions==4.0.1: https://github.com/Azure/azure-sdk-for-python/blob/3482c85d8500e465f16e1b84133068a07a0a03f5/sdk/core/azure-core/setup.py#L72

The dev_requirements.txt file is used for testing purposes only and the versions specified in that file are not used when installing the package.

[solidcloudio]

Well, I'm in a "specific context" Nvidia's Omniverse code that uses an embedded version of the python and pip libraries. The typing extensions library in this context is fixed to 3.7.4.3, which doesn't contain the "ParamSpec" type, so my code that tries to use azure-identity and azure-mgmt-resources fails in this context with:

`[Error] [omni.ext.impl.custom_importer] Failed to import python module mymodule. Error: cannot import name 'ParamSpec' from 'typing_extensions' (c:\users\xxx\appdata\local\ov\pkg\code-2022.2.0\kit\extscore\omni.kit.pip_archive\pip_prebundle\typing_extensions.py).

[kristapratico]

@solidcloudio I see. Maybe I miss something, but from how you describe it this problem appears to be related to your environment. The azure packages specify the version needed for typing-extensions, but it seems your environment chooses the fixed version.

I'm not familiar with Nvidia's omniverse, but wondering if you could maybe use the omni.kit.pipapi to install the 4.0.1 version: https://docs.omniverse.nvidia.com/py/kit/docs/guide/python.html#using-other-packages-from-pip

Also, if it is getting installed, but perhaps just not re-loaded in your env, you could try manually re-loading:

import typing_extensions
from importlib import reload
reload(typing_extensions)

[solidcloudio]

That seems to have resolved it. I can't call reload(typing_extensions) or it seems to reset some things. But, it is loading the azure python and pip packages now.. awesome! Thank you!!

[kristapratico]

Closing this issue since the problem is not in the SDK.