search

Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.63k stars 2.83k forks source link

Unable to authenticate to Azure ML Workspace using Service Principal #13871

Closed gison93 closed 3 years ago

gison93 commented 4 years ago

Describe the bug Unable to authenticate to Azure ML Workspace using Service Principal. I get the following error: AttributeError: 'AdalAuthentication' object has no attribute 'get_token'.

To Reproduce Steps to reproduce the behavior:

  1. pip install azureml-core==1.13.0
  2. 
from azureml.core import Workspace
from azureml.core.authentication import ServicePrincipalAuthentication

tenant = "tenant" client = "client" key = "key"

credentials = ServicePrincipalAuthentication( tenant_id=tenant, service_principal_id=client, service_principal_password=key)

workspace = Workspace.from_config("config.json", auth=credentials)


with config.json file containing information about the Workspace.

**Expected behavior**
To be able of attaching to the Workspace using Service Principal.
magrathj commented 4 years ago

Having a similar issue connection to Azureml using service principle

hari-buddhavarapu commented 4 years ago

Same issue I have been facing as well. It worked fine until yesterday. Unable to connect to Azureml using service principal

sajalda23409 commented 4 years ago

Same here. It was working fine till yesterday. From today, I am facing the same issue.

magrathj commented 4 years ago

Was working fine for me until around 6.30am GMT+1 this morning

jarandaf commented 4 years ago

Same here, started to fail today.

avtokit2700 commented 4 years ago

Hello guys. I'm facing the same problem. Everything was fine a couple of hours ago.

gison93 commented 4 years ago

Workaround/Possible Solution In azure\core\pipeline\policies_authentication.py in class BearerTokenCredentialPolicy instead of calling get_token use _token_retriever() and take the element with index one.

 def on_request(self, request):
        # type: (PipelineRequest) -> None
        """Adds a bearer token Authorization header to request and sends request to next policy.

        :param request: The pipeline request object
        :type request: ~azure.core.pipeline.PipelineRequest
        """
        self._enforce_https(request)
        if self._token is None or self._need_new_token:
            scheme, self._token, _ = self._credential._token_retriever()
        self._update_headers(request.http_request.headers, self._token)
hari-buddhavarapu commented 4 years ago

Hi, I have been using Azure Devops Pipelines with Microsoft Hosted Agents to connect to Azure ML. I am seeing this error "AttributeError: 'AdalAuthentication' object has no attribute 'get_token'"

jarandaf commented 4 years ago

Can confirm @gison93 proposal seems to solve this issue. I don't fully understand what may have changed if there has been no version update, though.

gison93 commented 4 years ago

Can confirm @gison93 proposal seems to solve this issue. I don't fully understand what may have changed if there has been no version update, though.

I suspect is due to the recent update of azure-mgmt-keyvault to version 7.0.0 some hours ago

hari-buddhavarapu commented 4 years ago

@gison93 can we use your proposal on microsoft hosted agents?

chengyu-liu-cs commented 4 years ago

Same here. It was working fine till yesterday. From today, I am facing the same issue. The codes work perfectly from my local computer but did not work from Azure hosted agents.

avtokit2700 commented 4 years ago

On local, workaround from @gison93 work's perfectly, but I'm using Azure DataBricks as a compute target and cannot change the source code of libraries. I find a solution with downgrade lib azure-mgmt-resource from 15.0.0 to 8.0.1.

sajalda23409 commented 4 years ago

Yes, I agree with @chengyu-liu-cs

chengyu-liu-cs commented 4 years ago

@gison93 can we use your proposal on microsoft hosted agents?

I think it is easier to roll back to the previous stable version of azure-mgmt-keyvault, i.e. force to use azure-mgmt-keyvault==2.2.0 in the meantime. This seems to solve the issue on my end.

This solution did not work for me.

jarandaf commented 4 years ago

@gison93 can we use your proposal on microsoft hosted agents?

I think it is easier to roll back to the previous stable version of azure-mgmt-keyvault, i.e. force to use azure-mgmt-keyvault==2.2.0 in the meantime. This seems to solve the issue on my end.

This solution did not work for me.

They are probably changing more stuff at the moment. This did work but it is not working anymore. We will have to wait for a proper fix I guess.

chengyu-liu-cs commented 4 years ago

I did further investigation comparing package differences. Actually, both local and hosted agents have azureml-core==1.12.0.post1. So My guess azureml-core version might not be the direct reason.

There were many packages that have different versions from my local versions. But I rolled back a couple of most possible ones and solved the issue (at least for now) azure-mgmt-keyvault==2.2.0 azure-mgmt-resource==10.2.0 azure-identity==1.2.0 azure-core==1.8.0 azure-mgmt-storage==11.1.0

sajalda23409 commented 4 years ago

A new version of Azure Mgmt Resource client library released 5 hours ago. Is that caused the issue?

https://pypi.org/project/azure-mgmt-resource/

abij commented 4 years ago

It's not because of AzureML. (downgrading to 1.12.0 causes the same issue.) As @sajalda23409 mentioned it's related to https://pypi.org/project/azure-mgmt-resource/15.0.0/

Fixed with pinning the previous version: pip install azure-mgmt-resource==10.2.0

Or if you are using Databricks:

if dbutils.library.installPyPI('azure-mgmt-resource', version="10.2.0"):
  dbutils.library.restartPython()
nickzhums commented 4 years ago

We are taking a look at this issue, and will provide update later For the time being, a workaround is using azure-mgmt-resources 10.2.0 cc @yonzhan @changlong-liu @00Kai0 @jsntcy

yonzhan commented 4 years ago

You can temporarily fixed with pinning the previous version: pip install azure-mgmt-resource==10.2.0

azeltov commented 4 years ago

it fixes the issue with the service principle by pinning mgmt-resource (azure-mgmt-resource==10.2.0) , but its breaking automl now: 957 except ValidationException as ex:

/databricks/python/lib/python3.7/site-packages/azureml/automl/core/package_utilities.py in _get_package_incompatibilities(packages, ignored_dependencies) 420 target=','.join(incompatible_packages.keys()), 421 missing_packages_message=';'.join(messages), --> 422 reference_code=ReferenceCodes._UNVERIFIED_PACKAGES 423 )) 424

ValidationException: ValidationException: Message: Please install specific versions of packages: We validated this package with azure-core<=1.8.0 but your environment has azure-core 1.8.1.;We validated this package with azure-mgmt-keyvault<=2.2.0 but your environment has azure-mgmt-keyvault 7.0.0.;We validated this package with azure-mgmt-storage<=11.2.0 but your environment has azure-mgmt-storage 16.0.0.;We validated this package with dotnetcore2<=2.1.14 but your environment has dotnetcore2 2.1.15.;We validated this package with interpret-community<=0.14.3 but your environment has interpret-community 0.14.4.;We validated this package with liac-arff<=2.4.0 but your environment has liac-arff 2.5.0.;We validated this package with msal<=1.4.3 but your environment has msal 1.5.0.;We validated this package with msrest<=0.6.18 but your environment has msrest 0.6.19.;We validated this package with ruamel.yaml.clib<=0.2.0 but your environment has ruamel.yaml.clib 0.2.2.;We validated this package with ruamel.yaml<=0.16.10 but your environment has ruamel.yaml 0.16.12.;We validated this package with tqdm<=4.48.2 but your environment has tqdm 4.49.0. InnerException: None ErrorResponse { "error": { "code": "UserError", "message": "Please install specific versions of packages: We validated this package with azure-core<=1.8.0 but your environment has azure-core 1.8.1.;We validated this package with azure-mgmt-keyvault<=2.2.0 but your environment has azure-mgmt-keyvault 7.0.0.;We validated this package with azure-mgmt-storage<=11.2.0 but your environment has azure-mgmt-storage 16.0.0.;We validated this package with dotnetcore2<=2.1.14 but your environment has dotnetcore2 2.1.15.;We validated this package with interpret-community<=0.14.3 but your environment has interpret-community 0.14.4.;We validated this package with liac-arff<=2.4.0 but your environment has liac-arff 2.5.0.;We validated this package with msal<=1.4.3 but your environment has msal 1.5.0.;We validated this package with msrest<=0.6.18 but your environment has msrest 0.6.19.;We validated this package with ruamel.yaml.clib<=0.2.0 but your environment has ruamel.yaml.clib 0.2.2.;We validated this package with ruamel.yaml<=0.16.10 but your environment has ruamel.yaml 0.16.12.;We validated this package with tqdm<=4.48.2 but your environment has tqdm 4.49.0.", "inner_error": { "code": "NotSupported", "inner_error": { "code": "IncompatibleOrMissingDependency" } }, "reference_code": "65e7ad52-ad1f-4915-85ad-440d0a5e221f" } }

sriramg96 commented 4 years ago

+1. There were also issues with creating virtual networks with the azure-mgmt-network package, and this was solved with reverting to the previous version: pip install azure-mgmt-network==10.2.0

cyly commented 4 years ago

Please uninstall the azure-mgmt-resource==15.0.0 and azureml-core==1.13.0.

Then install azureml-core==1.13.0.post1. It should fix the issue.

nickzhums commented 4 years ago

Hi guys, I wanted to provide an update here. The real cause of this behavior is the version upgrade for several Azure service SDKs (such as azure-mgmt-resources and azure-mgmt-network)

Basically, we did a major upgrade for Python SDK management libraries for some services. In the new version of SDK, the authentication mechanism has been changed, we have mentioned those changes in the release notes, please refer to this changelog: changelog

The new version of SDK offers a number of important features but also introduces some breaking changes.

You can either choose to upgrade to the new version or stay on the old version.

To upgrade:

There is migration guide here for updating to the new SDK. This describes the new authentication flow.

Detailed info of benefits of the new SDK as well as documentation/code samples can be found here

To stay on the old version

Please keep using 10.2.0

gison93 commented 4 years ago

Thank you @nickzhums! I will take a look at the new version

gison93 commented 4 years ago

Hi @nickzhums, I tried to use ClientSecretCredential instead of ServicePrincipalCredential with no luck.

from azureml.core import Workspace
from azure.identity import ClientSecretCredential

tenant = "tenant"
client = "client"
key = "key"

credentials = ClientSecretCredential(
            tenant_id=tenant,
            service_principal_id=client,
            service_principal_password=key)

workspace = Workspace.from_config("config.json", auth=credentials)

I get the following error: AttributeError: 'ClientSecretCredential' object has no attribute '_get_workspace'

Should I open a new issue?

nickzhums commented 4 years ago

It seems like you are directly passing credentials to Workspace.from_config method, which I'm not sure if it will support. The standard practice is to use a service client.

e.g. network_client = azure.mgmt.network.NetworkManagementClient(credential=credential, subscription_id=subscription_id)

Yes, please open an issue and we can track it from there

bendavis78 commented 3 years ago

@nickzhums The link to the migration guide is broken. Is there another link?

Also, does the updated authentication method still support get_client_from_auth_file? I'm running into an issue with that as well.

nickzhums commented 3 years ago

@bendavis78 please check this one : https://docs.microsoft.com/en-us/azure/developer/python/azure-sdk-overview#migrating-from-older-management-libraries I apologize for the broken link - we've been doing some cleanup

cc @00Kai0 and @jsntcy to help with get_client_from_auth_file question

jamescross91 commented 3 years ago

Same issue here. We've upgraded to azure-mgmt-resource==15.0.0 because we need some of the new features - now all of our Azure ML code is broken. We've tried both CLI auth (but doesnt work due to https://github.com/Azure/azure-cli/issues/15496), and service principle - where we hit this issue.

nickzhums commented 3 years ago

@jamescross91 were you able to resolve this issue using the instructions above?

jamescross91 commented 3 years ago

@nickzhums it's unclear to me how the migration instructions relate to the Azure ML SDK

nickzhums commented 3 years ago

Like mentioned above, once you upgrade to azure-mgmt-resource 15.0.0, the code breaks because we introduced a new authentication mechanism that integrates with Azure Identity. If your code breaks because of this, then you can relate to the guide above to see how if you can authenticate using the new library.

jamescross91 commented 3 years ago

Yup got that part - but the Azure ML SDK expects authentication to work like this: https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/manage-azureml-service/authentication-in-azureml/authentication-in-azureml.ipynb

nickzhums commented 3 years ago

@00Kai0 @jsntcy could you provide some insight on this

jsntcy commented 3 years ago

Close this as all track 2 migration issue is tracked in https://github.com/Azure/azure-sdk-for-python/issues/16984.