TokenLibrary does not guarantee b64 encoded string returned, will fail in blob service actions

newnativeabq commented 1 year ago

The following python code in Azure Synapse pipelines will fail as the provided TokenLibrary will not by default pass appropriately formatted credentials back to the interpreter:

from azure.storage.blob import BlobServiceClient

# TokenLibrary is default available in Azure Synapse notebook session.  May not be available locally
token_library = spark._jvm.com.microsoft.azure.synapse.tokenlibrary.TokenLibrary
linked_service_connection_string = token_library.getConnectionString(linked_service_name)

blob_service_client = BlobServiceClient(account_url=f"https://{storage_account}.blob.core.windows.net", credential=linked_service_connection_string)

# upload a blob
container_client = blob_service_client.get_container_client(container=container)
with open(temp_file_path, mode='rb') as data:
    blob_client = container_client.upload_blob(name='sample-blob.txt', data=data, overwrite=True)

Trace high level:

cluster-env/env/lib/python3.6/site-packages/azure/storage/blob/_shared/authentication.py in _add_authorization_header(self, request, string_to_sign)

Confirm by attempting to validate token

Manual validation of token output shows it isn't properly b64 encoded.

Guestimates

The ._jvc action is passing incorrect credentials to ipython client under certain circumstances in Azure Synapse Notebook. Possible some padding will work, meaning the credentials might work some of the times and not others. I have not been able to get it working.

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 56d638cd-6592-ce95-90aa-dc8d6e4e90ba
Version Independent ID: 5360961f-6bfc-ecad-cf49-22eee26bd010
Content: Secure access credentials with Linked Services in Apache Spark for Azure Synapse Analytics - Azure Synapse Analytics
Content Source: articles/synapse-analytics/spark/apache-spark-secure-credentials-with-tokenlibrary.md
Service: synapse-analytics
Sub-service: spark
GitHub Login: @mlee3gsd
Microsoft Alias: martinle

RamanathanChinnappan-MSFT commented 1 year ago

@newnativeabq

Thanks for your feedback! We will investigate and update as appropriate.

RamanathanChinnappan-MSFT commented 1 year ago

@newnativeabq

I've delegated this to @mlee3gsd, a content author, to review and share their valuable insights.

bandersmsft commented 1 month ago

Thanks for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. We are closing this issue for now, but if you feel that it's still a concern, please respond and let us know. If you determine another possible update to our documentation, please don't hesitate to reach out again. #please-close

MicrosoftDocs / azure-docs