microsoft / AzureStorageExplorer

Easily manage the contents of your storage account with Azure Storage Explorer. Upload, download, and manage blobs, files, queues, tables, and Cosmos DB entities. Gain easy access to manage your virtual machine disks. Work with either Azure Resource Manager or classic storage accounts, plus manage and configure cross-origin resource sharing (CORS) rules.
Creative Commons Attribution 4.0 International
377 stars 87 forks source link

Failure to Copy folder from one Storage Account to another #8278

Open Imran-githu8 opened 2 weeks ago

Imran-githu8 commented 2 weeks ago

Preflight Checklist

Storage Explorer Version

1.36.2

Regression From

N/A

Architecture

i86

Storage Explorer Build Number

20241029.2

Platform

Windows

OS Version

Windows 11

Bug Description

I am trying to copy a folder from one storage account to another storage account using Azure Storage Explorer and having the below:

{
  "name": null,
  "message": "\"failed to perform copy command due to error: cannot start job due to error: cannot list files due to reason GET https://xxxxxxxxxx.blob.core.windows.net/xxxxxxxxxx\\n--------------------------------------------------------------------------------\\nRESPONSE 403: 403 This request is not authorized to perform this operation.\\nERROR CODE: AuthorizationFailure\\n--------------------------------------------------------------------------------\\n<?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?><Error><Code>AuthorizationFailure</Code><Message>This request is not authorized to perform this operation.\\nRequestId:30353ea5-f01e-0032-0b5f-3052ba000000\\nTime:2024-11-06T15:23:16.0031713Z</Message></Error>\\n--------------------------------------------------------------------------------\\n.\\n\""
}

I have added my Ip addresses in the network settings for both the storage accounts. And I am able to read and create files/folders perfectly fine through Azure Storage Explorer or the Azure Portal.

Steps to Reproduce

  1. Lauch Storage Explorer.
  2. Expand to the Source location/Container in Storage account
  3. Copy the Folder inside the Container.
  4. Expand the Target Storage Account and Container inside it.
  5. Paste the Folder inside the Container.

Actual Experience

I am trying to copy a folder from one storage account to another storage account using Azure Storage Explorer and it is failing.

I have added my Ip addresses in the network settings for both the storage accounts. And I am able to read and create files/folders perfectly fine through Azure Storage Explorer or the Azure Portal but copy is not working.,

Expected Experience

It should copy the folder to another storage folder without any problems.

Additional Context

No response

craxal commented 2 weeks ago

@Imran-githu8 A copy operation requires authentication for both the source and destination. Authentication for the source is accomplished by generating either a storage account key-based SAS or a user-delegated SAS. This is done for security reasons, because we don't want to paste user credentials onto the clipboard, and the valid time range of SAS can be controlled.

Do you have permission to list account keys or generate user-delegated SAS for the source storage account?

Imran-githu8 commented 1 day ago

@craxal yes I do have permissions to create SAS token for the storage account.

Just to add I have tried with moving files and it works fine. But when I move folder with multiple folders and files inside it, then it fails.

craxal commented 1 day ago

@Imran-githu8 Are you working with ADLS Gen2 blob containers? If so, do you have the necessary permissions to read permissions to copy from the source items and the necessary write permissions to create the destination items?

@JasonYeMSFT Please add any additional comments if you have any.

craxal commented 1 day ago

Please also verify your machine architecture as well as the app architecture. We do not support i86 architectures.

Imran-githu8 commented 1 day ago

@craxal @JasonYeMSFT

I do have necessary permissions on both the storage account/containers. I have doubled checked it by reading and writing manually by going to each storage account (from and to copying).

the actual error I am having:

{ "name": null, "message": "\"failed to perform copy command due to error: cannot start job due to error: cannot list files due to reason GET https://xxxxxxxxxx.blob.core.windows.net/xxxxxxx\\n--------------------------------------------------------------------------------\\nRESPONSE 403: 403 This request is not authorized to perform this operation.\nERROR CODE: AuthorizationFailure\n--------------------------------------------------------------------------------\n<?xml version=\\"1.0\\" encoding=\\"utf-8\\"?>AuthorizationFailureThis request is not authorized to perform this operation.\nRequestId:841cffa9-001e-006b-6969-3bd539000000\nTime:2024-11-20T16:30:48.4902832Z\n--------------------------------------------------------------------------------\n.\n\"" }

My machines architecture:

Processor: x64-based processor OS: Windows 11 64-bit operating system App: x64

Imran-githu8 commented 1 day ago

@craxal @JasonYeMSFT

I checked the log files, and found the below but it didn't make much sense to me that why it is not able to check the destination file existence:

2024/11/21 10:59:39 ERR: [P#1-T#4118] COPYFAILED: https://xxxxxx.blob.core.windows.net/xxxxx/xxxx/DataFeedName=xxxxx/RunYear=xxxx/RunMonth=xx/RunDay=21/RunId=xxxxxxx/xxxxx-2657.json : 000 : Could not check destination file existence. context canceled Dst: https://xxxxxx.blob.core.windows.net/xxxx/xxxx/DataFeedName=xxxxx/RunYear=xxxx/RunMonth=xx/RunDay=21/RunId=xxxxxxxx/xxxxx-2657.json

craxal commented 15 hours ago

What happens if you run the AzCopy command directly in the command line? You can click the "Copy AzCopy Command to Clipboard" action from Storage Explorer and paste it in the command line (might have to modify the command name to point to the correct location).