johnsonr1994
commented
4 months ago I was able to get this exact functionality working using az storage blob upload-batch so it would seem that this issue appears to be isolated to the azcopy tool.
Open johnsonr1994 opened 5 months ago
johnsonr1994
commented
4 months ago I was able to get this exact functionality working using az storage blob upload-batch so it would seem that this issue appears to be isolated to the azcopy tool.
Which version of the AzCopy was used?
10.24.0
Which platform are you using? (ex: Windows, Mac, Linux)
Windows
What command did you run?
Authenticate with azcopy via service principal and run: "azcopy.exe" copy "C:\PathToFolderToCopy\*" "https://REDACTED.blob.core.windows.net/REDACTED/TESTING/testlocal/" --blob-type=BlockBlob --recursive=true
What problem was encountered?
Status Code of 400 received for all files and the xml response of "Response Details: <Code>InvalidResourceName</Code><Message>The specifed resource name contains invalid characters. </Message>"
Exact Response Block with Info:
2024/05/16 17:19:12 ==> REQUEST/RESPONSE (Try=1/21.3856ms, OpTime=75.4896ms) -- RESPONSE STATUS CODE ERROR PUT https://REDACTED.blob.core.windows.net/REDACTED/TESTING%2Ftestlocal%2FTest%20Files%2FSample5.PDF Accept: application/xml Authorization: REDACTED Content-Length: 51763 Content-Type: application/octet-stream User-Agent: AzCopy/10.24.0 azsdk-go-azblob/v1.3.1 (go1.19.12; Windows_NT) X-Ms-Client-Request-Id: f80a7b07-845a-4077-7efc-7f7e78853ecc x-ms-blob-content-type: application/pdf x-ms-blob-type: BlockBlob x-ms-version: 2023-08-03
RESPONSE Status: 400 The specifed resource name contains invalid characters. Content-Length: 243 Content-Type: application/xml Date: Thu, 16 May 2024 17:19:10 GMT Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0 X-Ms-Client-Request-Id: f80a7b07-845a-4077-7efc-7f7e78853ecc X-Ms-Error-Code: InvalidResourceName X-Ms-Request-Id: 4782f06f-101e-0027-61b5-a74005000000 X-Ms-Version: 2023-08-03 Response Details: The specifed resource name contains invalid characters.
InvalidResourceName2024/05/16 17:19:12 ERR: [P#0-T#7] UPLOADFAILED: \?\C:\PathToFolderToCopy\Test Files\Sample5.PDF : 400 : 400 The specifed resource name contains invalid characters.. When Uploading blob. X-Ms-Request-Id: 4782f06f-101e-0027-61b5-a74005000000
Dst: https://REDACTED.blob.core.windows.net/REDACTED/TESTING/testlocal/Test%20Files/Sample5.PDF
By just looking at the source file path, it looks like the front of path has a strange
\\?\encoding to it. That is what obviously sticks out as what could be the issue, but I am not sure where that encoding is coming from. The input format is simplyC:\PathToFolderToCopyThe PUT call is also getting URL encodeded with
%2Fgetting inserted for forward slashes. That could also be problematic.How can we reproduce the problem in the simplest way?
Authenticate with azcopy via an azure service principal and then run the above azcopy command with the source files input containing an absolute path to the directory to wholesale copy. Ensure that you are also doing a recursive copy to retain directory structure
Have you found a mitigation/solution?
No
The use case of this is running a powershell script to copy all subdirectories with files to azure blob storage under a certain blob prefix. I was able to get this to work using the
AzureFileCopy@6azure pipelines task, but the environment variable transformation limitations with yaml pipelines led me to try and recreate this functionality in a powershell script that can be executed elsewhere.I found an encoding issue that seems similar but it was affecting the c# azure sdk -> https://github.com/Azure/azure-sdk-for-net/issues/15064 I am not sure if this issue is related, but it appears to be an autoencoding issue that is causing this.