search

microsoft / AzureStorageExplorer

Easily manage the contents of your storage account with Azure Storage Explorer. Upload, download, and manage blobs, files, queues, tables, and Cosmos DB entities. Gain easy access to manage your virtual machine disks. Work with either Azure Resource Manager or classic storage accounts, plus manage and configure cross-origin resource sharing (CORS) rules.
Creative Commons Attribution 4.0 International
365 stars 85 forks source link

Azure Storage Explore cannot clone or copy blobs after first container backup to an Azure Backup vault #7882

Open rssllptrs opened 2 months ago

rssllptrs commented 2 months ago

Preflight Checklist

Storage Explorer Version

1.33.1 (98)

Regression From

No response

Architecture

x64

Storage Explorer Build Number

20240410.2

Platform

Windows

OS Version

Windows 11

Bug Description

After configuring a Storage Account (non ADLS) for backup to an Azure Backup Vault, and after the first backup, all clones and copy/paste operations of blobs from containers that were configured for backup yield an error "RESPONSE Status: 400 The metadata specified is invalid. It has characters that are not permitted." Note that copy and clone from the Azure Portal "Storage Browser" work find and as expected. Also copy/paste and clone operations from containers that are not backed up to an Azure Backup vault work as expected. 81c9c2d8-290e-784b-6cee-65db5696e2c9.log

Steps to Reproduce

For an existing storage account and an existing backup vault:

  1. Verify that clones and copies work as expected using storage explorer.
  2. configure the storage account for backup to the vault using the procedure in this document: https://learn.microsoft.com/en-us/azure/backup/blob-backup-configure-manage?tabs=operational-backup
  3. Issue a manual backup (or wait for the scheduled backup to occur.
  4. Attempt to clone or copy same blobs as in step 1 to show they no longer work. You can still download and upload fine and you can clone/copy from Azure portal fine.

Actual Experience

Paste and Clone simply do not work. They start, but fail at the very end (appears to be related to metadata issues).

Expected Experience

Expectation is for clone/copy to work like for containers that don't have Backup configured.

Additional Context

NA

JasonYeMSFT commented 2 months ago

@rssllptrs From the log, the copy/clone job is trying to set the metadata pair with name "E2bc6aa5-07df-4039-A05f-69a29f48c7e5_8179f480-Fb83-419b-90a7-B20c5a7233fd" and value "complete". I am pretty sure this is not a valid metadata name. It violates not only Storage Explorer's validation rule but also Azure Portal's. image

Could you view the properties of the source blob and see what Storage Explorer display as the metadata name? If it's different from what I pasted above, please let us know what Storage Explorer displays. Otherwise, if Azure Backup Vault somehow manages to set the metadata with an invalid name, it's a serious bug that they need to look at as soon as possible. Could you reach out to Azure Backup Vault team to find out?

rssllptrs commented 2 months ago

Hi Jason,

Using Storage explorer, when viewing the blob properties there is no custom metadata on the blob in question. However, within the "Storage Browser" in the Azure Portal, when viewing the properties, I do see a new set of fields under the heading "Object replication" (which only exists for blobs that are protected by an Azure Backup Vault policy). The fields are Policy ID, Rule ID, and Status. Incidentally, the Policy ID matches the invalid metadata key in the log file and the Status = "Completed" matches the corresponding invalid metadata value. Perhaps this isn't user modifiable metadata, but some internal (new) data that Storage Explorer is yet to support and is treating as metadata. I have opened a case in Azure describing the issue -- case: 2404190040009165.

On Thu, Apr 18, 2024 at 4:48 PM JasonYeMSFT @.***> wrote:

@rssllptrs https://github.com/rssllptrs From the log, the copy/clone job is trying to set the metadata pair with name "E2bc6aa5-07df-4039-A05f-69a29f48c7e5_8179f480-Fb83-419b-90a7-B20c5a7233fd" and value "complete". I am pretty sure this is not a valid metadata name. It violates not only Storage Explorer's validation rule but also Azure Portal's. image.png (view on web) https://github.com/microsoft/AzureStorageExplorer/assets/39359541/5adcbdc0-edf0-45c3-8894-a61766762fa0

Could you view the properties of the source blob and see what Storage Explorer display as the metadata name? If Azure Backup Vault somehow manages to set the metadata with an invalid name, it's a serious bug that they need to look at as soon as possible. Could you reach out to Azure Backup Vault team to find out?

— Reply to this email directly, view it on GitHub https://github.com/microsoft/AzureStorageExplorer/issues/7882#issuecomment-2065297919, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADTVTKWFNWJDSJDXCL5OYJDY6AWSVAVCNFSM6AAAAABGLZEJ72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRVGI4TOOJRHE . You are receiving this because you were mentioned.Message ID: @.***>

JasonYeMSFT commented 2 months ago

AzCopy is incorrectly treating this object replication header as metadata. I have notified their dev on this bug.

MRayermannMSFT commented 2 months ago

Currently slotted for 10.26.0 (July 24). Maybe we'll shift our release a bit to catch 10.26.0. We'll see. @JasonYeMSFT please track this for 1.35. :)