copy symlinks created blobs fails when copying to another container

microsoft / AzureStorageExplorer

Easily manage the contents of your storage account with Azure Storage Explorer. Upload, download, and manage blobs, files, queues, tables, and Cosmos DB entities. Gain easy access to manage your virtual machine disks. Work with either Azure Resource Manager or classic storage accounts, plus manage and configure cross-origin resource sharing (CORS) rules.

Creative Commons Attribution 4.0 International

370 stars 84 forks source link

copy symlinks created blobs fails when copying to another container #8127

Open dwr-psandhu opened 4 weeks ago

dwr-psandhu commented 4 weeks ago

Preflight Checklist

[X] I have installed the latest version of Storage Explorer.
[X] I have checked existing resources, including the troubleshooting guide and the release notes.
[X] I have searched for similar issues.

Storage Explorer Version

1.34.0

Regression From

N/A

Architecture

x64

Storage Explorer Build Number

No response

Platform

Windows

OS Version

No response

Bug Description

I used the --preserve-symlinks to copy from linux keeping the symbolic links as blobs. However these special blobs are not respected by storageexplorer when copying from one container to another. Downloads also simply fail to materialize a file on Windows.

Steps to Reproduce

Copy symbolic links from linux to a blob
Use storage explorer to copy these blobs to another container. This fails
Try to download it . It fails.

Actual Experience

Symbolic links disappear or are not copied over.

Expected Experience

No response

Additional Context

No response

craxal commented 3 weeks ago

@dwr-psandhu

How are you doing step 1? Are you able to successfully upload the symlinks using Storage Explorer?
Are you uploading to a standard blob container or an ADLS Gen2 container?
Are you trying to copy to a standard blob container or an ADLS Gen 2 container?
Can you provide the exact error messages you are seeing?

dwr-psandhu commented 3 weeks ago

Step 1 is by using azcopy --preserve-symlinks Both source and destination are Storage v2 general purpose No error messages. Just the blob that represent symlinks are not copied over to destination. They are missing.

craxal commented 3 weeks ago

Both source and destination are Storage v2 general purpose.

So hierarchical namespaces is not enabled, correct?

No error messages. Just the blob that represent symlinks are not copied over to destination. They are missing.

Interesting. Can you share both your app logs and your AzCopy logs? Can you copy the symlinks using AzCopy directly, and does it work as expected?

dwr-psandhu commented 2 weeks ago

Yes it says on the portal that "Hierarchical namespace Disabled" I am using azcopy command line from linux 10.24+ to copy symlinks. I can also use azcopy with --preserve-symlinks to copy between blob containers and that works fine. It is really storage explorer that doesn't have that option turned on, e.g. when i ask to see what azcopy command storage explorer uses it doesn't have "--preseve-symlinks" in it stated options.

craxal commented 2 weeks ago

@dwr-psandhu Correct, Storage Explorer is not using the --preserve-symlinks to copy. We will need to discuss the full implications of including this. We may just add an app setting to enable it.

dwr-psandhu commented 2 weeks ago

Ok, I believe it is a harmless option or perhaps there is another option that keeps metadata on blobs intact ?

craxal commented 2 weeks ago

@dwr-psandhu Metadata is a different concept than symbolic links in this context. Can you clarify what you mean?

dwr-psandhu commented 2 weeks ago

It is my understanding that when this feature was implemented it was as a blob with contents as name of the file being linked to and metadata property indicating it was a symlink https://github.com/Azure/azure-storage-azcopy/pull/1829/commits/873c86ab7001210fea258ce119d8f85c3970eb76