Azure / azure-cli

Azure Command-Line Interface
MIT License
3.98k stars 2.96k forks source link

Show Status of All Blobs during az storage blob copy start-batch #24032

Open jmpompeo opened 2 years ago

jmpompeo commented 2 years ago

Hello,

I'm trying to copy data between azure storage accounts and decided to use the az storage blob copy start-batch command. I noticed on Microsoft's documentation you can check the status of an individual blob, but you cannot check the overall status of the command, so it feels like I'm just sitting in limbo until it either succeeds or fails.

More detailed command below:

az storage blob copy start-batch --account-key 'key' --account-name 'name' --destination-container 'container' --source-account-key 'key1' --source-account-name 'name1' --source-container 'container1'.

I originally tried using azcopy for this but it was painfully slow, so I've been searching for an alternative that might work faster. AzCopy shows the status as it's copying the files and it also shows the speed at which it's copying. This just hangs on the screen, and I'm still waiting for it to complete with no idea if it will finish properly or will time out. Also on the documentation it says you can use az storage blob show to check the status of the blobs(screenshot below).

azcopy batch

Also, I should add these containers have subdirectories inside them that are being copied over, so even if we could specify a particular directory instead of the only option being a specific blob.

Another note is that the destination storage account is a premium data lake gen 2 account. I don't think that should make a difference, but I wanted to include any & all information.

Did I miss something in the documentation? If not, I think this could be a very useful feature to see the status of the copy operation as a whole.

yonzhan commented 2 years ago

@evelyn-ys for awareness

MoChilia commented 1 year ago

Hi @jmpompeo! If you want to check the status of the blobs in a particular directory when running az storage blob copy start-batch, I recommend you to use the command az storage blob list.

ghost commented 1 year ago

Hi @jmpompeo. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

ghost commented 1 year ago

Hi @jmpompeo, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.

mihaiandreiratoiu commented 1 year ago

/unresolve

ghost commented 1 year ago

Hi mihaiandreiratoiu, only the original author of the issue can ask that it be unresolved. Please open a new issue with your scenario and details if you would like to discuss this topic with the team.

mihaiandreiratoiu commented 1 year ago

I would suggest adding at least the --requires-sync from the az storage blob copy start

jmpompeo commented 1 year ago

/unresolve