az storage blob upload can overwrite a blob without needing the --overwrite parameter in az cli version 2.33.1 , however in version 2.34.1 it requires the overwrite parameter.

[miniGweek]

This is autogenerated. Please review and update as needed.

Describe the bug

Command Name az storage blob upload

Detailed command

az storage blob upload --account-name <my-storage-account> --container-name '$web' --file <my-file> --auth-mode login Errors:

The specified blob already exists.
RequestId:xxxx-xxxxx-xxxx
Time:2022-03-24T04:01:57.5064729Z
ErrorCode:BlobAlreadyExists
If you want to overwrite the existing one, please add --overwrite in your command.

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

• Install the latest azure cli on your workstation - As of raising this bug this is v2.34.1
• Attempt to overwrite an existing blob in a container
• az storage blob upload --account-name --container-name '$web' --file --auth-mode login
• You will get the error The specified blob already exists
• Now install the version v2.33.1 of azure cli and try the same steps as above
• The file can be re-uploaded/overwritten again and again without any error.
• So in summary:
  • If you have azure cli v2.33.1 - az storage blob upload allows the upload of the same file to the same location allowing it to be overwritten.
  • if you have azure cli v2.34.1 - az storage blob upload doesn't allow the upload of the same file to the same location, instead throws an error Blob already exists and requires the experimental parameter --overwrite to be supplied.

Expected Behavior

• I don't expect this breaking change to be introduced. Azure CLI tasks ( for azure pipelines ) is using azure CLI v2.33.1 and the file upload works smoothly, but I am afraid as soon as the Azure CLI task gets upgraded to v2.34.1 my pipeline code will break.

Environment Summary

Windows-10-10.0.19041-SP0
Python 3.8.9
Installer: MSI

azure-cli 2.34.1

Extensions:
azure-devops 0.22.0
desktopvirtualization 0.1.1
interactive 0.4.5
log-analytics 0.2.2
next 0.1.2
rdbms-connect 1.0.1

Dependencies:
msal 1.16.0
azure-mgmt-resource 20.0.0

Additional Context

[yonzhan]

storage

[calvinhzy]

We have included this as a breaking change in 2.34.1 as we think it is dangerous to overwrite files silently. --overwrite needs to be specified to overwrite files. Sorry for the inconvenience. https://docs.microsoft.com/en-us/cli/azure/storage/blob?view=azure-cli-latest#az-storage-blob-upload

[norb62]

This change breaks every piece of 3rd party code using upload or upload-batch as soon as the az client is updated to 2.34.1 or higher. To workaround this, I need to manually detect the az version, then issue 2 different commands, depending on the version, since the --overwrite option does not exist in az storage upload before 2.34.1. It's not optimal imo.

--overwrite should be true by default and when not specified, to ensure backward compatibility. If a user wants to ensure that its data can't be overwritten, it could specify --overwrite false.

[tspearconquest]

The azure cli should not introduce this change as GA in my humble opinon.

While it is dangerous to overwrite files silently, it is worse to break existing code without a major version release.

As a proposal which wouldn't break existing code and also would alleviate both sides of this issue, why not provide 2 flags

1. --overwrite
2. --no-overwrite

Default to having --overwrite enabled when not specified as mentioned by @norb62 above, and print a warning on the terminal about overwriting when the flag is not set and about switching the default in a future release. In the next major version of the CLI, or after n number of minor version releases (10?) then switch the default.

This way the warning message can notify users that in the next major version of the provider (or after n number of minor releases), --no-overwrite will become the default. If a user wants to specify no overwrites now, they can, and if they want to prepare their code for the next major version while keeping the existing functionality, then they can specify the --overwrite flag and it will also prevent the warning from appearing.

This would be a much better way to handle the change.

[tspearconquest]

A true/false option for --overwrite would work just as well as a separate --no-overwrite option would, by the way. Either one is fine.

[evelyn-ys]

We do have true/false option for --overwrite

[tspearconquest]

Yes but the default is not as mentioned in my first comment.

As a proposal which wouldn't break existing code and also would alleviate both sides of this issue, why not provide 2 flags

1. --overwrite
2. --no-overwrite

Default to having --overwrite enabled when not specified as mentioned by @norb62 above, and print a warning on the terminal about overwriting when the flag is not set and about switching the default in a future release. In the next major version of the CLI, or after n number of minor version releases (10?) then switch the default.

[itsvijayid]

To Add to above the I have noticed Running this in AzureCLIv2 Pipeline Task as ps inline script upload-batch runs fine with out using --overwrite on subsequent runs it throws an error and suggests to use --overwrite handle which takes away the error by --overwriting and not throwing error that blob already exists . Problem here is when I move into another environment with everything fixed. Script complaints about error unrecognised arguments -- overwrite

I'm looking to find out if there is a work-around for this so that we can keep -- overwrite argument for subsequent runs and to exclude it for the first run on a new environment or blob container.

[karataliu]

Why --overwrite is in preview? Previously it is supported as default behavior, but now it is in preview, which means a GA feature is pulled back to preview.

Could we at least make --overwrite not preview as first step?