az webapp deploy fails with just "Bad Request" (using `--src-url`)

kdambekalns commented 5 days ago

Describe the bug

When running the az webapp deploy command below, all I get is a "Bad Request" response. Pretty hard to debug that…

Related command

az webapp deploy --resource-group ACME-Dev-RG --name acmecom-dev --type zip --src-url 'https://acmecom.blob.core.windows.net/artifacts/release.zip?__SAS__'

(Tried with an explicit --async true as well, to avoid any side effects of https://github.com/Azure/azure-cli/issues/27506)

Errors

Initiating deployment
Deploying from URL: https://acmecom.blob.core.windows.net/artifacts/release.zip?__SAS__
Bad Request

Issue script & Debug output

This has been redacted and shortened a bit (removed the successful informaton retrieval about the webapp, for example).

I hope the relevant bits are all there.

cli.azure.cli.command_modules.appservice.custom: Initiating deployment
cli.azure.cli.command_modules.appservice.custom: Deploying from URL: https://acmecom.blob.core.windows.net/artifacts/release.zip?__SAS__
cli.azure.cli.core.util: Request URL: 'https://management.azure.com//subscriptions/__SUBSCRIPTION__/resourceGroups/ACME-Dev-RG/providers/Microsoft.Web/sites/acmecom?api-version=2023-12-01'
cli.azure.cli.core.util: Request method: 'GET'
cli.azure.cli.core.util: Request headers:
cli.azure.cli.core.util:     'User-Agent': 'python/3.11.9 (macOS-14.5-arm64-arm-64bit) AZURECLI/2.61.0 (HOMEBREW)'
cli.azure.cli.core.util:     'Accept-Encoding': 'gzip, deflate'
cli.azure.cli.core.util:     'Accept': '*/*'
cli.azure.cli.core.util:     'Connection': 'keep-alive'
cli.azure.cli.core.util:     'x-ms-client-request-id': 'e1e91944-6092-450a-b908-5a86e921d5c1'
cli.azure.cli.core.util:     'CommandName': 'webapp deploy'
cli.azure.cli.core.util:     'ParameterSetName': '--debug --resource-group --name --async --type --src-url'
cli.azure.cli.core.util:     'Authorization': 'Bearer eyJ0eXAiOiJKV...'
cli.azure.cli.core.util: Request body:
cli.azure.cli.core.util: None
cli.azure.cli.core.util: Response status: 200
cli.azure.cli.core.util: Response headers:
cli.azure.cli.core.util:     'Cache-Control': 'no-cache'
cli.azure.cli.core.util:     'Pragma': 'no-cache'
cli.azure.cli.core.util:     'Content-Length': '8309'
cli.azure.cli.core.util:     'Content-Type': 'application/json'
cli.azure.cli.core.util:     'Expires': '-1'
cli.azure.cli.core.util:     'ETag': '"1DAC8AFD47ACAA0"'
cli.azure.cli.core.util:     'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
cli.azure.cli.core.util:     'X-AspNet-Version': '4.0.30319'
cli.azure.cli.core.util:     'X-Powered-By': 'ASP.NET'
cli.azure.cli.core.util:     'x-ms-ratelimit-remaining-subscription-reads': '249'
cli.azure.cli.core.util:     'x-ms-ratelimit-remaining-subscription-global-reads': '3749'
cli.azure.cli.core.util:     'X-Content-Type-Options': 'nosniff'
cli.azure.cli.core.util:     'X-Cache': 'CONFIG_NOCACHE'
cli.azure.cli.core.util:     'Date': 'Mon, 01 Jul 2024 18:40:15 GMT'
cli.azure.cli.core.util: Response content:
cli.azure.cli.core.util: {"id":"/subscriptions/__SUBSCRIPTION__/resourceGroups/ACME-Dev-RG/providers/Microsoft.Web/sites/acmecom","name":"acmecom","type":"Microsoft.Web/sites","kind":"app,linux","location":"East US","tags":{"Environment":"Development"},"properties":{"name":"acmecom","state":"Running","hostNames":["acmecom.azurewebsites.net"],"webSpace":"ACME-Dev-RG-EastUSwebspace-Linux","selfLink":"https://waws-prod-blu-495.api.azurewebsites.windows.net:454/subscriptions/__SUBSCRIPTION__/webspaces/ACME-Dev-RG-EastUSwebspace-Linux/sites/acmecom","repositorySiteName":"acmecom","owner":null,"usageState":"Normal","enabled":true, … ,"sshEnabled":null}}
cli.azure.cli.command_modules.appservice.custom: Deployment API: https://management.azure.com/subscriptions/__SUBSCRIPTION__/resourceGroups/ACME-Dev-RG/providers/Microsoft.Web/sites/acmecom/extensions/onedeploy?api-version=2023-01-01
cli.azure.cli.core.util: Request URL: 'https://management.azure.com/subscriptions/__SUBSCRIPTION__/resourceGroups/ACME-Dev-RG/providers/Microsoft.Web/sites/acmecom/extensions/onedeploy?api-version=2023-01-01'
cli.azure.cli.core.util: Request method: 'PUT'
cli.azure.cli.core.util: Request headers:
cli.azure.cli.core.util:     'User-Agent': 'python/3.11.9 (macOS-14.5-arm64-arm-64bit) AZURECLI/2.61.0 (HOMEBREW)'
cli.azure.cli.core.util:     'Accept-Encoding': 'gzip, deflate'
cli.azure.cli.core.util:     'Accept': '*/*'
cli.azure.cli.core.util:     'Connection': 'keep-alive'
cli.azure.cli.core.util:     'Content-Type': 'application/json'
cli.azure.cli.core.util:     'CommandName': 'webapp deploy'
cli.azure.cli.core.util:     'ParameterSetName': '--debug --resource-group --name --async --type --src-url'
cli.azure.cli.core.util:     'Authorization': 'Bearer eyJ0eXAiOiJKV...'
cli.azure.cli.core.util:     'Content-Length': '392'
cli.azure.cli.core.util: Request body:
cli.azure.cli.core.util: {"properties": {"packageUri": "https://acmecom.blob.core.windows.net/artifacts/release.zip?__SAS__", "type": "zip"}}
cli.azure.cli.core.util: Response status: 400
cli.azure.cli.core.util: Response headers:
cli.azure.cli.core.util:     'Cache-Control': 'no-store, no-cache'
cli.azure.cli.core.util:     'Pragma': 'no-cache'
cli.azure.cli.core.util:     'Expires': '-1'
cli.azure.cli.core.util:     'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
cli.azure.cli.core.util:     'X-AspNet-Version': '4.0.30319'
cli.azure.cli.core.util:     'X-Powered-By': 'ASP.NET'
cli.azure.cli.core.util:     'x-ms-ratelimit-remaining-subscription-writes': '199'
cli.azure.cli.core.util:     'x-ms-ratelimit-remaining-subscription-global-writes': '2999'
cli.azure.cli.core.util:     'X-Content-Type-Options': 'nosniff'
cli.azure.cli.core.util:     'X-Cache': 'CONFIG_NOCACHE'
cli.azure.cli.core.util:     'Date': 'Mon, 01 Jul 2024 18:40:20 GMT'
cli.azure.cli.core.util:     'Content-Length': '0'
cli.azure.cli.core.util: Response content:
cli.azure.cli.core.util:

There seems to be no response body whatsoever, so all I have is 400 Bad request

Expected behavior

The deployment is triggered…

Environment Summary

azure-cli 2.61.0

core 2.61.0 telemetry 1.1.0

Extensions: webapp 0.4.0

Dependencies: msal 1.28.0 azure-mgmt-resource 23.1.1

Python location '/opt/homebrew/Cellar/azure-cli/2.61.0/libexec/bin/python' Extensions directory '/Users/karsten/.azure/cliextensions'

Python (Darwin) 3.11.9 (main, Apr 2 2024, 08:25:04) [Clang 15.0.0 (clang-1500.3.9.4)]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

The download URL given to the command is valid and works…

yonzhan commented 5 days ago

Thank you for opening this issue, we will look into it.

microsoft-github-policy-service[bot] commented 5 days ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @AzureAppServiceCLI, @antcp.

kdambekalns commented 4 days ago

FYI: I also tried using az rest to call the deployment API endpoit directly, as suggested in https://azure.github.io/AppService/2021/03/01/deploying-to-network-secured-sites-2.html - even though it seemed far-fetched, given that article is from March 2021. As expected it didn't make a difference.

kdambekalns commented 3 days ago

We did some further debugging… It fails with 400 Bad request for

a SAS URL created using az storage blob generate-sas --full-uri --permissions r --expiry '2024-07-03T09:09Z' --account-name acmecom -c artifacts -n release.zip --auth-mode login --as-user
a SAS URL created using az storage blob generate-sas --full-uri --permissions r --expiry '2024-07-03T09:09Z' --account-name acmecom -c artifacts -n release.zip (with account name & key)
an URL like https://www.acme.com/release.zip that points to a non-existing ZIP

When using a URL like https://www.acme.com/release.zip that points to an existing ZIP, it works – so it doesn't like the "fancy" SAS URL? For fun I passed a rawurlencoded URL of the SAS URL, and that failed with Invalid packageUrl in the JSON request.

So the SAS URL is the issue – either it is checked and access fails (even though I can use it just fine!) or some validation goes wrong!?

kdambekalns commented 3 days ago

Could it be the network blocks access to the storage container from the deployment service? We have "Public network access: Enabled from selected virtual networks and IP addresses" configured.

Additionally enabling "Allow Azure services on the trusted services list to access this storage account." does not fix it…
Changing it to "Public network access: Enabled from all networks" does fix it.

I'll try adding a private endpoint and see where that leads me.

kdambekalns commented 3 days ago

I'll try adding a private endpoint and see where that leads me.

The combination of a private endpoint and SAS URL does work fine, it was indeed the inaccessible ZIP that caused the 400 Bad request.

If there could be a better error message for such a case, that would have saved me some hours… 😬

Azure / azure-cli