az webapp deploy tracking stuck 'Pending' deployments

glenthomas commented 2 months ago

Describe the bug

My web app deployments are getting stuck in a ‘Pending’ state and I think the CLI gets confused with which deployment it should be tracking. Here is a screenshot of the deployment logs of the web app taken just now:

As you can see, there is a deployment at 10:56:24, which has status ‘Pending’. There are 3 subsequent deployments with status ‘Success’. The azure CLI appears to be triggering new deployments, but then tracks the status of the Pending deployment. So, in our deployment logs (client-side) we see this.

Wed, 18 Sep 2024 09:56:16 GMT WARNING: Initiating deployment
Wed, 18 Sep 2024 09:56:16 GMT WARNING: Deploying from local path: ./checkout-release.zip
Wed, 18 Sep 2024 10:02:25 GMT Warning: Attempt 1 failed. Reason: Timeout of 360000ms hit
Wed, 18 Sep 2024 10:02:26 GMT WARNING: Initiating deployment
Wed, 18 Sep 2024 10:02:26 GMT WARNING: Deploying from local path: ./checkout-release.zip
Wed, 18 Sep 2024 10:08:36 GMT Warning: Attempt 2 failed. Reason: Timeout of 360000ms hit
Wed, 18 Sep 2024 10:08:36 GMT WARNING: Initiating deployment
Wed, 18 Sep 2024 10:08:36 GMT WARNING: Deploying from local path: ./checkout-release.zip
Wed, 18 Sep 2024 10:14:46 GMT Warning: Attempt 3 failed. Reason: Timeout of 360000ms hit
Wed, 18 Sep 2024 10:14:47 GMT WARNING: Initiating deployment
Wed, 18 Sep 2024 10:14:47 GMT WARNING: Deploying from local path: ./checkout-release.zip
Wed, 18 Sep 2024 10:20:56 GMT Warning: Attempt 4 failed. Reason: Timeout of 360000ms hit
Wed, 18 Sep 2024 10:20:57 GMT WARNING: Initiating deployment
Wed, 18 Sep 2024 10:20:57 GMT WARNING: Deploying from local path: ./checkout-release.zip
Wed, 18 Sep 2024 10:27:07 GMT Error: Final attempt failed. Timeout of 360000ms hit

As you can see, the timestamps of these az webapp deploy commands align with the deployment logs of the web app. The web app thinks that attempts 2, 3 and 4 succeeded, but the CLI did not receive a success status. I suspect that on each attempt it was tracking the Pending deployment from 10:56 instead of the newly triggered deployments.

This is a frequent problem for us. Here is another example of deployment logs from today from another web app. You can see 2 deployments are stuck in a Pending state with a later deployment succeeding:

If we run another az webapp deploy command we will end up with another deployment, but I think the CLI gets stuck looking at the Pending deployment and never completes the command.

Previously the CLI would have errored with a message like "a deployment is in progress, please wait before starting a new deployment. you can track it here...", but the deploy functionality appears to have recently changed and new bugs introduced.

Related command

az webapp deploy

Errors

No specific error

Issue script & Debug output

This is not something I can easily do in my current environment.

Expected behavior

The command should complete successfully.

Environment Summary

azure-cli 2.64.0

Additional context

No response

yonzhan commented 2 months ago

Thank you for opening this issue, we will look into it.

microsoft-github-policy-service[bot] commented 2 months ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @AzureAppServiceCLI, @antcp.

btardif commented 2 months ago

@tulikac can you take a look?

copdips commented 6 days ago

hello, can we have some updates please ?

Azure / azure-cli