microsoft / azure-container-apps

Roadmap and issues for Azure Container Apps
MIT License
372 stars 29 forks source link

Cannot start a new revision on default configuration, set to Degraded #1347

Closed omni-htg closed 3 days ago

omni-htg commented 4 days ago

Please provide us with the following information:

This issue is a: (mark with an x)

Issue description

Apologies for the awful title! Starting today, deployment and startup of a new Revision seems to fail and puts the whole revision in a Degraded state. The Container Apps we use are configured with either default scaling rules (ie. the one you get without making a new scale rule) or no Ingress at all. We have a separate environment where the same image was able to be deployed without issues on Friday, but today we are experiencing these logs:

{"TimeStamp":"2024-11-18T11:34:43Z","Type":"Normal","ContainerAppName":null,"RevisionName":null,"ReplicaName":null,"Msg":"Connecting to the events collector...","Reason":"StartingGettingEvents","EventSource":"ContainerAppController","Count":1} {"TimeStamp":"2024-11-18T11:34:44Z","Type":"Normal","ContainerAppName":null,"RevisionName":null,"ReplicaName":null,"Msg":"Successfully connected to events server","Reason":"ConnectedToEventsServer","EventSource":"ContainerAppController","Count":1} {"TimeStamp":"2024-11-18 11:34:22 \u002B0000 UTC","Type":"Warning","ContainerAppName":"","RevisionName":"","ReplicaName":"","Msg":"ScaledObject doesn\u0027t have correct triggers specification","Reason":"ScaledObjectCheckFailed","EventSource":"KEDA","Count":9} {"TimeStamp":"2024-11-18 11:34:24 \u002B0000 UTC","Type":"Warning","ContainerAppName":"","RevisionName":"","ReplicaName":"","Msg":"ScaledObject doesn\u0027t have correct triggers specification","Reason":"ScaledObjectCheckFailed","EventSource":"KEDA","Count":10} {"TimeStamp":"2024-11-18 11:34:29 \u002B0000 UTC","Type":"Warning","ContainerAppName":"","RevisionName":"","ReplicaName":"","Msg":"ScaledObject doesn\u0027t have correct triggers specification","Reason":"ScaledObjectCheckFailed","EventSource":"KEDA","Count":11} {"TimeStamp":"2024-11-18 11:34:37 \u002B0000 UTC","Type":"Normal","ContainerAppName":"","RevisionName":"","ReplicaName":"-744d76667b-79pdh","Msg":"Pulling image \u0027\u0027","Reason":"PullingImage","EventSource":"ContainerAppController","Count":1} {"TimeStamp":"2024-11-18 11:34:40 \u002B0000 UTC","Type":"Warning","ContainerAppName":"","RevisionName":"","ReplicaName":"","Msg":"ScaledObject doesn\u0027t have correct triggers specification","Reason":"ScaledObjectCheckFailed","EventSource":"KEDA","Count":12} {"TimeStamp":"2024-11-18 11:34:49 \u002B0000 UTC","Type":"Warning","ContainerAppName":"","RevisionName":"","ReplicaName":"","Msg":"ScaledObject was removed from KEDA watch and would not be auto-scaled. Please check https://learn.microsoft.com/en-us/azure/container-apps/dapr-overview","Reason":"ScaledObjectDeleted","EventSource":"KEDA","Count":1} {"TimeStamp":"2024-11-18 11:35:00 \u002B0000 UTC","Type":"Warning","ContainerAppName":"","RevisionName":"","ReplicaName":"","Msg":"ScaledObject doesn\u0027t have correct triggers specification","Reason":"ScaledObjectCheckFailed","EventSource":"KEDA","Count":13} {"TimeStamp":"2024-11-18 11:35:41 \u002B0000 UTC","Type":"Warning","ContainerAppName":"","RevisionName":"","ReplicaName":"","Msg":"ScaledObject doesn\u0027t have correct triggers specification","Reason":"ScaledObjectCheckFailed","EventSource":"KEDA","Count":14} {"TimeStamp":"2024-11-18T11:36:42Z","Type":"Normal","ContainerAppName":null,"RevisionName":null,"ReplicaName":null,"Msg":"No events since last 60 seconds","Reason":"NoNewEvents","EventSource":"ContainerAppController","Count":1} {"TimeStamp":"2024-11-18 11:37:03 \u002B0000 UTC","Type":"Warning","ContainerAppName":"","RevisionName":"","ReplicaName":"","Msg":"ScaledObject doesn\u0027t have correct triggers specification","Reason":"ScaledObjectCheckFailed","EventSource":"KEDA","Count":15} {"TimeStamp":"2024-11-18T11:38:04Z","Type":"Normal","ContainerAppName":null,"RevisionName":null,"ReplicaName":null,"Msg":"No events since last 60 seconds","Reason":"NoNewEvents","EventSource":"ContainerAppController","Count":1} {"TimeStamp":"2024-11-18T11:39:04Z","Type":"Normal","ContainerAppName":null,"RevisionName":null,"ReplicaName":null,"Msg":"No events since last 60 seconds","Reason":"NoNewEvents","EventSource":"ContainerAppController","Count":1} {"TimeStamp":"2024-11-18 11:39:47 \u002B0000 UTC","Type":"Warning","ContainerAppName":"","RevisionName":"","ReplicaName":"","Msg":"ScaledObject doesn\u0027t have correct triggers specification","Reason":"ScaledObjectCheckFailed","EventSource":"KEDA","Count":16} {"TimeStamp":"2024-11-18T11:40:48Z","Type":"Normal","ContainerAppName":null,"RevisionName":null,"ReplicaName":null,"Msg":"No events since last 60 seconds","Reason":"NoNewEvents","EventSource":"ContainerAppController","Count":1} {"TimeStamp":"2024-11-18T11:41:48Z","Type":"Normal","ContainerAppName":null,"RevisionName":null,"ReplicaName":null,"Msg":"No events since last 60 seconds","Reason":"NoNewEvents","EventSource":"ContainerAppController","Count":1}

Any information about this is very much appreciated, will be still checking out possible issues with our image in the meantime.

Steps to reproduce

Deploy a new Revision with default configuration on scaling.

Expected behavior [What you expected to happen.] Revision repiclas should initialize without issue.

Actual behavior [What actually happened.] Replicas seem to start and give logs as usual, but Revision is never promoted to Successful.

Screenshots
If applicable, add screenshots to help explain your problem. Image

Additional context

Ex. Did this issue occur in the CLI or the Portal? We deploy to ACR, then use the Azure CLI to deploy a new Revision in order to control the pacing of deployment. CLI completes successfully, then on the Portal we can see the Degraded state.

Thank you for your time! Kind regards.

omni-htg commented 3 days ago

Somehow, after the reporting the issue to Microsoft Support, this has "fixed itself" and now it works again. Closing down the ticket.