search

Azure / azure-dev

A developer CLI that reduces the time it takes for you to get started on Azure. The Azure Developer CLI (azd) provides a set of developer-friendly commands that map to key stages in your workflow - code, build, deploy, monitor, repeat.
https://aka.ms/azd
MIT License
402 stars 195 forks source link

[Issue] Failing invoking action 'deploy', no deployment manifests found for AKS deployment #3486

Closed dminkovski closed 6 months ago

dminkovski commented 6 months ago

Output from azd version azd version 1.6.1 (commit eba2c978b5443fdb002c95add4011d9e63c2e76f)

Describe the bug When running the AKS deployment of our repo: https://github.com/Azure-Samples/azure-search-openai-demo-java/tree/aca I am facing the following error during the "Verifying deployment" progress stage of azd up:

(x) Failed: Deploying service api

ERROR: failed deploying service 'api': failing invoking action 'deploy', no deployment manifests found

ERROR: error executing step command 'deploy --all': failed deploying service 'api': failing invoking action 'deploy', no deployment manifests found

There are k8s manifests files in the folder and also it seems to be found during the "Applying k8s manifests" stage.

To Reproduce

PS: You might face a different Resource Group error - which will be gone if you run it a second time...also it helps to run as Administrator to be sure in terms of rights.

Expected behavior On version 1.5.1 - the applications were successfully running and the k8s manifests were applied without troubles.

Additional context It seems to me something fails during the "waitForDeployment" operation in https://github.com/Azure/azure-dev/blob/main/cli/azd/pkg/project/service_target_aks.go#L331 because the last progress message I could observe was "Verifying deployment".

To me this error "no deployment manifests found" from line https://github.com/Azure/azure-dev/blob/main/cli/azd/pkg/project/service_target_aks.go#L258 seems a bit misleading to me...because I am getting the deployment output when using the --debug flag and I can see it in my cluster that my deployment is running:


    "items": [
        {
            "apiVersion": "apps/v1",
            "kind": "Deployment",
            "metadata": {
                "annotations": {
                    "deployment.kubernetes.io/revision": "8",
                    "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"backend\"},\"name\":\"backend-deployment\",\"namespace\":\"azure-open-ai\"},\"spec\":{\"replicas\":1,\"selector\":{\"matchLabels\":{\"app\":\"backend\"}},\"template\":{\"metadata\":{\"labels\":{\"app\":\"backend\"}},\"spec\":{\"containers\":[{\"envFrom\":[{\"configMapRef\":{\"name\":\"azd-env-configmap\"}}],\"image\":\"REMOVED",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"backend\",\"ports\":[{\"containerPort\":8080}],\"resources\":{\"requests\":{\"memory\":\"2Gi\"}}}]}}}}\n"
                },
                "creationTimestamp": "2024-03-05T21:25:57Z",
                "generation": 8,
                "labels": {
                    "app": "backend"
                },
                "name": "backend-deployment",
                "namespace": "azure-open-ai",
                "resourceVersion": "178246",
                "uid": "REMOVED"
            },
            "spec": {
                "progressDeadlineSeconds": 600,
                "replicas": 1,
                "revisionHistoryLimit": 10,
                "selector": {
                    "matchLabels": {
                        "app": "backend"
                    }
                },
                "strategy": {
                    "rollingUpdate": {
                        "maxSurge": "25%",
                        "maxUnavailable": "25%"
                    },
                    "type": "RollingUpdate"
                },
                "template": {
                    "metadata": {
                        "creationTimestamp": null,
                        "labels": {
                            "app": "backend"
                        }
                    },
                    "spec": {
                        "containers": [
                            {
                                "envFrom": [
                                    {
                                        "configMapRef": {
                                            "name": "azd-env-configmap"
                                        }
                                    }
                                ],
                                "image": "REMOVED",
                                "imagePullPolicy": "IfNotPresent",
                                "name": "backend",
                                "ports": [
                                    {
                                        "containerPort": 8080,
                                        "protocol": "TCP"
                                    }
                                ],
                                "resources": {
                                    "requests": {
                                        "memory": "2Gi"
                                    }
                                },
                                "terminationMessagePath": "/dev/termination-log",
                                "terminationMessagePolicy": "File"
                            }
                        ],
                        "dnsPolicy": "ClusterFirst",
                        "restartPolicy": "Always",
                        "schedulerName": "default-scheduler",
                        "securityContext": {},
                        "terminationGracePeriodSeconds": 30
                    }
                }
            },
            "status": {
                "availableReplicas": 1,
                "conditions": [
                    {
                        "lastTransitionTime": "2024-03-05T21:26:06Z",
                        "lastUpdateTime": "2024-03-05T21:26:06Z",
                        "message": "Deployment has minimum availability.",
                        "reason": "MinimumReplicasAvailable",
                        "status": "True",
                        "type": "Available"
                    },
                    {
                        "lastTransitionTime": "2024-03-05T21:25:57Z",
                        "lastUpdateTime": "2024-03-06T08:11:19Z",
                        "message": "ReplicaSet \"backend-deployment-57ff65ff7c\" has successfully progressed.",
                        "reason": "NewReplicaSetAvailable",
                        "status": "True",
                        "type": "Progressing"
                    }
                ],
                "observedGeneration": 8,
                "readyReplicas": 1,
                "replicas": 1,
                "updatedReplicas": 1
            }
        }
    ],
    "kind": "List",
    "metadata": {
        "resourceVersion": ""
    }
}```
rajeshkamal5050 commented 6 months ago

@wbreza added to Germanium bucket. Feel free to pull it into respective milestones.