Serverless integration tests randomly failing

kwiatekus commented 8 months ago

Description Serverless integration tests failing in non deterministic way

Symptoms 1) Time out while waiting on Serverless module readiness

kubectl apply -n kyma-system \
    -f ../../../../config/samples/default-serverless-cr.yaml
serverless.operator.kyma-project.io/default created
# wait some time to make sure operator starts the reconciliation first
sleep 5
kubectl wait --for condition=Available -n kyma-system deployment serverless-operator --timeout=[60](https://github.com/kyma-project/serverless/actions/runs/7608217076/job/20716954387#step:5:61)s
error: timed out waiting for the condition on deployments/serverless-operator

2) strange 405 response from test git server

{"level":"error","msg":"Error in Commit changes to Git Function, error: while replacing file content in git repository: while cloning repository: unexpected client error: unexpected requesting \"http://127.0.0.1:8001/api/v1/namespaces/test-serverless-gitops-11h10m55s/services/gitserver:80/proxy/function.git/info/refs?service=git-upload-pack\" status code: 405","test":"serverless-integration-gitops","time":"2024-01-17T11:11:48Z"}

It is very likely that integration tests are starting before images are ready.

Add more diagnostic info to the test logs. (i.e if the deployment is not ready, the test should print the deployment status)

kwiatekus commented 7 months ago

Observing recent flakiness for https://github.com/kyma-project/serverless/actions/workflows/operator-verify.yaml

kwiatekus commented 7 months ago

Those tests still suffer from random fails

kyma-project / serverless

Serverless integration tests randomly failing #600