grafeas / kritis

Deploy-time Policy Enforcer for Kubernetes applications
https://github.com/grafeas/kritis/blob/master/docs/binary-authorization.md
Apache License 2.0
699 stars 135 forks source link

Investigate and make int test less flaky. #243

Closed tejal29 closed 5 years ago

tejal29 commented 6 years ago

Int tests are randomly failing due to

--- FAIL: TestKritisCron (15.88s)
    --- FAIL: TestKritisCron/nginx-no-digest-breakglass (1.56s)
        run_test.go:560: testCase cmd failed:  Running [kubectl apply -f integration/testdata/nginx/nginx-no-digest-breakglass.yaml]: stdout , stderr: Error from server (InternalError): error when creating "integration/testdata/nginx/nginx-no-digest-breakglass.yaml": Internal error occurred: failed calling admission webhook "kritis-validation-hook.grafeas.io": Post https://kritis-validation-hook-768510490aa498223d56be0eafc2ea17.768510490aa498223d56be0eafc2ea17.svc:443/: service "kritis-validation-hook-768510490aa498223d56be0eafc2ea17" not found
            , err: exit status 1: exit status 1

            NAME      AGE
            my-isp    3s
        run_test.go:510: kubectl delete failed:  Running [kubectl delete -f integration/testdata/nginx/nginx-no-digest-breakglass.yaml]: stdout , stderr: Error from server (NotFound): error when stopping "integration/testdata/nginx/nginx-no-digest-breakglass.yaml": pods "nginx-no-digest-breakglass" not found
            , err: exit status 1: exit status 1
    --- FAIL: TestKritisCron/image-with-acceptable-vulnz (1.63s)
        run_test.go:560: testCase cmd failed:  Running [kubectl apply -f integration/testdata/vulnz/acceptable-vulnz.yaml]: stdout , stderr: Error from server (InternalError): error when creating "integration/testdata/vulnz/acceptable-vulnz.yaml": Internal error occurred: failed calling admission webhook "kritis-validation-hook.grafeas.io": Post https://kritis-validation-hook-768510490aa498223d56be0eafc2ea17.768510490aa498223d56be0eafc2ea17.svc:443/: service "kritis-validation-hook-768510490aa498223d56be0eafc2ea17" not found
            , err: exit status 1: exit status 1

            NAME      AGE
            my-isp    5s
        run_test.go:532: kubectl delete failed:  Running [kubectl delete -f integration/testdata/vulnz/acceptable-vulnz.yaml]: stdout , stderr: Error from server (NotFound): error when stopping "integration/testdata/vulnz/acceptable-vulnz.yaml": pods "image-with-acceptable-vulnz" not found
            , err: exit status 1: exit status 1
    run_test.go:162: testing error: Running [helm delete --purge misty-swan]: stdout , stderr: Error: Internal error occurred: failed calling admission webhook "kritis-validation-hook.grafeas.io": Post https://kritis-validation-hook-768510490aa498223d56be0eafc2ea17.768510490aa498223d56be0eafc2ea17.svc:443/: service "kritis-validation-hook-768510490aa498223d56be0eafc2ea17" not found
        , err: exit status 1: exit status 1
FAIL
aaron-prindle commented 6 years ago

I believe this happens when a ValidatingWebhookConfiguration is not cleaned up. Even if the webhook is scoped to a namespace, if the underlying service is deleted that webhook will error and cause the next deploy to fail regardless of namespace