Closed shaojini closed 1 year ago
saschagrunert
commented
1 year ago Hey @shaojini, can you please ensure that cert-manager is healthy before installing the Security Profiles Operator? It looks like that we cannot create the cert resources.
shaojini
commented
1 year ago Thanks for the reply.
Yes. The cer-manager has been installed and the pods of the cer-manager seem ok.
root@k8s-master:~# kubectl --namespace cert-manager wait --for condition=ready pod -l app.kubernetes.io/instance=cert-manager pod/cert-manager-cainjector-84cdd9dd4b-7v694 condition met pod/cert-manager-d6cd78457-slgtm condition met pod/cert-manager-webhook-56ffdd7c44-vfmjq condition met
root@k8s-master:~# kubectl get pods -n cert-manager NAME READY STATUS RESTARTS AGE cert-manager-cainjector-84cdd9dd4b-7v694 1/1 Running 0 5h11m cert-manager-d6cd78457-slgtm 1/1 Running 0 5h11m cert-manager-webhook-56ffdd7c44-vfmjq 1/1 Running 0 5h11m
From the following information of logs, I don't know where is wrong:
E0814 13:12:12.833213 1 controller.go:324] "msg"="Reconciler error" "error"="creating cert manager resources: creating issuer: Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: unexpected error when reading response body. Please retry. Original error: context deadline exceeded" "SecurityProfilesOperatorDaemon"={"name":"spod","namespace":"security-profiles-operator"} "controller"="spod-config" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfilesOperatorDaemon" "name"="spod" "namespace"="security-profiles-operator" "reconcileID"="28aa2761-addb-4b29-8582-0b3a8fabf545"
shaojini
commented
1 year ago Hi, @saschagrunert .
It confirms that the problem is the cert-manager (Error: i/o timeout (connectivity issue)) when I checked it from https://cert-manager.io/docs/troubleshooting/webhook/.
I think the main cause is the private network of the cluster. I have the same problem of the private cluster in the "packetserver service" creation when I installed OLM v0.24.0 (https://github.com/operator-framework/operator-lifecycle-manager/issues/2968#issuecomment-1574558486). Anyway, in OLM v0.25.0, the problem of the "packetserver service" creation has been fixed.
Any idea to fix the problem?
Br. Shaoji
saschagrunert
commented
1 year ago @shaojini unfortunately not, since I'm not able to reproduce the issue locally :thinking:
shaojini
commented
1 year ago Hi, @saschagrunert.
After my fixing of the problem in the cert-manager installation, the security-profiles-operator has been installed successfully.
Thanks. The issue can be closed.
Two VM based worker nodes cluster and Olm has been installed.
root@k8s-master:~# kubectl get pods -n olm NAME READY STATUS RESTARTS AGE catalog-operator-569cd6998d-lwngp 1/1 Running 0 4d olm-operator-6fbbcd8c8b-rj8kc 1/1 Running 0 4d operatorhubio-catalog-qz5sz 1/1 Running 0 6h7m packageserver-775979bc97-p8lkq 1/1 Running 0 4d packageserver-775979bc97-v9ffl 1/1 Running 0 4d
root@k8s-master:~# kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.12.3/cert-manager.yaml namespace/cert-manager created customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created serviceaccount/cert-manager-cainjector created serviceaccount/cert-manager created serviceaccount/cert-manager-webhook created configmap/cert-manager-webhook created clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created clusterrole.rbac.authorization.k8s.io/cert-manager-view created clusterrole.rbac.authorization.k8s.io/cert-manager-edit created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io created clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests created clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests created clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews created role.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created role.rbac.authorization.k8s.io/cert-manager:leaderelection created role.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created rolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created rolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection created rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created service/cert-manager created service/cert-manager-webhook created deployment.apps/cert-manager-cainjector created deployment.apps/cert-manager created deployment.apps/cert-manager-webhook created mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
root@k8s-master:~# kubectl --namespace cert-manager wait --for condition=ready pod -l app.kubernetes.io/instance=cert-manager pod/cert-manager-cainjector-84cdd9dd4b-7v694 condition met pod/cert-manager-d6cd78457-slgtm condition met pod/cert-manager-webhook-56ffdd7c44-vfmjq condition met root@k8s-master:~# kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/security-profiles-operator/main/deploy/operator.yaml customresourcedefinition.apiextensions.k8s.io/profilebindings.security-profiles-operator.x-k8s.io unchanged customresourcedefinition.apiextensions.k8s.io/profilerecordings.security-profiles-operator.x-k8s.io unchanged customresourcedefinition.apiextensions.k8s.io/seccompprofiles.security-profiles-operator.x-k8s.io unchanged customresourcedefinition.apiextensions.k8s.io/securityprofilenodestatuses.security-profiles-operator.x-k8s.io unchanged customresourcedefinition.apiextensions.k8s.io/securityprofilesoperatordaemons.security-profiles-operator.x-k8s.io unchanged customresourcedefinition.apiextensions.k8s.io/rawselinuxprofiles.security-profiles-operator.x-k8s.io unchanged customresourcedefinition.apiextensions.k8s.io/selinuxprofiles.security-profiles-operator.x-k8s.io unchanged customresourcedefinition.apiextensions.k8s.io/apparmorprofiles.security-profiles-operator.x-k8s.io configured namespace/security-profiles-operator unchanged serviceaccount/security-profiles-operator unchanged serviceaccount/spod unchanged serviceaccount/spo-webhook unchanged clusterrole.rbac.authorization.k8s.io/security-profiles-operator unchanged role.rbac.authorization.k8s.io/security-profiles-operator unchanged clusterrole.rbac.authorization.k8s.io/spod unchanged role.rbac.authorization.k8s.io/spod unchanged clusterrole.rbac.authorization.k8s.io/spo-webhook unchanged role.rbac.authorization.k8s.io/spo-webhook unchanged clusterrolebinding.rbac.authorization.k8s.io/security-profiles-operator unchanged rolebinding.rbac.authorization.k8s.io/security-profiles-operator unchanged clusterrolebinding.rbac.authorization.k8s.io/spod unchanged rolebinding.rbac.authorization.k8s.io/spod unchanged clusterrolebinding.rbac.authorization.k8s.io/spo-webhook unchanged rolebinding.rbac.authorization.k8s.io/spo-webhook unchanged mutatingwebhookconfiguration.admissionregistration.k8s.io/spo-mutating-webhook-configuration unchanged clusterrole.rbac.authorization.k8s.io/spo-metrics-client unchanged clusterrolebinding.rbac.authorization.k8s.io/spo-metrics-client unchanged secret/metrics-token unchanged configmap/security-profiles-operator-profile unchanged deployment.apps/security-profiles-operator unchanged root@k8s-master:~# kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/security-profiles-operator/main/examples/olm/operatorhub-io.yaml namespace/security-profiles-operator configured operatorgroup.operators.coreos.com/security-profiles-operator created subscription.operators.coreos.com/security-profiles-operator-sub created
root@k8s-master:~# root@k8s-master:~# kubectl get pods -n security-profiles-operator NAME READY STATUS RESTARTS AGE security-profiles-operator-8588b78997-9cn2b 1/1 Running 0 139m security-profiles-operator-8588b78997-llgdh 1/1 Running 0 139m security-profiles-operator-8588b78997-rbg6h 1/1 Running 0 139m
What happened:
Only three pods of "security-operator-operator" are created.
root@k8s-master:~# kubectl -n security-profiles-operator get spod spod NAME STATE spod PENDING
What you expected to happen:
After successful installation, except for creating three pods of "security-operator-operator", one pod of spod and three pods of security-operator-operator-webhook should be created.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Some logs:
root@k8s-master:~# kubectl logs -f deployment/security-profiles-operator Found 3 pods, using pod/security-profiles-operator-8588b78997-llgdh I0814 05:48:34.178680 1 main.go:260] "msg"="Set logging verbosity to 0" I0814 05:48:34.178950 1 main.go:266] "msg"="Profiling support enabled: false" I0814 05:48:34.180191 1 main.go:286] setup "msg"="starting component: security-profiles-operator" "buildDate"="1980-01-01T00:00:00Z" "buildTags"="netgo,osusergo,seccomp,apparmor" "cgoldFlags"="unknown" "compiler"="gc" "dependencies"="cloud.google.com/go/compute/metadata v0.2.3 ,cuelang.org/go v0.5.0 ,filippo.io/edwards25519 v1.0.0 ,github.com/AliyunContainerService/ack-ram-tool/pkg/credentials/alibabacloudsdkgo/helper v0.2.0 ,github.com/Azure/azure-sdk-for-go v68.0.0+incompatible ,github.com/Azure/go-autorest/autorest v0.11.29 ,github.com/Azure/go-autorest/autorest/adal v0.9.22 ,github.com/Azure/go-autorest/autorest/azure/auth v0.5.12 ,github.com/Azure/go-autorest/autorest/azure/cli v0.4.6 ,github.com/Azure/go-autorest/autorest/date v0.3.0 ,github.com/Azure/go-autorest/logger v0.2.1 ,github.com/Azure/go-autorest/tracing v0.6.0 ,github.com/OneOfOne/xxhash v1.2.8 ,github.com/ProtonMail/go-crypto v0.0.0-20230518184743-7afd39499903 ,github.com/acobaugh/osrelease v0.1.0 ,github.com/agnivade/levenshtein v1.1.1 ,github.com/alibabacloud-go/alibabacloud-gateway-spi v0.0.4 ,github.com/alibabacloud-go/cr-20160607 v1.0.1 ,github.com/alibabacloud-go/cr-20181201 v1.0.10 ,github.com/alibabacloud-go/darabonba-openapi v0.1.18 ,github.com/alibabacloud-go/debug v0.0.0-20190504072949-9472017b5c68 ,github.com/alibabacloud-go/endpoint-util v1.1.1 ,github.com/alibabacloud-go/openapi-util v0.0.11 ,github.com/alibabacloud-go/tea v1.1.18 ,github.com/alibabacloud-go/tea-utils v1.4.4 ,github.com/alibabacloud-go/tea-xml v1.1.2 ,github.com/aliyun/credentials-go v1.2.3 ,github.com/aquasecurity/libbpfgo v0.4.9-libbpf-1.2.0 ,github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 ,github.com/aws/aws-sdk-go-v2 v1.18.1 ,github.com/aws/aws-sdk-go-v2/config v1.18.27 ,github.com/aws/aws-sdk-go-v2/credentials v1.13.26 ,github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.4 ,github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.34 ,github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.28 ,github.com/aws/aws-sdk-go-v2/internal/ini v1.3.35 ,github.com/aws/aws-sdk-go-v2/service/ecr v1.15.0 ,github.com/aws/aws-sdk-go-v2/service/ecrpublic v1.12.0 ,github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.28 ,github.com/aws/aws-sdk-go-v2/service/sso v1.12.12 ,github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.12 ,github.com/aws/aws-sdk-go-v2/service/sts v1.19.2 ,github.com/aws/smithy-go v1.13.5 ,github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20220228164355-396b2034c795 ,github.com/beorn7/perks v1.0.1 ,github.com/blang/semver v3.5.1+incompatible ,github.com/blang/semver/v4 v4.0.0 ,github.com/buildkite/agent/v3 v3.49.0 ,github.com/cert-manager/cert-manager v1.12.3 ,github.com/cespare/xxhash/v2 v2.2.0 ,github.com/chrismellard/docker-credential-acr-env v0.0.0-20220119192733-fe33c00cee21 ,github.com/clbanning/mxj/v2 v2.5.6 ,github.com/cloudflare/circl v1.3.3 ,github.com/cockroachdb/apd/v2 v2.0.2 ,github.com/common-nighthawk/go-figure v0.0.0-20210622060536-734e95fb86be ,github.com/containerd/stargz-snapshotter/estargz v0.14.3 ,github.com/containers/common v0.55.3 ,github.com/coreos/go-oidc/v3 v3.6.0 ,github.com/cpuguy83/go-md2man/v2 v2.0.2 ,github.com/cyberphone/json-canonicalization v0.0.0-20230514072755-504adb8a8af1 ,github.com/davecgh/go-spew v1.1.1 ,github.com/digitorus/pkcs7 v0.0.0-20221212123742-001c36b64ec3 ,github.com/digitorus/timestamp v0.0.0-20221019182153-ef3b63b79b31 ,github.com/dimchansky/utfbom v1.1.1 ,github.com/docker/cli v24.0.0+incompatible ,github.com/docker/distribution v2.8.2+incompatible ,github.com/docker/docker v24.0.2+incompatible ,github.com/docker/docker-credential-helpers v0.7.0 ,github.com/emicklei/go-restful/v3 v3.9.0 ,github.com/emicklei/proto v1.10.0 ,github.com/evanphx/json-patch/v5 v5.6.0 ,github.com/fsnotify/fsnotify v1.6.0 ,github.com/gabriel-vasile/mimetype v1.4.2 ,github.com/ghodss/yaml v1.0.0 ,github.com/go-chi/chi v4.1.2+incompatible ,github.com/go-jose/go-jose/v3 v3.0.0 ,github.com/go-logr/logr v1.2.4 ,github.com/go-logr/stdr v1.2.2 ,github.com/go-openapi/analysis v0.21.4 ,github.com/go-openapi/errors v0.20.3 ,github.com/go-openapi/jsonpointer v0.19.6 ,github.com/go-openapi/jsonreference v0.20.1 ,github.com/go-openapi/loads v0.21.2 ,github.com/go-openapi/runtime v0.26.0 ,github.com/go-openapi/spec v0.20.9 ,github.com/go-openapi/strfmt v0.21.7 ,github.com/go-openapi/swag v0.22.4 ,github.com/go-openapi/validate v0.22.1 ,github.com/go-playground/locales v0.14.1 ,github.com/go-playground/universal-translator v0.18.1 ,github.com/go-playground/validator/v10 v10.14.0 ,github.com/gobwas/glob v0.2.3 ,github.com/gogo/protobuf v1.3.2 ,github.com/golang-jwt/jwt/v4 v4.5.0 ,github.com/golang/glog v1.1.0 ,github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da ,github.com/golang/protobuf v1.5.3 ,github.com/golang/snappy v0.0.4 ,github.com/google/certificate-transparency-go v1.1.6 ,github.com/google/gnostic v0.6.9 ,github.com/google/go-cmp v0.5.9 ,github.com/google/go-containerregistry v0.16.1 ,github.com/google/go-github/v50 v50.2.0 ,github.com/google/go-querystring v1.1.0 ,github.com/google/gofuzz v1.2.0 ,github.com/google/s2a-go v0.1.4 ,github.com/google/uuid v1.3.0 ,github.com/googleapis/enterprise-certificate-proxy v0.2.4 ,github.com/hashicorp/go-cleanhttp v0.5.2 ,github.com/hashicorp/go-retryablehttp v0.7.2 ,github.com/hashicorp/hcl v1.0.0 ,github.com/imdario/mergo v0.3.16 ,github.com/in-toto/in-toto-golang v0.9.0 ,github.com/jedisct1/go-minisign v0.0.0-20211028175153-1c139d1cc84b ,github.com/jellydator/ttlcache/v3 v3.0.1 ,github.com/jmespath/go-jmespath v0.4.0 ,github.com/josharian/intern v1.0.0 ,github.com/json-iterator/go v1.1.12 ,github.com/klauspost/compress v1.16.6 ,github.com/leodido/go-urn v1.2.4 ,github.com/letsencrypt/boulder v0.0.0-20230213213521-fdfea0d469b6 ,github.com/magiconair/properties v1.8.7 ,github.com/mailru/easyjson v0.7.7 ,github.com/matttproud/golang_protobuf_extensions v1.0.4 ,github.com/mitchellh/go-homedir v1.1.0 ,github.com/mitchellh/go-wordwrap v1.0.1 ,github.com/mitchellh/mapstructure v1.5.0 ,github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd ,github.com/modern-go/reflect2 v1.0.2 ,github.com/mozillazg/docker-credential-acr-helper v0.3.0 ,github.com/mpvl/unique v0.0.0-20150818121801-cbe035fff7de ,github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 ,github.com/nozzle/throttler v0.0.0-20180817012639-2ea982251481 ,github.com/nxadm/tail v1.4.8 ,github.com/oklog/ulid v1.3.1 ,github.com/open-policy-agent/opa v0.52.0 ,github.com/opencontainers/go-digest v1.0.0 ,github.com/opencontainers/image-spec v1.1.0-rc4 ,github.com/opencontainers/runtime-spec v1.1.0 ,github.com/openshift/api v0.0.0-20221205111557-f2fbb1d1cd5e ,github.com/opentracing/opentracing-go v1.2.0 ,github.com/pborman/uuid v1.2.1 ,github.com/pelletier/go-toml/v2 v2.0.8 ,github.com/pjbgf/go-apparmor v0.1.2 ,github.com/pkg/errors v0.9.1 ,github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.67.1 ,github.com/prometheus/client_golang v1.16.0 ,github.com/prometheus/client_model v0.4.0 ,github.com/prometheus/common v0.42.0 ,github.com/prometheus/procfs v0.10.1 ,github.com/protocolbuffers/txtpbfmt v0.0.0-20220428173112-74888fd59c2b ,github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 ,github.com/russross/blackfriday/v2 v2.1.0 ,github.com/sassoftware/relic v7.2.1+incompatible ,github.com/seccomp/libseccomp-golang v0.10.0 ,github.com/secure-systems-lab/go-securesystemslib v0.6.0 ,github.com/segmentio/ksuid v1.0.4 ,github.com/shibumi/go-pathspec v1.3.0 ,github.com/sigstore/cosign/v2 v2.1.1 ,github.com/sigstore/fulcio v1.3.1 ,github.com/sigstore/rekor v1.2.2-0.20230601122533-4c81ff246d12 ,github.com/sigstore/sigstore v1.7.1 ,github.com/sigstore/timestamp-authority v1.1.1 ,github.com/sirupsen/logrus v1.9.3 ,github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 ,github.com/spf13/afero v1.9.5 ,github.com/spf13/cast v1.5.1 ,github.com/spf13/cobra v1.7.0 ,github.com/spf13/jwalterweatherman v1.1.0 ,github.com/spf13/pflag v1.0.5 ,github.com/spf13/viper v1.16.0 ,github.com/spiffe/go-spiffe/v2 v2.1.6 ,github.com/subosito/gotenv v1.4.2 ,github.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d ,github.com/tchap/go-patricia/v2 v2.3.1 ,github.com/theupdateframework/go-tuf v0.5.2 ,github.com/titanous/rocacheck v0.0.0-20171023193734-afe73141d399 ,github.com/tjfoc/gmsm v1.3.2 ,github.com/transparency-dev/merkle v0.0.2 ,github.com/urfave/cli/v2 v2.25.7 ,github.com/vbatts/tar-split v0.11.3 ,github.com/xanzy/go-gitlab v0.86.0 ,github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb ,github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 ,github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 ,github.com/yashtewari/glob-intersection v0.1.0 ,github.com/zeebo/errs v1.3.0 ,go.mongodb.org/mongo-driver v1.11.3 ,go.opencensus.io v0.24.0 ,go.opentelemetry.io/otel v1.16.0 ,go.opentelemetry.io/otel/metric v1.16.0 ,go.opentelemetry.io/otel/trace v1.16.0 ,go.step.sm/crypto v0.32.1 ,go.uber.org/atomic v1.10.0 ,go.uber.org/multierr v1.11.0 ,go.uber.org/zap v1.24.0 ,golang.org/x/crypto v0.12.0 ,golang.org/x/exp v0.0.0-20230522175609-2e198f4a06a1 ,golang.org/x/mod v0.12.0 ,golang.org/x/net v0.14.0 ,golang.org/x/oauth2 v0.9.0 ,golang.org/x/sync v0.3.0 ,golang.org/x/sys v0.11.0 ,golang.org/x/term v0.11.0 ,golang.org/x/text v0.12.0 ,golang.org/x/time v0.3.0 ,gomodules.xyz/jsonpatch/v2 v2.3.0 ,google.golang.org/api v0.128.0 ,google.golang.org/appengine v1.6.7 ,google.golang.org/genproto/googleapis/rpc v0.0.0-20230530153820-e85fd2cbaebc ,google.golang.org/grpc v1.57.0 ,google.golang.org/protobuf v1.31.0 ,gopkg.in/go-jose/go-jose.v2 v2.6.1 ,gopkg.in/inf.v0 v0.9.1 ,gopkg.in/ini.v1 v1.67.0 ,gopkg.in/square/go-jose.v2 v2.6.0 ,gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 ,gopkg.in/yaml.v2 v2.4.0 ,gopkg.in/yaml.v3 v3.0.1 ,k8s.io/api v0.27.4 ,k8s.io/apiextensions-apiserver v0.27.2 ,k8s.io/apimachinery v0.27.4 ,k8s.io/client-go v0.27.4 ,k8s.io/component-base v0.27.2 ,k8s.io/klog/v2 v2.100.1 ,k8s.io/kube-openapi v0.0.0-20230515203736-54b630e78af5 ,k8s.io/utils v0.0.0-20230505201702-9f6742963106 ,oras.land/oras-go/v2 v2.2.1 ,sigs.k8s.io/controller-runtime v0.15.1 ,sigs.k8s.io/gateway-api v0.7.0 ,sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd ,sigs.k8s.io/release-utils v0.7.4 ,sigs.k8s.io/structured-merge-diff/v4 v4.2.3 ,sigs.k8s.io/yaml v1.3.0 " "gitCommit"="bcbe39ab9b98a15c5b1a4eeed41a03e9bfac4fed" "gitCommitDate"="2023-08-11T10:41:26Z" "gitTreeState"="clean" "goVersion"="go1.20.4" "ldFlags"="unknown" "libbpf"="v1.2" "libseccomp"="2.5.4" "platform"="linux/amd64" "version"="0.8.1-dev" I0814 05:48:34.180733 1 main.go:365] setup "msg"="watching all namespaces" I0814 05:48:34.181303 1 listener.go:44] controller-runtime/metrics "msg"="Metrics server is starting to listen" "addr"=":8080" I0814 05:48:34.266630 1 setup.go:180] spod-config "msg"="using selinuxd image from envVar" "image"="quay.io/security-profiles-operator/selinuxd" I0814 05:48:34.277825 1 main.go:348] setup "msg"="starting manager" I0814 05:48:34.378631 1 server.go:50] "msg"="starting server" "addr"={"IP":"::","Port":8080,"Zone":""} "kind"="metrics" "path"="/metrics" I0814 05:48:34.378642 1 leaderelection.go:245] attempting to acquire leader lease security-profiles-operator/security-profiles-operator-lock... I0814 05:48:34.406683 1 leaderelection.go:255] successfully acquired lease security-profiles-operator/security-profiles-operator-lock I0814 05:48:34.406912 1 controller.go:177] "msg"="Starting EventSource" "controller"="nodestatus" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfileNodeStatus" "source"="kind source: v1alpha1.SecurityProfileNodeStatus" I0814 05:48:34.406939 1 controller.go:185] "msg"="Starting Controller" "controller"="nodestatus" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfileNodeStatus" I0814 05:48:34.407122 1 controller.go:177] "msg"="Starting EventSource" "controller"="pods" "controllerGroup"="" "controllerKind"="Pod" "source"="kind source: v1.Pod" I0814 05:48:34.407174 1 controller.go:185] "msg"="Starting Controller" "controller"="pods" "controllerGroup"="" "controllerKind"="Pod" I0814 05:48:34.407649 1 controller.go:177] "msg"="Starting EventSource" "controller"="spod-config" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfilesOperatorDaemon" "source"="kind source: v1alpha1.SecurityProfilesOperatorDaemon" I0814 05:48:34.407694 1 controller.go:177] "msg"="Starting EventSource" "controller"="spod-config" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfilesOperatorDaemon" "source"="kind source: v1.DaemonSet" I0814 05:48:34.407706 1 controller.go:185] "msg"="Starting Controller" "controller"="spod-config" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfilesOperatorDaemon" I0814 05:48:34.408049 1 controller.go:177] "msg"="Starting EventSource" "controller"="policymerger" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="ProfileRecording" "source"="kind source: v1alpha1.ProfileRecording" I0814 05:48:34.408068 1 controller.go:185] "msg"="Starting Controller" "controller"="policymerger" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="ProfileRecording" I0814 05:48:34.507574 1 controller.go:219] "msg"="Starting workers" "controller"="pods" "controllerGroup"="" "controllerKind"="Pod" "worker count"=1 I0814 05:48:34.507684 1 controller.go:219] "msg"="Starting workers" "controller"="nodestatus" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfileNodeStatus" "worker count"=1 I0814 05:48:34.511988 1 controller.go:219] "msg"="Starting workers" "controller"="policymerger" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="ProfileRecording" "worker count"=1 I0814 05:48:34.512071 1 controller.go:219] "msg"="Starting workers" "controller"="spod-config" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfilesOperatorDaemon" "worker count"=1 I0814 05:48:34.512293 1 spod_controller.go:244] spod-config "msg"="Adding an initial status to the SPOD Instance" "namespace"="security-profiles-operator" "profile"="spod" I0814 05:48:34.627266 1 ca.go:63] spod-config "msg"="Using cert-manager as certificate provider" I0814 05:48:34.629897 1 spod_controller.go:318] spod-config "msg"="Deploying cert manager resources" E0814 05:48:44.641297 1 controller.go:324] "msg"="Reconciler error" "error"="creating cert manager resources: creating issuer: Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": context deadline exceeded" "SecurityProfilesOperatorDaemon"={"name":"spod","namespace":"security-profiles-operator"} "controller"="spod-config" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfilesOperatorDaemon" "name"="spod" "namespace"="security-profiles-operator" "reconcileID"="63e1a54a-88e3-406f-840b-424f242ca0c2" I0814 05:48:44.647862 1 ca.go:63] spod-config "msg"="Using cert-manager as certificate provider" I0814 05:48:44.648078 1 spod_controller.go:318] spod-config "msg"="Deploying cert manager resources" E0814 05:48:54.651828 1 controller.go:324] "msg"="Reconciler error" "error"="creating cert manager resources: creating webhook cert: Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": context deadline exceeded" "SecurityProfilesOperatorDaemon"={"name":"spod","namespace":"security-profiles-operator"} "controller"="spod-config" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfilesOperatorDaemon" "name"="spod" "namespace"="security-profiles-operator" "reconcileID"="e741f1d7-a89b-4650-81a3-09763d3624a4" I0814 05:48:54.664037 1 ca.go:63] spod-config "msg"="Using cert-manager as certificate provider" I0814 05:48:54.664273 1 spod_controller.go:318] spod-config "msg"="Deploying cert manager resources" E0814 05:49:04.669317 1 controller.go:324] "msg"="Reconciler error" "error"="creating cert manager resources: creating metrics cert: Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": context deadline exceeded" "SecurityProfilesOperatorDaemon"={"name":"spod","namespace":"security-profiles-operator"} "controller"="spod-config" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfilesOperatorDaemon" "name"="spod" "namespace"="security-profiles-operator" "reconcileID"="56fedde6-f496-4003-ab12-627b8557c5cd" I0814 05:49:04.692174 1 ca.go:63] spod-config "msg"="Using cert-manager as certificate provider" I0814 05:49:04.692493 1 spod_controller.go:318] spod-config "msg"="Deploying cert manager resources" E0814 05:49:14.696571 1 controller.go:324] "msg"="Reconciler error" "error"="creating cert manager resources: creating metrics cert: Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": dial tcp 10.101.199.99:443: i/o timeout" "SecurityProfilesOperatorDaemon"={"name":"spod","namespace":"security-profiles-operator"} "controller"="spod-config" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfilesOperatorDaemon" "name"="spod" "namespace"="security-profiles-operator" "reconcileID"="29129826-6d03-43d9-9cc8-69f9cc33b099" I0814 05:49:14.739367 1 ca.go:63] spod-config "msg"="Using cert-manager as certificate provider" I0814 05:49:14.739806 1 spod_controller.go:318] spod-config "msg"="Deploying cert manager resources" E0814 05:49:24.744133 1 controller.go:324] "msg"="Reconciler error" "error"="creating cert manager resources: creating issuer: Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": dial tcp 10.101.199.99:443: i/o timeout" "SecurityProfilesOperatorDaemon"={"name":"spod","namespace":"security-profiles-operator"} "controller"="spod-config" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfilesOperatorDaemon" "name"="spod" "namespace"="security-profiles-operator" "reconcileID"="5e1b835b-4939-4779-ad6b-0f85fdb1823f" I0814 05:49:24.826680 1 ca.go:63] spod-config "msg"="Using cert-manager as certificate provider" I0814 05:49:24.826936 1 spod_controller.go:318] spod-config "msg"="Deploying cert manager resources" E0814 05:49:34.830865 1 controller.go:324] "msg"="Reconciler error" "error"="creating cert manager resources: creating issuer: Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": dial tcp 10.101.199.99:443: i/o timeout" "SecurityProfilesOperatorDaemon"={"name":"spod","namespace":"security-profiles-operator"} "controller"="spod-config" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfilesOperatorDaemon" "name"="spod" "namespace"="security-profiles-operator" "reconcileID"="1485993c-fe84-4b8f-bc33-d2b612106a8c" E0814 05:49:34.832557 1 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"spod.177b29e8c64ee7a7", GenerateName:"", Namespace:"security-profiles-operator", SelfLink:"", UID:"", ResourceVersion:"12936146", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:( int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SecurityProfilesOperatorDaemon", Namespace:"security-profiles-operator", Name:"spod", UID:"2a96dac2-dec0-4f18-8846-102bc9e9b1da", APIVersion:"security-profiles-operator.x-k8s.io/v1alpha1", ResourceVersion:"12935908", FieldPath:""}, Reason:"CannotCreateSPOD", Message:"creating cert manager resources: creating issuer: Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": dial tcp 10.101.199.99:443: i/o timeout", Source:v1.EventSource{Component:"spod-config", Host:""}, FirstTimestamp:time.Date(2023, time.August, 14, 5, 49, 24, 0, time.Local), LastTimestamp:time.Date(2023, time.August, 14, 5, 49, 34, 830279650, time.Local), Count:2, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(v1.EventSeries)(nil), Action:"", Related:(v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "spod.177b29e8c64ee7a7" is forbidden: User "system:serviceaccount:security-profiles-operator:security-profiles-operator" cannot patch resource "events" in API group "" in the namespace "security-profiles-operator"' (will not retry!)
Environment:
cat /etc/os-release):uname -a):VM based cluster, root@k8s-master:~# uname -a Linux k8s-master 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux