Closed VF-mbrauer closed 1 year ago
@VF-mbrauer I'm unable to reproduce this issue , could you please share more info on config or chain of events before and after issue occur
Also seeing this issue following a trivy-operator
helm upgrade from 0.10.2
to 0.12.1
. Kubernetes : 1.22.17
{"level":"error","ts":"2023-03-23T16:42:54Z","logger":"reconciler.clustercompliancereport","msg":"failed to generate compliance report","compliance report":"/pss-baseline","error":"no matches for kind \"ClusterInfraAssessmentReport\" in version \"aquasecurity.github.io/v1alpha1\"","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/compliance.(*ClusterComplianceReportReconciler).generateComplianceReport.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/compliance/clustercompliancereport.go:68\nk8s.io/client-go/util/retry.OnError.func1\n\t/home/runner/go/pkg/mod/k8s.io/client-go@v0.26.1/util/retry/util.go:51\nk8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1\n\t/home/runner/go/pkg/mod/k8s.io/apimachinery@v0.26.2/pkg/util/wait/wait.go:222\nk8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext\n\t/home/runner/go/pkg/mod/k8s.io/apimachinery@v0.26.2/pkg/util/wait/wait.go:235\nk8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection\n\t/home/runner/go/pkg/mod/k8s.io/apimachinery@v0.26.2/pkg/util/wait/wait.go:228\nk8s.io/apimachinery/pkg/util/wait.ExponentialBackoff\n\t/home/runner/go/pkg/mod/k8s.io/apimachinery@v0.26.2/pkg/util/wait/wait.go:423\nk8s.io/client-go/util/retry.OnError\n\t/home/runner/go/pkg/mod/k8s.io/client-go@v0.26.1/util/retry/util.go:50\nk8s.io/client-go/util/retry.RetryOnConflict\n\t/home/runner/go/pkg/mod/k8s.io/client-go@v0.26.1/util/retry/util.go:104\ngithub.com/aquasecurity/trivy-operator/pkg/compliance.(*ClusterComplianceReportReconciler).generateComplianceReport\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/compliance/clustercompliancereport.go:50\ngithub.com/aquasecurity/trivy-operator/pkg/compliance.(*ClusterComplianceReportReconciler).reconcileComplianceReport.func1\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/compliance/clustercompliancereport.go:44\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/reconcile/reconcile.go:102\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:235"}
helm get values
USER-SUPPLIED VALUES:
operator:
builtInTrivyServer: true
clusterComplianceEnabled: false
infraAssessmentScannerEnabled: false
....
Before upgrade, I had not specified clusterComplianceEnabled
at all (it still is not actually referenced in the base chart values.yaml
) and had not been seeing this error. Adding it after errors surfaced has not had any effect.
I see this issue with ClusterComplianceReports.
# kubectl get clustercompliancereports.aquasecurity.github.io
NAME AGE
nsa 13h
pss-baseline 13h
pss-restricted 13h
The cis
ClusterComplianceReport is missing because I deleted it to check if it gets re-created.
Trivy Operator version: ghcr.io/aquasecurity/trivy-operator:0.19.4
Trivy Operator Helm chart version: 0.21.4
Environment Variables:
# trivy-operator container
- env:
- name: OPERATOR_NAMESPACE
value: trivy
- name: OPERATOR_TARGET_NAMESPACES
- name: OPERATOR_EXCLUDE_NAMESPACES
- name: OPERATOR_TARGET_WORKLOADS
- name: OPERATOR_SERVICE_ACCOUNT
value: trivy-operator
envFrom:
- configMapRef:
name: trivy-operator-config
# trivy-operator-config configmap
CONTROLLER_CACHE_SYNC_TIMEOUT: 5m
OPERATOR_ACCESS_GLOBAL_SECRETS_SERVICE_ACCOUNTS: "true"
OPERATOR_BATCH_DELETE_DELAY: 10s
OPERATOR_BATCH_DELETE_LIMIT: "10"
OPERATOR_BUILT_IN_TRIVY_SERVER: "false"
OPERATOR_CACHE_REPORT_TTL: 120h
OPERATOR_CLUSTER_COMPLIANCE_ENABLED: "false"
OPERATOR_CLUSTER_SBOM_CACHE_ENABLED: "false"
OPERATOR_CONCURRENT_NODE_COLLECTOR_LIMIT: "1"
OPERATOR_CONCURRENT_SCAN_JOBS_LIMIT: "5"
OPERATOR_CONFIG_AUDIT_SCANNER_ENABLED: "false"
OPERATOR_CONFIG_AUDIT_SCANNER_SCAN_ONLY_CURRENT_REVISIONS: "true"
OPERATOR_EXPOSED_SECRET_SCANNER_ENABLED: "true"
OPERATOR_HEALTH_PROBE_BIND_ADDRESS: :9090
OPERATOR_INFRA_ASSESSMENT_SCANNER_ENABLED: "false"
OPERATOR_LEADER_ELECTION_ENABLED: "true"
OPERATOR_LEADER_ELECTION_ID: trivyoperator-lock
OPERATOR_LOG_DEV_MODE: "false"
OPERATOR_MERGE_RBAC_FINDING_WITH_CONFIG_AUDIT: "false"
OPERATOR_METRICS_BIND_ADDRESS: :8080
OPERATOR_METRICS_CLUSTER_COMPLIANCE_INFO_ENABLED: "false"
OPERATOR_METRICS_CONFIG_AUDIT_INFO_ENABLED: "false"
OPERATOR_METRICS_EXPOSED_SECRET_INFO_ENABLED: "false"
OPERATOR_METRICS_FINDINGS_ENABLED: "true"
OPERATOR_METRICS_IMAGE_INFO_ENABLED: "false"
OPERATOR_METRICS_INFRA_ASSESSMENT_INFO_ENABLED: "false"
OPERATOR_METRICS_RBAC_ASSESSMENT_INFO_ENABLED: "false"
OPERATOR_METRICS_VULN_ID_ENABLED: "true"
OPERATOR_PRIVATE_REGISTRY_SCAN_SECRETS_NAMES: '{"trivy":"trivy-operator-registry-credentials"}'
OPERATOR_RBAC_ASSESSMENT_SCANNER_ENABLED: "false"
OPERATOR_SBOM_GENERATION_ENABLED: "false"
OPERATOR_SCAN_JOB_RETRY_AFTER: 5m
OPERATOR_SCAN_JOB_TIMEOUT: 15m
OPERATOR_SCAN_JOB_TTL: 1h
OPERATOR_SCANNER_REPORT_TTL: 72h
OPERATOR_SEND_DELETED_REPORTS: "false"
OPERATOR_VULNERABILITY_SCANNER_ENABLED: "true"
OPERATOR_VULNERABILITY_SCANNER_SCAN_ONLY_CURRENT_REVISIONS: "true"
OPERATOR_WEBHOOK_BROADCAST_TIMEOUT: 30s
OPERATOR_WEBHOOK_BROADCAST_URL: ""
TRIVY_SERVER_HEALTH_CHECK_CACHE_EXPIRATION: 10h
I guess the reason why I have ClusterComplianceReports in my cluster is because the files in deploy/helm/templates/specs get deployed.
Wouldn't it make sense to only deploy them when .Values.operator.clusterComplianceEnabled
is true?
@elchenberg yes it does make sense, would you like to raise a PR ?
Yes, I would like to. Maybe at the weekend.
What steps did you take and what happened:
Even after disabling the clusterComplaince and infraAssessement with
the logs still showing that there is an error with that:
What did you expect to happen:
Anything else you would like to add:
Environment: