search

aquasecurity / trivy-operator

Kubernetes-native security toolkit
https://aquasecurity.github.io/trivy-operator/latest
Apache License 2.0
1.28k stars 213 forks source link

No more VulnerabilityReport + "illegal base64 data at input byte 4" or "cannot unmarshal number (...)" #942

Closed ledroide closed 1 year ago

ledroide commented 1 year ago

Symptoms

After months running fine, with many VulnerabilityReports in all namespaces, we discovered that there was no more VulnerabilityReport in any namespace, for any of our k8s clusters.

NAME COMPLETIONS DURATION AGE job.batch/scan-vulnerabilityreport-67679699c8 1/1 16s 39h $ $ kubectl logs -l app.kubernetes.io/managed-by=trivy-operator,vulnerabilityReport.scanner=Trivy -n trivy-system Defaulted container "node-cache" out of: node-cache, e0073c41-3c2f-49ef-a8b3-f84bb4cd143e (init) y48MKDTFL5nsozxSuwMguemrO+wcbBebJtsrYg2PgsN5CeH++zoinFT/H4EiOsiIx8uT3/xHtyN7 +SUWq1Lfg3nLBiiSRvmtI4jrUtUWuLC6z9/hh+9vpH0avTipY634PJQn0EBWyonPfZ8yc0ierBQj mRSEKqqkz5nYunMUqVD8RwsmionV59Ob/mbA1DvRef0pWyDvFsXxYRAu1eARqas+Lh2bnNzHXJI4 Yhg3XAuudMYTuhkWi0sui7B9tmWCmn7FtMLaLFlyEOwU7OJp05Yj5DyxosbRAhNgYSIdDs7jPEiE IdiaE5ooV6Z8+T0Qzllu5MuaZ2hhDNLwPDUXXvMbGbhApMgfCkwbNy42t9Vw23pyGqIWsNZ0YIOS QVQ8d49h9o65aDo4qJHUY7WpUEOouVn0k9SAW4GAX/DHArnccBG4OUxLQihayABAL6tg68gd2FUM mIKGEVDhptG4SDtH+NHgk4DpZPoOSCL3kT6al1iMfZP76h/Eb/3vBHpSJVcP9JTxSRR4I2VwtHod vS6HRSlSEbpO2+520/1PY8vK8XbMpH/B52fKPrGIE+VfN80L0Op7D5wVHbH8MSexaGbTgeofghvM z/Ygf7qAkRQB/8w+4/sf4h/xkH/r/0qhs2plkCMv/aQiTOQRv+wPWcA8GH/+LuSKcKEh4Z6WzA==

* because we observe the same issue for all our clusters, we know that scan jobs run for any kind of image : private registry, docker hub, ghcr.io, etc. ; it's obviously not a question of credentials, or image similarity, or remote registry, or registry quota.
* logs from the trivy-operator controller report lots of `illegal base64 data at input byte 4` errors :
```bash
$ kubectl logs deploy/trivy-operator -n trivy-system
[...]
{"level":"error","ts":"2023-02-10T08:12:40Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"scan-vulnerabilityreport-67679699c8","namespace":"trivy-system"},"namespace":"trivy-system","name":"scan-vulnerabilityreport-67679699c8","reconcileID":"ee597f18-0f67-4c0e-bcf0-c322762002d1","error":"illegal base64 data at input byte 4","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:235"}

What we have tried

Environment

chen-keinan commented 1 year ago

@ledroide thank you for sharing this issue (this line could cause the issue Defaulted container "node-cache" out of: node-cache, e0073c41-3c2f-49ef-a8b3-f84bb4cd143e (init)) ,

Related #757

ledroide commented 1 year ago

@chen-keinan Thanks for your answer.

parameters

Trying :

Only the first one is set with scanJob.compressLogs: "false" and trivy.severity: HIGH,CRITICAL.

$ kubectl logs -l app.kubernetes.io/managed-by=trivy-operator,vulnerabilityReport.scanner=Trivy -n trivy-system --context poc
Defaulted container "nginx-proxy" out of: nginx-proxy, 67eb91b1-1c3c-405b-8ae6-086b8f8bed1a (init)
            "https://ubuntu.com/security/notices/USN-5845-2",
            "https://www.openssl.org/news/secadv/20230207.txt"
          ],
          "PublishedDate": "2023-02-08T20:15:00Z",
          "LastModifiedDate": "2023-02-09T14:26:00Z"
        }
      ]
    }
  ]
}

Others are set with default values, including scanJob.compressLogs: "false" and trivy.severity: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL.

$ kubectl logs -l app.kubernetes.io/managed-by=trivy-operator,vulnerabilityReport.scanner=Trivy -n trivy-system --context dev
Defaulted container "trident-main" out of: trident-main, driver-registrar, 8444699c-583c-4968-ba8b-6aa15ed2b162 (init)
HuQG64NRU2JjCF5IwnEuM8SIxIwpGKhqldlL0hv0nSbS63MWVjRMLKC6jt4CuQcyyZCDYe28iubj
eOkHVtHhbhp2F9HNqYoHJRK9pQcHqnDdLgplhckFN8WqzwuoLpkCz55E6t4BDjuKobwZEugS5HrM
tj6+HtHaES45jEuxbLWPYpYatj1yo2MblcZiUS3rMEILtq3gizi4ghoaYfQiHHZuEpgHqKvV5xLw
B2Q7nwPa28sqOgLEbLG9J35hea/ZbuzE3DsW7z5DbmuLCKSkG1GVRSuVKYrzSvDnLvDiEhIp7K85
GyUiOishe+k8i8ob4kahP49QdMeoEusvFtavkdgalrsAPRM79byIdbrzTyVQhaAv9Dl2oxLWZs5d
9XIGxQeeWulmGDNU3feJa9oRH5HsKdk5kH+5Ua65hMSmnMmd0yBMMGkzBAh2oCYOyCA469jsxgaw
50DwFmyWXFAaka2NS8O6GRmmaKggq4IB0vsXiwH+XuNgGsMTpEtoZkQqdkg4LqBnIGCG72Gr43oQ
YBDrCgFEoy0hWkRvHk0xJ2R01kLYxbq36IcKKoVZKBHwggM6XgymFuIYzpKgyZDAxxCmkhBJjEdU
wcGSpdURQgmDU1VLSVqSPWUC5W2A4usSiFuD5ivMJVcQiJfQOyjK0M9hV42v5YhWINFviBgsEsyT
Wh4gxTuTHaMH6/fBdoUUyMFP8XckU4UJBqpdmFA=
$ kubectl logs -l app.kubernetes.io/managed-by=trivy-operator,vulnerabilityReport.scanner=Trivy -n trivy-system --context qal
Defaulted container "kube-apiserver" out of: kube-apiserver, fccb553c-adb5-4467-a09e-279cc212bf8f (init)
Defaulted container "cadvisor" out of: cadvisor, bd10b70d-d4a6-420d-a93b-e00d115dbc07 (init)
Defaulted container "opa-scorecard-exporter-ldscm" out of: opa-scorecard-exporter-ldscm, 85ab8993-5b75-4bb9-938e-a5a55f33f796 (init)
Defaulted container "etcd" out of: etcd, bdb78fcb-e637-49e2-a668-2bebf11f9d8c (init)
3tSxhtA5HUFOE8UQO+QhN2GmOjpt1PVPbG3YcRxkQtoiHbkhX4PemYdBWxsdx3NUdLrDyCVgmWrh
SKUy0ToCUN625hpGFiWFZC1g86xmUsnZAkJ7dwEEYPGPMEJB66V0ivXNrptidM2lJRRRRIwIRNZS
MB5Bg9EBtne0HVmFNTmQIHO9pNCQXtDmRJbb0snNPk7FTBOxWxyWHumfOZgbuvucqlsREQBj8iRc
OmhBMGwjfKoSYtQ9oFIHqIabdxel7UJDxxyIv0VyTab9BaLcWTK6kdLfKTJ9fS7EShbhImRy+YZM
0ohZkQcT3lwIfmMLT1HD6noZLgJ7nizIarORU/hP07ygQmCn+gk2Fd62syhN3GNqWkDTSyYZuswS
ZQbdjOT0q7mlcGe10E00wb9m0wtbWVua2XUtMrlhZDFupleZMckdgb9jgxqbBJVAmCjtW2ltnhD3
kXiHcQ6uEmkwhQtLmW9jQrGMQEkGcVTDrYQXaUYU6WOJEOQ3YcJAqPjBlgYp0RDCYpind6H8wsEO
9f9j2fJyHc2sxcH5g/tCCGuvdcmxxJqctl4tSptDZtUILsZqDS7S2crBS2MDnL7hxMlMVUxuZdtL
0PBoWeR7Ej5P3HoWCXQVMFSDzpkb6STKGDz7HM4JXGVLbursbYYuyBFI60m1DcHibyAamujxkDYe
Uj+KkUP681nocmIYwJ+SATgJU/+LuSKcKEhtEBi8gA==
/2LOxtiTWep7W30fTPLsdj18awv1zadSIZQ0RSHRLbV933M0aaaONyp1LFKtFUp22kWQoo7C1nSM
7i0lzITcvyvQt+8pbsoy1JpKwGxvHQ3SRceZ6D0jFsRxnYfZndfDZCbi1jJ6keTcj8jDFHFIp+zR
dx69MYyqI+hbbJcVlIJbQjxBBN/m2pm15u5HFoylFytM99cqy4TGG+tb69+Tvssjffl+4NiMGtLN
ViLQ1UMnc8nBd1oh9TN+39FaPkbfdMSbUdT+Jcsi9IebpXsuPO++ok8C5pidkhfTPaJ7XxZsUN7M
l2+Teqdg4Chkq6RZsYL0cJYsXvDF9UqQLG3e3emjLHBykzdq5QGoFkMu0itLrtF+N9vMLzKGKmCm
VxowieLPdTPFFy7O5wUsykq+0kecpRey53rOOXno9WyLtEdd1acWghgzmTACxtmKmzokCSCyOS9G
lxUTYsirxfv3OUSGl45MmOibrN9bsltzwiQxDdx7RlBOKbDv4h/vD3P+ZG2DrPi6g8p0k6MrilRO
bfPuH7ylIxwDi0bA4dSRV7xMOdIxbOXZPhdfwkXDtXO2H7pq6tmbVFEu4EaShphcTetay7YhZmng
sv8nF7BmXvmf5uD1tDA7bCzfEZimg3D03qh57PQKk++ngK5t6xYlbdWZKshZkWcFrCBBIOqgaQoO
dlwX3jP8Rm3DIKP2InoRIC+z863/+KOrMH8/2aNPusxxkP+EjEzz/8F3JFOFCQf7673A
zN9acXAPkig81Jiick5QIowm0BHuG9AdUjXbAu527DKSkhOXOpfBcRtdI4wCMC4gzAJ2GQjlFRBX
TO7pBPWOAj2nbIk7/u0+aGgkhzgiNuoGJpIS9UoYsfKxNDfDS6X1kjtRfnj9SmQjbdEYHVcizfdU
sAaMoNrwzZoJ2wZWFkFYFkCEVEuB2qlZpUZCjGJKRWsFtrYsuuIDZEWRZuclSZW9nkIETVgUAlQS
1sILiCgq2LhMQRvwigxIhCAoo8b3AmJYmJzwETrAw12iQlzIrtgjQKgqvhY4IAhqQ8+MDVymDqFy
ux35QKXWCVFI3hsJf9ub5yIjDerksMLvJUBCm6Rm0CZrBmbiIlIgTaZMsjLgrZWuASoiE5bxVXgF
lHSxU/mWScPREEShUdYzEzFVc79chQNeteA3dQ9wHiJiCHkJu2NogCISon1iO7nIUcpan7QE/iA3
8cUlnxvASOJJJQRABuhB5y9Bogz5rhuS+GGIPMHhpxK4cDQg6aG8AzVaieo6x0g+EazpPtwwJQJI
SASPMaAjaUlHZc1SIoFBGGa5zjoAPb7NRXMSuByn+d11wj22DnBt8kNhbiCB6AqeKuA1gD4wUd4O
sr+ySE4EmdmgnnRrwcrfEwfw5zd5ymgR0u4RZBCqJCiGKYdxuF6jrbMRyHeZCow2t2cQbTKfmQ/k
rAAH9fpR/kSwrBWoPeIFgJg/8XckU4UJCKa2qWA=
sFVYup7sFoUpKj0ESWSoRFIts1FHCkoplIiXWkziMRSxaB7ijINzX7agGAqL8+AaGDhFNzcaccdl
EO4LgBsqRxXHhiq2C7lq6CcgVPzf73aNMrEic0LlLRzugqy0uWjzcDcBMXqWJAqICIZSIIF37ACV
KUAqIDCoDWL87Y/oofViWFV+tx5A8/wZhzZVEj1MkosfeESmbqSAQpchiC+oILaJDZf210wVCmLf
I8FoQ0DrDL4UHlNdGxCBDZIRNgodCI8v7ZLOQpxqHnEDUgX9f3E6njhJrMSKlQr8W1y9SQjMp/RZ
YNoWEAaPQFDtwuvTOk3UoegggaArvTm2fpL2Wehj8xBEVSHZl81hkhXwKJ4f2Oh+R5pz+qW2QsqS
ExGl7qTXK1MWYFi15LFQyoMFlpiFpni12SiCkgZJmf76JFRKwLJHZRKTJUUOBhVQUmRMQQUeC5RI
acahSKBkmkNQ6lSEKTCjASIK3vnXdCwlwWF4UYASqj1PGJ8jAskAn3y7nXFE3HK4JCgVbWRUkU49
gOuhUW6CuzrqkAlwkNR/lRzCJ6B0RVoCfDM5rA+I05H1oK0f3RXSSSjnI0nBU40h/WUE2NND/I+a
in/RP4e6T+KkVXs+u0HE9R8kEH7ryea0M9BuPaBsYn/AQP/ZQAYQIQf2MfI/qPzD99i/t/tSBLZt
gqhJJR/FCBIkISVqbH/4u5IpwoSCSQ+nkA==
$ kubectl logs -l app.kubernetes.io/managed-by=trivy-operator,vulnerabilityReport.scanner=Trivy -n trivy-system --context prd
Defaulted container "node-cache" out of: node-cache, e0073c41-3c2f-49ef-a8b3-f84bb4cd143e (init)
y48MKDTFL5nsozxSuwMguemrO+wcbBebJtsrYg2PgsN5CeH++zoinFT/H4EiOsiIx8uT3/xHtyN7
+SUWq1Lfg3nLBiiSRvmtI4jrUtUWuLC6z9/hh+9vpH0avTipY634PJQn0EBWyonPfZ8yc0ierBQj
mRSEKqqkz5nYunMUqVD8RwsmionV59Ob/mbA1DvRef0pWyDvFsXxYRAu1eARqas+Lh2bnNzHXJI4
Yhg3XAuudMYTuhkWi0sui7B9tmWCmn7FtMLaLFlyEOwU7OJp05Yj5DyxosbRAhNgYSIdDs7jPEiE
IdiaE5ooV6Z8+T0Qzllu5MuaZ2hhDNLwPDUXXvMbGbhApMgfCkwbNy42t9Vw23pyGqIWsNZ0YIOS
QVQ8d49h9o65aDo4qJHUY7WpUEOouVn0k9SAW4GAX/DHArnccBG4OUxLQihayABAL6tg68gd2FUM
mIKGEVDhptG4SDtH+NHgk4DpZPoOSCL3kT6al1iMfZP76h/Eb/3vBHpSJVcP9JTxSRR4I2VwtHod
vS6HRSlSEbpO2+520/1PY8vK8XbMpH/B52fKPrGIE+VfN80L0Op7D5wVHbH8MSexaGbTgeofghvM
z/Ygf7qAkRQB/8w+4/sf4h/xkH/r/0qhs2plkCMv/aQiTOQRv+wPWcA8GH/+LuSKcKEh4Z6WzA==
$ kubectl logs -l app.kubernetes.io/managed-by=trivy-operator,vulnerabilityReport.scanner=Trivy -n trivy-system --context usd1wkld01
Defaulted container "logs-collector" out of: logs-collector, 39636705-4bdf-4e78-9be1-4ff2544f4943 (init)
Defaulted container "cluster-health-extension" out of: cluster-health-extension, 22248875-3141-49f5-83b5-a302e71be70f (init)
Defaulted container "ingress-nginx-controller" out of: ingress-nginx-controller, 9a770ca0-c26f-4529-8466-60bccb43a08d (init)
Defaulted container "pollux" out of: pollux, index-generator, f24abbf6-e7cf-49a3-b75f-61ede53aefd6 (init)
Defaulted container "agentupdater-workload" out of: agentupdater-workload, 0d13cdc7-16ca-4516-a0b3-eb9cb49bccd2 (init)
ad1ba4ed36e249b574bc3a6beaf926921ff87e07964e812e5732e25faf535fafoPdKTMy1lx4Km5TJTimHVacEOcmcVJUw8NE0bmFDBLqbqlylS+ai1qXMKLKSpMJFlLLWUYUS6lKK
LKGzF0jCXUopkjjPVDZEMlSaNmSyWdmSzVuVAtOySXUEizxSRzLOKpSSojfJTZsLF7bKdmC0ZKXv
yaLROcsWVwSfVVUqkLK6JITVIyubmqrNzepPHYsOqLILKJIdaTZmna7uSODxf5x+cidQ/IMPsn/q
I8fB6OiT7ojqeiRRsp6LFM41TvWvllTJuKN/X6wyKiNVCd/4fZeHg3NHm9HcexvDT8Y/fUhURRqe
5KntjKHVN7zeDNvW9zglHIkWWX6JFmZGGZUSO88CmKllMIklIon7GRuXjudBtHeqI+LkauD1k+rl
G90dHFqsUtiuKWw/jH4Rk4PdPFHuRujxnc/epkzsU9HbEk5xFIll+jIyTohMXNGS0YSbsUYKMLKL
XXXQWUyRaLxWEvhnU3yb4zdpKkngoyalGi36ApUPsmEXDyevnGEu1PUlGSUZqRGFkun6mI3iw0vS
4HEwQcUH0Gsz2CwBpDKWkFIlw1dt7CeFSlQ5opmVI9XhGIix7Es6F1sl1vFUyLqUpmwpZmxkuvfm
uaMlmajNS5TLDNdgoq7JPNRhilKNFTLExEUve7RUkjSkS6kKmRZBaIzWe0yki0ngsaubVGSfiraQ
YLigEzG5N4pqP8wHaIwAJ9Ozz9Z77D6I3FsUog/oqNQcn/xdyRThQkP1+OJA

controller pod

From cluster where scanJob.compressLogs: "false" and trivy.severity: HIGH,CRITICAL :

trivy-operator-7ddfd74ccd-fr8jr trivy-operator {"level":"error","ts":"2023-02-10T13:41:08Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"scan-vulnerabilityreport-599465f897","namespace":"trivy-system"},"namespace":"trivy-system","name":"scan-vulnerabilityreport-599465f897","reconcileID":"0093af77-caa2-4f82-9185-e9fb73f56933","error":"json: cannot unmarshal number into Go value of type trivy.ScanReport","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:235"}
trivy-operator-7ddfd74ccd-fr8jr trivy-operator {"level":"error","ts":"2023-02-10T13:57:48Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"scan-vulnerabilityreport-599465f897","namespace":"trivy-system"},"namespace":"trivy-system","name":"scan-vulnerabilityreport-599465f897","reconcileID":"3b16ba70-b04a-4370-9adb-ca649c984740","error":"json: cannot unmarshal number into Go value of type trivy.ScanReport","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:235"}
trivy-operator-7ddfd74ccd-fr8jr trivy-operator {"level":"error","ts":"2023-02-10T14:14:28Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"scan-vulnerabilityreport-599465f897","namespace":"trivy-system"},"namespace":"trivy-system","name":"scan-vulnerabilityreport-599465f897","reconcileID":"392b6496-8757-47ce-92d9-37f8a7516d9d","error":"json: cannot unmarshal number into Go value of type trivy.ScanReport","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:235"}

From another one where scanJob.compressLogs: "true" and trivy.severity: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL

trivy-operator-66f6b9bc-zfh5p trivy-operator {"level":"error","ts":"2023-02-10T13:36:25Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"scan-vulnerabilityreport-697c7fb8d7","namespace":"trivy-system"},"namespace":"trivy-system","name":"scan-vulnerabilityreport-697c7fb8d7","reconcileID":"2b1073af-72d8-46cf-9fe2-866481dc1708","error":"illegal base64 data at input byte 4; illegal base64 data at input byte 4","errorCauses":[{"error":"illegal base64 data at input byte 4"},{"error":"illegal base64 data at input byte 4"}],"stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:235"}
trivy-operator-66f6b9bc-zfh5p trivy-operator {"level":"error","ts":"2023-02-10T13:53:05Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"scan-vulnerabilityreport-697c7fb8d7","namespace":"trivy-system"},"namespace":"trivy-system","name":"scan-vulnerabilityreport-697c7fb8d7","reconcileID":"0ac46a8d-5b0b-47d6-8319-b341be42d03a","error":"illegal base64 data at input byte 4; illegal base64 data at input byte 4","errorCauses":[{"error":"illegal base64 data at input byte 4"},{"error":"illegal base64 data at input byte 4"}],"stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:235"}
trivy-operator-66f6b9bc-zfh5p trivy-operator {"level":"error","ts":"2023-02-10T14:09:45Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"scan-vulnerabilityreport-697c7fb8d7","namespace":"trivy-system"},"namespace":"trivy-system","name":"scan-vulnerabilityreport-697c7fb8d7","reconcileID":"c68a6f4a-4a4a-44df-a401-d91801208869","error":"illegal base64 data at input byte 4; illegal base64 data at input byte 4","errorCauses":[{"error":"illegal base64 data at input byte 4"},{"error":"illegal base64 data at input byte 4"}],"stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:235"}
trivy-operator-66f6b9bc-zfh5p trivy-operator {"level":"error","ts":"2023-02-10T14:26:25Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"scan-vulnerabilityreport-697c7fb8d7","namespace":"trivy-system"},"namespace":"trivy-system","name":"scan-vulnerabilityreport-697c7fb8d7","reconcileID":"fdacadd7-0579-47ec-8c40-afc99c3892cd","error":"illegal base64 data at input byte 4; illegal base64 data at input byte 4","errorCauses":[{"error":"illegal base64 data at input byte 4"},{"error":"illegal base64 data at input byte 4"}],"stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.2/pkg/internal/controller/controller.go:235"}
chen-keinan commented 1 year ago

@ledroide thanks you for response with many examples , I need to see the output as operator query it from go k8s cli, I need one more example (not by kubectl logs --selector .... ) ,

if you could run the following when compressLogs=true : kubectl logs <scan job pod name> -n trivy-system

ledroide commented 1 year ago

@chen-keinan : Here is what you have asked for : 

$ kubectl get pod -o wide -n trivy-system
NAME                                        READY   STATUS      RESTARTS   AGE     IP               NODE             NOMINATED NODE   READINESS GATES
scan-vulnerabilityreport-599465f897-2lg9m   0/1     Completed   0          3h19m   10.233.115.117   iqdackorclust1   <none>           <none>
trivy-operator-7ddfd74ccd-fr8jr             1/1     Running     0          3h38m   10.233.115.73    iqdackorclust1   <none>           <none>

$ kubectl logs scan-vulnerabilityreport-599465f897-2lg9m -n trivy-system
Defaulted container "nginx-proxy" out of: nginx-proxy, 67eb91b1-1c3c-405b-8ae6-086b8f8bed1a (init)
2023-02-10T13:19:11.623Z        WARN    '--security-checks' is deprecated. Use '--scanners' instead.
2023-02-10T13:19:11.623Z        WARN    '--skip-update' is deprecated. Use '--skip-db-update' instead.
{
  "SchemaVersion": 2,
  "ArtifactName": "docker.io/library/nginx:1.23.2-alpine",
  "ArtifactType": "container_image",
  "Metadata": {
    "OS": {
      "Family": "alpine",
      "Name": "3.16.3"
    },
    "ImageID": "sha256:19dd4d73108a1feefc29d299f3727467ac02486c83474fc3979e4a7637291fe6",
    "DiffIDs": [
      "sha256:e5e13b0c77cbb769548077189c3da2f0a764ceca06af49d8d558e759f5c232bd",
      "sha256:07099189e7ec257e501d9625507b55e0ea32c330e38c90d8533b3fa2a7a97069",
      "sha256:fcf860bf48b4e20f24f44ba02115dc9f23eef6d41d69e9a050889bf25104e12a",
      "sha256:6636f46e559dffe6373b200c359773488f201ed2153507fb8d8fe3f04fdf477e",
      "sha256:9365b1fffb04e52b8f6abf1c8737ba4da02e134c1d8550e0ace4cb562d12f070",
      "sha256:bd502c2dee4c0bc2cf334c7d289e5a14ededd6c9c361137d128d3c12e4babf5d"
    ],
    "RepoTags": [
      "nginx:1.23.2-alpine"
    ],
    "RepoDigests": [
      "nginx@sha256:455c39afebd4d98ef26dd70284aa86e6810b0485af5f4f222b19b89758cabf1e"
    ],
    "ImageConfig": {
      "architecture": "amd64",
      "container": "3d6e40973806c9e8769cd1315bbbf54ed590c4f5febcb1ac80ced2c8da530e0a",
      "created": "2022-11-12T06:27:47.33774049Z",
      "docker_version": "20.10.12",
      "history": [
        {
          "created": "2022-11-12T04:19:23.05154209Z",
          "created_by": "/bin/sh -c #(nop) ADD file:ceeb6e8632fafc657116cbf3afbd522185a16963230b57881073dad22eb0e1a3 in / "
        },
        {
          "created": "2022-11-12T04:19:23.199716539Z",
          "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/sh\"]",
          "empty_layer": true
        },
        {
          "created": "2022-11-12T06:27:39.250897562Z",
          "created_by": "/bin/sh -c #(nop)  LABEL maintainer=NGINX Docker Maintainers \u003cdocker-maint@nginx.com\u003e",
          "empty_layer": true
        },
        {
          "created": "2022-11-12T06:27:39.350990952Z",
          "created_by": "/bin/sh -c #(nop)  ENV NGINX_VERSION=1.23.2",
          "empty_layer": true
        },
        {
          "created": "2022-11-12T06:27:39.461575962Z",
          "created_by": "/bin/sh -c #(nop)  ENV NJS_VERSION=0.7.7",
          "empty_layer": true
        },
        {
          "created": "2022-11-12T06:27:39.567489916Z",
          "created_by": "/bin/sh -c #(nop)  ENV PKG_RELEASE=1",
          "empty_layer": true
        },
        {
          "created": "2022-11-12T06:27:46.416512795Z",
          "created_by": "/bin/sh -c set -x     \u0026\u0026 addgroup -g 101 -S nginx     \u0026\u0026 adduser -S -D -H -u 101 -h /var/cache/nginx -s /sbin/nologin -G nginx -g nginx nginx     \u0026\u0026 apkArch=\"$(cat /etc/apk/arch)\"     \u0026\u0026 nginxPackages=\"         nginx=${NGINX_VERSION}-r${PKG_RELEASE}         nginx-module-xslt=${NGINX_VERSION}-r${PKG_RELEASE}         nginx-module-geoip=${NGINX_VERSION}-r${PKG_RELEASE}         nginx-module-image-filter=${NGINX_VERSION}-r${PKG_RELEASE}         nginx-module-njs=${NGINX_VERSION}.${NJS_VERSION}-r${PKG_RELEASE}     \"     \u0026\u0026 apk add --no-cache --virtual .checksum-deps         openssl     \u0026\u0026 case \"$apkArch\" in         x86_64|aarch64)             set -x             \u0026\u0026 KEY_SHA512=\"e7fa8303923d9b95db37a77ad46c68fd4755ff935d0a534d26eba83de193c76166c68bfe7f65471bf8881004ef4aa6df3e34689c305662750c0172fca5d8552a *stdin\"             \u0026\u0026 wget -O /tmp/nginx_signing.rsa.pub https://nginx.org/keys/nginx_signing.rsa.pub             \u0026\u0026 if [ \"$(openssl rsa -pubin -in /tmp/nginx_signing.rsa.pub -text -noout | openssl sha512 -r)\" = \"$KEY_SHA512\" ]; then                 echo \"key verification succeeded!\";                 mv /tmp/nginx_signing.rsa.pub /etc/apk/keys/;             else                 echo \"key verification failed!\";                 exit 1;             fi             \u0026\u0026 apk add -X \"https://nginx.org/packages/mainline/alpine/v$(egrep -o '^[0-9]+\\.[0-9]+' /etc/alpine-release)/main\" --no-cache $nginxPackages             ;;         *)             set -x             \u0026\u0026 tempDir=\"$(mktemp -d)\"             \u0026\u0026 chown nobody:nobody $tempDir             \u0026\u0026 apk add --no-cache --virtual .build-deps                 gcc                 libc-dev                 make                 openssl-dev                 pcre2-dev                 zlib-dev                 linux-headers                 libxslt-dev                 gd-dev                 geoip-dev                 perl-dev                 libedit-dev                 bash                 alpine-sdk                 findutils             \u0026\u0026 su nobody -s /bin/sh -c \"                 export HOME=${tempDir}                 \u0026\u0026 cd ${tempDir}                 \u0026\u0026 curl -f -O https://hg.nginx.org/pkg-oss/archive/${NGINX_VERSION}-${PKG_RELEASE}.tar.gz                 \u0026\u0026 PKGOSSCHECKSUM=\\\"98d244d5dea3f0c49692843b1857e21dc7353e749f9ff8a526036a3beeea299e156183b6a98070ffc68a23d191e1f24c577d7ea874f8cc27ce01f4dc832658b6 *${NGINX_VERSION}-${PKG_RELEASE}.tar.gz\\\"                 \u0026\u0026 if [ \\\"\\$(openssl sha512 -r ${NGINX_VERSION}-${PKG_RELEASE}.tar.gz)\\\" = \\\"\\$PKGOSSCHECKSUM\\\" ]; then                     echo \\\"pkg-oss tarball checksum verification succeeded!\\\";                 else                     echo \\\"pkg-oss tarball checksum verification failed!\\\";                     exit 1;                 fi                 \u0026\u0026 tar xzvf ${NGINX_VERSION}-${PKG_RELEASE}.tar.gz                 \u0026\u0026 cd pkg-oss-${NGINX_VERSION}-${PKG_RELEASE}                 \u0026\u0026 cd alpine                 \u0026\u0026 make all                 \u0026\u0026 apk index -o ${tempDir}/packages/alpine/${apkArch}/APKINDEX.tar.gz ${tempDir}/packages/alpine/${apkArch}/*.apk                 \u0026\u0026 abuild-sign -k ${tempDir}/.abuild/abuild-key.rsa ${tempDir}/packages/alpine/${apkArch}/APKINDEX.tar.gz                 \"             \u0026\u0026 cp ${tempDir}/.abuild/abuild-key.rsa.pub /etc/apk/keys/             \u0026\u0026 apk del .build-deps             \u0026\u0026 apk add -X ${tempDir}/packages/alpine/ --no-cache $nginxPackages             ;;     esac     \u0026\u0026 apk del .checksum-deps     \u0026\u0026 if [ -n \"$tempDir\" ]; then rm -rf \"$tempDir\"; fi     \u0026\u0026 if [ -n \"/etc/apk/keys/abuild-key.rsa.pub\" ]; then rm -f /etc/apk/keys/abuild-key.rsa.pub; fi     \u0026\u0026 if [ -n \"/etc/apk/keys/nginx_signing.rsa.pub\" ]; then rm -f /etc/apk/keys/nginx_signing.rsa.pub; fi     \u0026\u0026 apk add --no-cache --virtual .gettext gettext     \u0026\u0026 mv /usr/bin/envsubst /tmp/         \u0026\u0026 runDeps=\"$(         scanelf --needed --nobanner /tmp/envsubst             | awk '{ gsub(/,/, \"\\nso:\", $2); print \"so:\" $2 }'             | sort -u             | xargs -r apk info --installed             | sort -u     )\"     \u0026\u0026 apk add --no-cache $runDeps     \u0026\u0026 apk del .gettext     \u0026\u0026 mv /tmp/envsubst /usr/local/bin/     \u0026\u0026 apk add --no-cache tzdata     \u0026\u0026 apk add --no-cache curl ca-certificates     \u0026\u0026 ln -sf /dev/stdout /var/log/nginx/access.log     \u0026\u0026 ln -sf /dev/stderr /var/log/nginx/error.log     \u0026\u0026 mkdir /docker-entrypoint.d"
        },
        {
          "created": "2022-11-12T06:27:46.606923928Z",
          "created_by": "/bin/sh -c #(nop) COPY file:7b307b62e82255f040c9812421a30090bf9abf3685f27b02d77fcca99f997911 in / "
        },
        {
          "created": "2022-11-12T06:27:46.72048155Z",
          "created_by": "/bin/sh -c #(nop) COPY file:5c18272734349488bd0c94ec8d382c872c1a0a435cca13bd4671353d6021d2cb in /docker-entrypoint.d "
        },
        {
          "created": "2022-11-12T06:27:46.83133408Z",
          "created_by": "/bin/sh -c #(nop) COPY file:abbcbf84dc17ee4454b6b2e3cf914be88e02cf84d344ec45a5b31235379d722a in /docker-entrypoint.d "
        },
        {
          "created": "2022-11-12T06:27:46.944034729Z",
          "created_by": "/bin/sh -c #(nop) COPY file:e57eef017a414ca793499729d80a7b9075790c9a804f930f1417e56d506970cf in /docker-entrypoint.d "
        },
        {
          "created": "2022-11-12T06:27:47.037384315Z",
          "created_by": "/bin/sh -c #(nop)  ENTRYPOINT [\"/docker-entrypoint.sh\"]",
          "empty_layer": true
        },
        {
          "created": "2022-11-12T06:27:47.135428797Z",
          "created_by": "/bin/sh -c #(nop)  EXPOSE 80",
          "empty_layer": true
        },
        {
          "created": "2022-11-12T06:27:47.235819248Z",
          "created_by": "/bin/sh -c #(nop)  STOPSIGNAL SIGQUIT",
          "empty_layer": true
        },
        {
          "created": "2022-11-12T06:27:47.33774049Z",
          "created_by": "/bin/sh -c #(nop)  CMD [\"nginx\" \"-g\" \"daemon off;\"]",
          "empty_layer": true
        }
      ],
      "os": "linux",
      "rootfs": {
        "type": "layers",
        "diff_ids": [
          "sha256:e5e13b0c77cbb769548077189c3da2f0a764ceca06af49d8d558e759f5c232bd",
          "sha256:07099189e7ec257e501d9625507b55e0ea32c330e38c90d8533b3fa2a7a97069",
          "sha256:fcf860bf48b4e20f24f44ba02115dc9f23eef6d41d69e9a050889bf25104e12a",
          "sha256:6636f46e559dffe6373b200c359773488f201ed2153507fb8d8fe3f04fdf477e",
          "sha256:9365b1fffb04e52b8f6abf1c8737ba4da02e134c1d8550e0ace4cb562d12f070",
          "sha256:bd502c2dee4c0bc2cf334c7d289e5a14ededd6c9c361137d128d3c12e4babf5d"
        ]
      },
      "config": {
        "Cmd": [
          "nginx",
          "-g",
          "daemon off;"
        ],
        "Entrypoint": [
          "/docker-entrypoint.sh"
        ],
        "Env": [
          "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
          "NGINX_VERSION=1.23.2",
          "NJS_VERSION=0.7.7",
          "PKG_RELEASE=1"
        ],
        "Image": "sha256:49c0d13884b7b451b306c6323aab0b6d259df7c9f44fc3ebdb29e76f94466ba9",
        "Labels": {
          "maintainer": "NGINX Docker Maintainers \u003cdocker-maint@nginx.com\u003e"
        },
        "ExposedPorts": {
          "80/tcp": {}
        },
        "StopSignal": "SIGQUIT"
      }
    }
  },
  "Results": [
    {
      "Target": "docker.io/library/nginx:1.23.2-alpine (alpine 3.16.3)",
      "Class": "os-pkgs",
      "Type": "alpine",
      "Vulnerabilities": [
        {
          "VulnerabilityID": "CVE-2022-43551",
          "PkgID": "curl@7.83.1-r4",
          "PkgName": "curl",
          "InstalledVersion": "7.83.1-r4",
          "FixedVersion": "7.83.1-r5",
          "Layer": {
            "Digest": "sha256:76a48b0f58980a64d28bc3575ae4733eb337f7b82403559122b13d5e2ced3921",
            "DiffID": "sha256:07099189e7ec257e501d9625507b55e0ea32c330e38c90d8533b3fa2a7a97069"
          },
          "SeveritySource": "nvd",
          "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2022-43551",
          "DataSource": {
            "ID": "alpine",
            "Name": "Alpine Secdb",
            "URL": "https://secdb.alpinelinux.org/"
          },
          "Title": "curl: HSTS bypass via IDN",
          "Description": "A vulnerability exists in curl \u003c7.87.0 HSTS check that could be bypassed to trick it to keep using HTTP. Using its HSTS support, curl can be instructed to use HTTPS instead of using an insecure clear-text HTTP step even when HTTP is provided in the URL. However, the HSTS mechanism could be bypassed if the host name in the given URL first uses IDN characters that get replaced to ASCII counterparts as part of the IDN conversion. Like using the character UTF-8 U+3002 (IDEOGRAPHIC FULL STOP) instead of the common ASCII full stop (U+002E) `.`. Then in a subsequent request, it does not detect the HSTS state and makes a clear text transfer. Because it would store the info IDN encoded but look for it IDN decoded.",
          "Severity": "HIGH",
          "CweIDs": [
            "CWE-319"
          ],
          "CVSS": {
            "nvd": {
              "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N",
              "V3Score": 7.5
            },
            "redhat": {
              "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N",
              "V3Score": 7.5
            }
          },
          "References": [
            "https://access.redhat.com/security/cve/CVE-2022-43551",
            "https://curl.se/docs/CVE-2022-43551.html",
            "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-43551",
            "https://hackerone.com/reports/1755083",
            "https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/TVWZW5CNSJ7UYAF2BGSYAWAEXDJYUBHA/",
            "https://nvd.nist.gov/vuln/detail/CVE-2022-43551",
            "https://ubuntu.com/security/notices/USN-5788-1"
          ],
          "PublishedDate": "2022-12-23T15:15:00Z",
          "LastModifiedDate": "2023-01-10T15:44:00Z"
        },
        {
          "VulnerabilityID": "CVE-2023-0286",
          "PkgID": "libcrypto1.1@1.1.1s-r0",
          "PkgName": "libcrypto1.1",
          "InstalledVersion": "1.1.1s-r0",
          "FixedVersion": "1.1.1t-r0",
          "Layer": {
            "Digest": "sha256:ca7dd9ec2225f2385955c43b2379305acd51543c28cf1d4e94522b3d94cce3ce",
            "DiffID": "sha256:e5e13b0c77cbb769548077189c3da2f0a764ceca06af49d8d558e759f5c232bd"
          },
          "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2023-0286",
          "DataSource": {
            "ID": "alpine",
            "Name": "Alpine Secdb",
            "URL": "https://secdb.alpinelinux.org/"
          },
          "Title": "There is a type confusion vulnerability relating to X.400 address proc ...",
          "Description": "There is a type confusion vulnerability relating to X.400 address processing inside an X.509 GeneralName. X.400 addresses were parsed as an ASN1_STRING but the public structure definition for GENERAL_NAME incorrectly specified the type of the x400Address field as ASN1_TYPE. This field is subsequently interpreted by the OpenSSL function GENERAL_NAME_cmp as an ASN1_TYPE rather than an ASN1_STRING. When CRL checking is enabled (i.e. the application sets the X509_V_FLAG_CRL_CHECK flag), this vulnerability may allow an attacker to pass arbitrary pointers to a memcmp call, enabling them to read memory contents or enact a denial of service. In most cases, the attack requires the attacker to provide both the certificate chain and CRL, neither of which need to have a valid signature. If the attacker only controls one of these inputs, the other input must already contain an X.400 address as a CRL distribution point, which is uncommon. As such, this vulnerability is most likely to only affect applications which have implemented their own functionality for retrieving CRLs over a network.",
          "Severity": "HIGH",
          "References": [
            "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-0286",
            "https://github.com/advisories/GHSA-x4qr-2fvf-3mr5",
            "https://github.com/pyca/cryptography/security/advisories/GHSA-x4qr-2fvf-3mr5",
            "https://nvd.nist.gov/vuln/detail/CVE-2023-0286",
            "https://rustsec.org/advisories/RUSTSEC-2023-0006.html",
            "https://ubuntu.com/security/notices/USN-5844-1",
            "https://ubuntu.com/security/notices/USN-5845-1",
            "https://ubuntu.com/security/notices/USN-5845-2",
            "https://www.openssl.org/news/secadv/20230207.txt"
          ],
          "PublishedDate": "2023-02-08T20:15:00Z",
          "LastModifiedDate": "2023-02-09T14:26:00Z"
        },
        {
          "VulnerabilityID": "CVE-2022-43551",
          "PkgID": "libcurl@7.83.1-r4",
          "PkgName": "libcurl",
          "InstalledVersion": "7.83.1-r4",
          "FixedVersion": "7.83.1-r5",
          "Layer": {
            "Digest": "sha256:76a48b0f58980a64d28bc3575ae4733eb337f7b82403559122b13d5e2ced3921",
            "DiffID": "sha256:07099189e7ec257e501d9625507b55e0ea32c330e38c90d8533b3fa2a7a97069"
          },
          "SeveritySource": "nvd",
          "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2022-43551",
          "DataSource": {
            "ID": "alpine",
            "Name": "Alpine Secdb",
            "URL": "https://secdb.alpinelinux.org/"
          },
          "Title": "curl: HSTS bypass via IDN",
          "Description": "A vulnerability exists in curl \u003c7.87.0 HSTS check that could be bypassed to trick it to keep using HTTP. Using its HSTS support, curl can be instructed to use HTTPS instead of using an insecure clear-text HTTP step even when HTTP is provided in the URL. However, the HSTS mechanism could be bypassed if the host name in the given URL first uses IDN characters that get replaced to ASCII counterparts as part of the IDN conversion. Like using the character UTF-8 U+3002 (IDEOGRAPHIC FULL STOP) instead of the common ASCII full stop (U+002E) `.`. Then in a subsequent request, it does not detect the HSTS state and makes a clear text transfer. Because it would store the info IDN encoded but look for it IDN decoded.",
          "Severity": "HIGH",
          "CweIDs": [
            "CWE-319"
          ],
          "CVSS": {
            "nvd": {
              "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N",
              "V3Score": 7.5
            },
            "redhat": {
              "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N",
              "V3Score": 7.5
            }
          },
          "References": [
            "https://access.redhat.com/security/cve/CVE-2022-43551",
            "https://curl.se/docs/CVE-2022-43551.html",
            "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-43551",
            "https://hackerone.com/reports/1755083",
            "https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/TVWZW5CNSJ7UYAF2BGSYAWAEXDJYUBHA/",
            "https://nvd.nist.gov/vuln/detail/CVE-2022-43551",
            "https://ubuntu.com/security/notices/USN-5788-1"
          ],
          "PublishedDate": "2022-12-23T15:15:00Z",
          "LastModifiedDate": "2023-01-10T15:44:00Z"
        },
        {
          "VulnerabilityID": "CVE-2023-0286",
          "PkgID": "libssl1.1@1.1.1s-r0",
          "PkgName": "libssl1.1",
          "InstalledVersion": "1.1.1s-r0",
          "FixedVersion": "1.1.1t-r0",
          "Layer": {
            "Digest": "sha256:ca7dd9ec2225f2385955c43b2379305acd51543c28cf1d4e94522b3d94cce3ce",
            "DiffID": "sha256:e5e13b0c77cbb769548077189c3da2f0a764ceca06af49d8d558e759f5c232bd"
          },
          "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2023-0286",
          "DataSource": {
            "ID": "alpine",
            "Name": "Alpine Secdb",
            "URL": "https://secdb.alpinelinux.org/"
          },
          "Title": "There is a type confusion vulnerability relating to X.400 address proc ...",
          "Description": "There is a type confusion vulnerability relating to X.400 address processing inside an X.509 GeneralName. X.400 addresses were parsed as an ASN1_STRING but the public structure definition for GENERAL_NAME incorrectly specified the type of the x400Address field as ASN1_TYPE. This field is subsequently interpreted by the OpenSSL function GENERAL_NAME_cmp as an ASN1_TYPE rather than an ASN1_STRING. When CRL checking is enabled (i.e. the application sets the X509_V_FLAG_CRL_CHECK flag), this vulnerability may allow an attacker to pass arbitrary pointers to a memcmp call, enabling them to read memory contents or enact a denial of service. In most cases, the attack requires the attacker to provide both the certificate chain and CRL, neither of which need to have a valid signature. If the attacker only controls one of these inputs, the other input must already contain an X.400 address as a CRL distribution point, which is uncommon. As such, this vulnerability is most likely to only affect applications which have implemented their own functionality for retrieving CRLs over a network.",
          "Severity": "HIGH",
          "References": [
            "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-0286",
            "https://github.com/advisories/GHSA-x4qr-2fvf-3mr5",
            "https://github.com/pyca/cryptography/security/advisories/GHSA-x4qr-2fvf-3mr5",
            "https://nvd.nist.gov/vuln/detail/CVE-2023-0286",
            "https://rustsec.org/advisories/RUSTSEC-2023-0006.html",
            "https://ubuntu.com/security/notices/USN-5844-1",
            "https://ubuntu.com/security/notices/USN-5845-1",
            "https://ubuntu.com/security/notices/USN-5845-2",
            "https://www.openssl.org/news/secadv/20230207.txt"
          ],
          "PublishedDate": "2023-02-08T20:15:00Z",
          "LastModifiedDate": "2023-02-09T14:26:00Z"
        }
      ]
    }
  ]
}

$ kubectl get vulnerabilityreports.aquasecurity.github.io -A
No resources found
chen-keinan commented 1 year ago

@ledroide have you changed the imageRef to ghcr.io/aquasecurity/trivy:latest ? is this issue happening with imagRef: ghcr.io/aquasecurity/trivy:0.36.0 (default in operator v0.11.0) ?

there was a Breaking / Deprecation changes in API with trivy 0.37.x

ledroide commented 1 year ago

have you changed the imageRef to ghcr.io/aquasecurity/trivy:latest ?

Yes, it was an attempt to try fixing this issue, but with no effect. I'll try again with tags 0.36.0 and 0.37.1 and let you know.

chen-keinan commented 1 year ago

Yes, it was an attempt to try fixing this issue, but with no effect. I'll try again with tags 0.36.0 and 0.37.1 and let you know.

thanks , it will be great to get (with trivy v0.36.0)

kubectl logs -n trivy-system

where compressLogs=true

note: need to make sure that scan jobs from prev check are deleted before trying again with imageRef of trivy v0.36.0

jrhunger commented 1 year ago

@chen-keinan is the 0.37.x breaking issue you mentioned this one: https://github.com/aquasecurity/trivy/pull/3467 I am seeing the same base64 errors. We use compressed logs. Here is an example output:

# dev-car2: kubectl -n trivy-system logs jobs/scan-vulnerabilityreport-554f599ffb | head
2023-02-10T17:34:06.830Z        WARN    '--security-checks' is deprecated. Use '--scanners' instead.
QlpoOTFBWSZTWeNoOqsBd+/fgHVQeu//+v////6////6YHceHnya927A6AkHbCgAB6ANFUVEBQFF
Nbdfe+vlF9tTOUUBnc5waqDZdt8wHbnVBdMPu76p7KK2EzH0O3udRTUgqZ6uByK9M9tD3K9cAPcw
DUjE+7ORsZoLVo2VqyVjW2ZaBUVBEHoQVuyV8HtdgCQpq2AGlNraBQAzIvVGrrema1IQkeex69ce
b0Pet22tWkJvLQPbAAOxldFtem1ZugADJEnd3oHkij2U2NlALNibTaqetEnbIVFKo1ZQBQC2kKBR

If i grep -v WARN on that and pipe through base64 --decode it results in a bzip2 json file.

chen-keinan commented 1 year ago

@chen-keinan is the 0.37.x breaking issue you mentioned this one: aquasecurity/trivy#3467 I am seeing the same base64 errors. We use compressed logs. Here is an example output:

# dev-car2: kubectl -n trivy-system logs jobs/scan-vulnerabilityreport-554f599ffb | head
2023-02-10T17:34:06.830Z        WARN    '--security-checks' is deprecated. Use '--scanners' instead.
QlpoOTFBWSZTWeNoOqsBd+/fgHVQeu//+v////6////6YHceHnya927A6AkHbCgAB6ANFUVEBQFF
Nbdfe+vlF9tTOUUBnc5waqDZdt8wHbnVBdMPu76p7KK2EzH0O3udRTUgqZ6uByK9M9tD3K9cAPcw
DUjE+7ORsZoLVo2VqyVjW2ZaBUVBEHoQVuyV8HtdgCQpq2AGlNraBQAzIvVGrrema1IQkeex69ce
b0Pet22tWkJvLQPbAAOxldFtem1ZugADJEnd3oHkij2U2NlALNibTaqetEnbIVFKo1ZQBQC2kKBR

If i grep -v WARN on that and pipe through base64 --decode it results in a bzip2 json file.

@jrhunger true , that's why we didn't move to v0.37.0 yet in trivy-operator , it require code change, it need to replace --security-checks with --scanners on container command

bogdanioanliviu commented 1 year ago

@ledroide have you changed the imageRef to ghcr.io/aquasecurity/trivy:latest ? is this issue happening with imagRef: ghcr.io/aquasecurity/trivy:0.36.0 (default in operator v0.11.0) ?

there was a Breaking / Deprecation changes in API with trivy 0.37.x

Yes, Trivy-operator 0.11 with trivy 0.36.0, fix this error "error": "illegal base64 data at input byte 4". For me the fix was : delete Trivy-operator and all related resources, redeploy with the trivy 0.36.0.

chen-keinan commented 1 year ago

Related #948

ledroide commented 1 year ago

@chen-keinan : You have found the root cause. Thanks a lot. I confirm that trivy-operator 0.11.0 :

I have seen that #949 is merged, and will wait for the next trivy-operator tag and image build to retry with trivy 0.37.x.