search

aquasecurity / starboard

Moved to https://github.com/aquasecurity/trivy-operator
https://aquasecurity.github.io/starboard/
Apache License 2.0
1.35k stars 197 forks source link

Reconciler error: "error":"getting logs for pod" #1031

Open albertschwarzkopf opened 2 years ago

albertschwarzkopf commented 2 years ago

What steps did you take and what happened:

After few days starboard operator stucks in following error:

"level":"error","ts":1647269605.345396,"logger":"controller.job","msg":"Reconciler error","reconciler group":"batch","reconciler kind":"Job","name":"scan-vulnerabilityreport-787ccf9b67","namespace":"starboard-system","error":"getting logs for pod \"starboard-system/scan-vulnerabilityreport-787ccf9b67\": getting pod controlled by job: \"starboard-system/scan-vulnerabilityreport-787ccf9b67\": pod not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:227"}

I can see the finished job "scan-vulnerabilityreport-787ccf9b67" in its status "Complete". But there is no pod for this job. Maybe the pod was deleted because the worker node was terminated (because we use spot instances in AWS EKS). Is it possible that such completed job are deleted after X hours,days,...? E.g. ttlSecondsAfterFinished for K8s-jobs?

Environment:

We use Starboard-Operator combined with trivy in client-server-mode. Deployed via Helm Charts:

Starboard Operator Helm-Chart-Version: 0.9.1 (app-version 0.14.1) Trivy-Server Helm-Chart-Version: 0.4.10 (app-version: 0.24.0) AWS EKS 1.21 (Bottlerocket OS and AmazonLinux 2)

albertschwarzkopf commented 2 years ago

Today I have again an "scan-vulnerabilityreport" pod and corresponding job which are in status "Completed".

But starboard operator has following error:

{"level":"error","ts":1647341024.8277369,"logger":"controller.job","msg":"Reconciler error","reconciler group":"batch","reconciler kind":"Job","name":"scan-vulnerabilityreport-7b89599899","namespace":"starboard-system","error":"unexpected EOF","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:227"}

kubectl -n starboard-system logs scan-vulnerabilityreport-7b89599899-tlmpt

The last part of the log: ......

        {
          "VulnerabilityID": "CVE-2013-4235",
          "PkgName": "passwd",
          "InstalledVersion": "1:4.5-1.1",
          "Layer": {
            "Digest": "sha256:15115158dd02a1bf2fd28724e3c1024394033fb0e9a5d3e451ed2715b6ae312d",
            "DiffID": "sha256:e5baccb54724b971f73bbfa46d477b947c9066e4040d0e002e8f04314f58b58f"
          },
          "SeveritySource": "debian",
          "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2013-4235",
          "DataSource": {
            "ID": "debian",
            "Name": "Debian Security Tracker",
            "URL": "https://salsa.debian.org/security-tracker-team/security-tracker"
          },
          "Title": "shadow-utils: TOCTOU race conditions by copying and removing directory trees",
          "Description": "shadow: TOCTOU (time-of-check time-of-use) race condition when copying and removing directory trees",
          "Severity": "LOW",
          "CweIDs": [
            "CWE-367"
          ],
          "CVSS": {
            "nvd": {
              "V2Vector": "AV:L/AC:M/Au:N/C:N/I:P/A:P",
              "V3Vector": "CVSS:3.1/AV:L/AC:H/PR:L/UI:N/S:U/C:N/I:H/A:N",
              "V2Score": 3.3,
              "V3Score": 4.7
            },
            "redhat": {
              "V2Vector": "AV:L/AC:H/Au:N/C:P/I:P/A:P",
              "V3Vector": "CVSS:3.1/AV:L/AC:H/PR:L/UI:R/S:U/C:N/I:H/A:N",
              "V2Score": 3.7,
              "V3Score": 4.4
            }
          },
          "References": [
            "https://access.redhat.com/security/cve/CVE-2013-4235",
            "https://access.redhat.com/security/cve/cve-2013-4235",
            "https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2013-4235",
            "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-4235",
            "https://lists.apache.org/thread.html/rf9fa47ab66495c78bb4120b0754dd9531ca2ff0430f6685ac9b07772@%3Cdev.mina.apache.org%3E",
            "https://security-tracker.debian.org/tracker/CVE-2013-4235"
          ],
          "PublishedDate": "2019-12-03T15:15:00Z",
          "LastModifiedDate": "2021-02-25T17:15:00Z"
        },
        {
          "VulnerabilityID": "CVE-2018-7169",
          "PkgName": "passwd",
          "InstalledVersion": "1:4.5-1.1",
          "Layer": {
            "Digest": "sha256:15115158dd02a1bf2fd28724e3c1024394033fb0e9a5d3%
danielpacak commented 2 years ago

What about VulnerabilityReport? Is it created after all?

albertschwarzkopf commented 2 years ago

What about VulnerabilityReport? Is it created after all?

Yes the VR for the specific image exists. But new VR were not created.

danielpacak commented 2 years ago

I'm not sure I understood. What do you mean by "new VR"?

albertschwarzkopf commented 2 years ago

I'm not sure I understood. What do you mean by "new VR"?

We use the "OPERATOR_VULNERABILITY_SCANNER_REPORT_TTL" (vulnerabilityScannerReportTTL) parameter. So that the VulnerabilityReports (VRs) are generated every 24h.

cnwaldron commented 2 years ago

I'm also getting the "error":"unexpected EOF" message on some scans. Looking at the log of the job, I see the JSON response ends at:

{
...,
"Results": [
....
]

The interesting thing is one of the images this happens on has already been scanned from another deployment and had no issues. The vulnerability report is NOT created when this happens.

Additional Note: I copied the scan job yaml for the failing scan, I deployed it as a separate job to see what would happen and the JSON results came back just fine.

albertschwarzkopf commented 2 years ago

Today it happend again:

╰─ kubectl -n starboard-system get pods
NAME                                        READY   STATUS      RESTARTS   AGE
scan-vulnerabilityreport-77444bf746-lzlq7   0/1     Completed   0          24h
starboard-exporter-6fc5c8f9c6-6bhx5         1/1     Running     0          53d
starboard-operator-866776846f-tdcg8         1/1     Running     0          19d
trivy-server-0                              1/1     Running     0          30d

╰─ kubectl -n starboard-system get job 
NAME                                  COMPLETIONS   DURATION   AGE
scan-vulnerabilityreport-77444bf746   1/1           3s         24h

╰─ kubectl -n starboard-system logs scan-vulnerabilityreport-77444bf746-lzlq7
        {
          "VulnerabilityID": "CVE-2020-16156",
          "PkgName": "perl-base",
          "InstalledVersion": "5.28.1-6+deb10u1",
          "Layer": {
            "Digest": "sha256:6552179c3509e3c4314b4065e0d2790563d01cd474e2fdd58be4d46acd48af6a",
            "DiffID": "sha256:f18b02b14138b6f9808f9843cc645e2edd64b02ca1c87e671355f56d1b4b5ec6"
          },
          "SeveritySource": "nvd",
          "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2020-16156",
          "DataSource": {
            "ID": "debian",
            "Name": "Debian Security Tracker",
            "URL": "https://salsa.debian.org/security-tracker-team/security-tracker"
          },
          "Title": "perl-CPAN: Bypass of verification of signatures in CHECKSUMS files",
          "Description": "CPAN 2.28 allows Signature Verification Bypass.",
          "Severity": "HIGH",
          "CweIDs": [
            "CWE-347"
          ],
          "CVSS": {
            "nvd": {
              "V2Vector": "AV:N/AC:M/Au:N/C:P/I:P/A:P",
              "V3Vector": "CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H",
              "V2Score": 6.8,
              "V3Score": 7.8
            },
            "redhat": {
              "V3Vector": "CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H",
              "V3Score": 7.8
            }
          },
          "References": [
            "http://blogs.per%                                                               

╰─ kubectl -n starboard-system logs starboard-operator-866776846f-tdcg8

{"level":"error","ts":1649768062.94792,"logger":"controller.job","msg":"Reconciler error","reconciler group":"batch","reconciler kind":"Job","name":"scan-vulnerabilityreport-77444bf746","namespace":"starboard-system","error":"unexpected EOF","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:227"}

One of the scan-jobs is completed, but the above last block in the JSON is invalid, because it ends suddenly. And then the starboard-operator does not start other scans.

Only after deletion of the "hanging" scan-job, starboard-operator starts other scans.

elchenberg commented 2 years ago

I can confirm the issue that @albertschwarzkopf mentioned in https://github.com/aquasecurity/starboard/issues/1031#issuecomment-1067836429 with two different kind of errors.

I guess I can avoid the errors with changes on my side by fixing some security issues with the xdebug image (then the EOF error should disappear) and fixing the version skew with the starboard-operator and trivy images that I use (I did not notice that the public.ecr.aws/aquasecurity/starboard-operator image is not up-to-date and that I am multiple versions ahead with the trivy image from what the starboard chart uses).

Still, I wish starboard-operator would be more fault-tolerant because when scanJobsConcurrentLimit have been completed (successfully) and they do not get removed Starboard stops scanning altogether (until I notice and delete the jobs manually).

detailed versions, logs, error messages, etc. - Kubernetes (bare metal): ``` Client Version: v1.23.6 Server Version: v1.23.6 # and Client Version: v1.24.0 Kustomize Version: v4.5.4 Server Version: v1.24.0 ``` - Starboard: ``` # from the statefulset's .metadata-labels: app.kubernetes.io/name: starboard-operator app.kubernetes.io/version: 0.15.4 helm.sh/chart: starboard-operator-0.10.4 # actual image used: public.ecr.aws/aquasecurity/starboard-operator:0.14.1 ``` - Trivy: ``` # from the statefulset's .metadata-labels: app.kubernetes.io/name: trivy app.kubernetes.io/version: 0.25.0 helm.sh/chart: trivy-0.4.13 # actual image used: image: ghcr.io/aquasecurity/trivy:0.28.0 ``` ### Error: unexpected EOF From the starboard-operator logs: ```json { "level": "error", "ts": 1652870248.4415047, "logger": "controller.job", "msg": "Reconciler error", "reconciler group": "batch", "reconciler kind": "Job", "name": "scan-vulnerabilityreport-5c57c4d49", "namespace": "starboard-operator", "error": "unexpected EOF", "stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:227" } ``` The pod has 3 containers but only one of them contains invalid JSON: ``` # kubectl --namespace starboard-operator logs pod/scan-vulnerabilityreport-5c57c4d49-kdjgt -c xdebug | jq parse error: Unfinished JSON term at EOF at line 13638, column 3 ``` I won't paste all 13638 log lines here, but it looks like valid JSON until the output stops in the middle: ```json { "VulnerabilityID": "CVE-2018-25032", "VendorIDs": [ "DSA-5111-1" ], "PkgName": "zlib1g", "InstalledVersion": "1:1.2.11.dfsg-2", "FixedVersion": "1:1.2.11.dfsg-2+deb11u1", "Layer": { "Digest": "sha256:7d63c13d9b9b6ec5f05a2b07daadacaa9c610d01102a662ae9b1d082105f1ffa", "DiffID": "sha256:e8b689711f21f9301c40bf2131ce1a1905c3aa09def1de5ec43cf0adf652576e" }, "SeveritySource": "nvd", "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2018-25032", "DataSource": { "ID": "debian", "Name": "Debian Security Tracker", "URL": "https://salsa.debian.org/security-tracker-team/security-tracker" }, "Title": "zlib: A flaw found in zlib when compressing (not decompressing) certain inputs", "Description": "zlib before 1.2.12 allows memory corruption when deflating (i.e., when compressing) if the input has many distant matches.", "Severity": "HIGH", "CweIDs": [ "CWE-787" ], "CVSS": { "nvd": { "V2Vector": "AV:N/AC:L/Au:N/C:N/I:N/A:P", "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H", "V2Score": 5, "V3Score": 7.5 }, "redhat": { "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:L/A:H", "V3Score": 8.2 } }, "References": [ "http://www.openwall.com/lists/oss-security/2022/03/25/2", "http://www.openwall.com/lists/oss-security/2022/03/26/1", "https://access.redhat.com/security/cve/CVE-2018-25032", "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-25032", "https://github.com/madler/zlib/commit/5c44459c3b28a9bd3283aaceab7c615f8020c531", ``` ### Error: json: cannot unmarshal number into Go value of type trivy.ScanReport From the starboard-operator logs: ```json { "level": "error", "ts": 1652869244.7950468, "logger": "controller.job", "msg": "Reconciler error", "reconciler group": "batch", "reconciler kind": "Job", "name": "scan-vulnerabilityreport-54f9f659bb", "namespace": "starboard-operator", "error": "json: cannot unmarshal number into Go value of type trivy.ScanReport", "stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:227" } ``` Command of the corresponding pod: ```yaml args: - --quiet - client - --format - json - --remote - http://trivy:4954 - quay.io/ceph/ceph:v16.2.7@sha256:00965b7e88c0cef116e6a47107051a4bfe952139e7b94c6cefd6607cf38a3f0f command: - trivy ``` Logs of the corresponding pod:
``` 2022-05-18T02:11:57.739Z WARN `client` subcommand is deprecated now. See https://github.com/aquasecurity/trivy/discussions/2119 { "SchemaVersion": 2, "ArtifactName": "quay.io/ceph/ceph:v16.2.7@sha256:00965b7e88c0cef116e6a47107051a4bfe952139e7b94c6cefd6607cf38a3f0f", "ArtifactType": "container_image", "Metadata": { "OS": { "Family": "redhat", "Name": "8.5" }, "ImageID": "sha256:463ec4b1fdc0111fbd413d233319c92eeb7c038c6c26e54af828e1637d94a2d1", "DiffIDs": [ "sha256:3813924f3fa4d6d1223ec6c5e8bb2f5f9acd1e49d87a9251364ff99ab52b3296", "sha256:c86122b5e4d3e8a0d438442340c75c8e35c343945b1c750b15e53b5efdcbc27d", "sha256:e9b312bba1abbd4bd396d9d543063338470cfcc884ba98ce56bbb7a40a981882", "sha256:5d9ef8a0564d47b299fb4c6f92181d876cf985d392f9426d116366b0bafd603a", "sha256:5232e08c6f588f9455b167705bfc14894a64ef2cf9ce6f72cb38733e97c55223" ], "RepoDigests": [ "quay.io/ceph/ceph@sha256:00965b7e88c0cef116e6a47107051a4bfe952139e7b94c6cefd6607cf38a3f0f" ], "ImageConfig": { "architecture": "amd64", "container": "b863ef8a0d77ef6cc9ec0f888a7a0f1a1b003cf46d65d2373301b912362aef3d", "created": "2022-05-10T13:12:52.545152509Z", "docker_version": "20.10.7", "history": [ { "created": "2022-04-29T11:48:25.316216031Z", "comment": "Imported from -" }, { "created": "2022-04-29T11:48:36.951429Z" }, { "created": "2022-05-05T13:58:20.221551743Z", "created_by": "/bin/sh -c #(nop) LABEL maintainer=\"The CentOS Project\"", "empty_layer": true }, { "created": "2022-05-05T13:58:20.276348007Z", "created_by": "/bin/sh -c #(nop) LABEL com.redhat.component=\"centos-stream-container\" name=\"centos-stream\" version=\"8\"", "empty_layer": true }, { "created": "2022-05-05T13:58:20.315016722Z", "created_by": "/bin/sh -c #(nop) LABEL com.redhat.license_terms=\"https://centos.org/legal/licensing-policy/\"", "empty_layer": true }, { "created": "2022-05-05T13:58:20.352707598Z", "created_by": "/bin/sh -c #(nop) LABEL summary=\"Provides a CentOS Stream container based on the Red Hat Universal Base Image\"", "empty_layer": true }, { "created": "2022-05-05T13:58:20.391057433Z", "created_by": "/bin/sh -c #(nop) LABEL description=\"CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream\"", "empty_layer": true }, { "created": "2022-05-05T13:58:20.430921058Z", "created_by": "/bin/sh -c #(nop) LABEL io.k8s.display-name=\"CentOS Stream 8\"", "empty_layer": true }, { "created": "2022-05-05T13:58:20.470189581Z", "created_by": "/bin/sh -c #(nop) LABEL io.openshift.expose-services=\"\"", "empty_layer": true }, { "created": "2022-05-05T13:58:20.512837381Z", "created_by": "/bin/sh -c #(nop) LABEL io.openshift.tags=\"base centos centos-stream\"", "empty_layer": true }, { "created": "2022-05-05T13:58:27.848320829Z", "created_by": "/bin/sh -c dnf download --repofrompath=centos,http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/ --disablerepo=* --enablerepo=centos centos-stream-release centos-stream-repos centos-gpg-keys" }, { "created": "2022-05-05T13:59:02.744513473Z", "created_by": "/bin/sh -c rpm -ivh --nodeps --replacefiles *.rpm \u0026\u0026 rm *.rpm \u0026\u0026 rpm -e redhat-release \u0026\u0026 dnf --setopt=tsflags=nodocs --setopt=install_weak_deps=false -y distro-sync \u0026\u0026 dnf remove -y subscription-manager dnf-plugin-subscription-manager \u0026\u0026 dnf clean all \u0026\u0026 rm -f /etc/yum.repos.d/ubi.repo" }, { "created": "2022-05-10T13:01:48.858095731Z", "created_by": "/bin/sh -c #(nop) ENV I_AM_IN_A_CONTAINER=1", "empty_layer": true }, { "created": "2022-05-10T13:01:50.116086562Z", "created_by": "/bin/sh -c #(nop) LABEL maintainer=Guillaume Abrioux \u003cgabrioux@redhat.com\u003e", "empty_layer": true }, { "created": "2022-05-10T13:01:51.332136283Z", "created_by": "/bin/sh -c #(nop) LABEL ceph=True", "empty_layer": true }, { "created": "2022-05-10T13:01:52.556492822Z", "created_by": "/bin/sh -c #(nop) LABEL RELEASE=HEAD", "empty_layer": true }, { "created": "2022-05-10T13:01:53.755821612Z", "created_by": "/bin/sh -c #(nop) LABEL GIT_REPO=https://github.com/ceph/ceph-container.git", "empty_layer": true }, { "created": "2022-05-10T13:01:54.955083228Z", "created_by": "/bin/sh -c #(nop) LABEL GIT_BRANCH=HEAD", "empty_layer": true }, { "created": "2022-05-10T13:01:56.212610001Z", "created_by": "/bin/sh -c #(nop) LABEL GIT_COMMIT=c7639748589442d7aeaac988ada4549145ccaa4d", "empty_layer": true }, { "created": "2022-05-10T13:01:57.486849555Z", "created_by": "/bin/sh -c #(nop) LABEL GIT_CLEAN=True", "empty_layer": true }, { "created": "2022-05-10T13:08:46.50798259Z", "created_by": "/bin/sh -c #(nop) LABEL CEPH_POINT_RELEASE=-16.2.7", "empty_layer": true }, { "created": "2022-05-10T13:08:47.966380091Z", "created_by": "/bin/sh -c #(nop) ENV CEPH_VERSION=pacific", "empty_layer": true }, { "created": "2022-05-10T13:08:49.232133321Z", "created_by": "/bin/sh -c #(nop) ENV CEPH_POINT_RELEASE=-16.2.7", "empty_layer": true }, { "created": "2022-05-10T13:08:50.489484086Z", "created_by": "/bin/sh -c #(nop) ENV CEPH_DEVEL=false", "empty_layer": true }, { "created": "2022-05-10T13:08:51.738582744Z", "created_by": "/bin/sh -c #(nop) ENV CEPH_REF=pacific", "empty_layer": true }, { "created": "2022-05-10T13:08:53.004420274Z", "created_by": "/bin/sh -c #(nop) ENV OSD_FLAVOR=default", "empty_layer": true }, { "created": "2022-05-10T13:12:52.545152509Z", "created_by": "/bin/sh -c yum install -y epel-release \u0026\u0026 yum install -y jq \u0026\u0026 bash -c ' if [ -n \"nfs-ganesha nfs-ganesha-ceph nfs-ganesha-rgw nfs-ganesha-rados-grace nfs-ganesha-rados-urls\" ]; then echo \"[ganesha]\" \u003e /etc/yum.repos.d/ganesha.repo ; echo \"name=ganesha\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; if [[ \"${CEPH_VERSION}\" == master ]]; then echo \"baseurl=https://buildlogs.centos.org/centos/\\$releasever/storage/\\$basearch/nfsganesha-3/\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"gpgcheck=0\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"enabled=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; elif [[ \"${CEPH_VERSION}\" == quincy ]]; then echo \"baseurl=https://buildlogs.centos.org/centos/\\$releasever/storage/\\$basearch/nfsganesha-4/\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"gpgcheck=0\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"enabled=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; elif [[ \"${CEPH_VERSION}\" == pacific ]]; then echo \"baseurl=https://download.ceph.com/nfs-ganesha/rpm-V3.5-stable/$CEPH_VERSION/el\\$releasever/\\$basearch/\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"gpgcheck=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"enabled=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"[ganesha-noarch]\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"name=ganesha-noarch\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"baseurl=https://download.ceph.com/nfs-ganesha/rpm-V3.5-stable/$CEPH_VERSION/el\\$releasever/noarch/\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"gpgcheck=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"enabled=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; elif [[ \"${CEPH_VERSION}\" == octopus ]]; then echo \"baseurl=https://download.ceph.com/nfs-ganesha/rpm-V3.3-stable/$CEPH_VERSION/el\\$releasever/\\$basearch/\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"gpgcheck=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"enabled=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"[ganesha-noarch]\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"name=ganesha-noarch\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"baseurl=https://download.ceph.com/nfs-ganesha/rpm-V3.3-stable/$CEPH_VERSION/el\\$releasever/noarch/\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"gpgcheck=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"enabled=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; elif [[ \"${CEPH_VERSION}\" == nautilus ]]; then echo \"baseurl=https://download.ceph.com/nfs-ganesha/rpm-V2.8-stable/$CEPH_VERSION/\\$basearch/\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"gpgcheck=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"enabled=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; else echo \"baseurl=https://download.ceph.com/nfs-ganesha/rpm-V2.7-stable/$CEPH_VERSION/\\$basearch/\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"gpgcheck=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; echo \"enabled=1\" \u003e\u003e /etc/yum.repos.d/ganesha.repo ; fi ; fi ; if [ -n \"tcmu-runner ceph-iscsi python3-rtslib\" ]; then curl -s -L https://shaman.ceph.com/api/repos/tcmu-runner/master/latest/centos/8/repo?arch=$(arch) -o /etc/yum.repos.d/tcmu-runner.repo ; if [[ \"${CEPH_VERSION}\" =~ master ]]; then curl -s -L https://shaman.ceph.com/api/repos/ceph-iscsi/master/latest/centos/8/repo -o /etc/yum.repos.d/ceph-iscsi.repo ; elif [[ \"${CEPH_VERSION}\" =~ nautilus|octopus|pacific|quincy ]]; then curl -s -L https://download.ceph.com/ceph-iscsi/3/rpm/el8/ceph-iscsi.repo -o /etc/yum.repos.d/ceph-iscsi.repo ; else curl -s -L https://download.ceph.com/ceph-iscsi/2/rpm/el8/ceph-iscsi.repo -o /etc/yum.repos.d/ceph-iscsi.repo ; fi ; fi' \u0026\u0026 yum update -y --setopt=install_weak_deps=False \u0026\u0026 rpm --import 'https://download.ceph.com/keys/release.asc' \u0026\u0026 if [[ \"${CEPH_VERSION}\" == nautilus ]]; then CEPH_MGR_K8SEVENTS=\"ceph-mgr-k8sevents-16.2.7\"; if [[ -n \"-16.2.7\" ]]; then CPR=-16.2.7 ; if [[ ${CPR:1:2} -eq 14 ]] \u0026\u0026 [[ ${CPR:4:1} -eq 2 ]] \u0026\u0026 [[ ${CPR:6} -lt 5 ]]; then CEPH_MGR_K8SEVENTS=\"\" ; fi ; fi ; fi \u0026\u0026 bash -c ' if [[ \"${CEPH_VERSION}\" =~ master ]] || ${CEPH_DEVEL}; then ARCH=$(arch); if [[ \"${ARCH}\" == \"aarch64\" ]]; then ARCH=\"arm64\"; fi ; REPO_URL=$(curl -s \"https://shaman.ceph.com/api/search/?project=ceph\u0026distros=centos/8/${ARCH}\u0026flavor=${OSD_FLAVOR}\u0026ref=${CEPH_REF}\u0026sha1=latest\" | jq -r .[0].url); RELEASE_VER=0 ; if [[ \"${OSD_FLAVOR}\" == \"crimson\" ]]; then CRIMSON_PACKAGES=\"ceph-crimson-osd-16.2.7\"; fi ; if [[ \"${CEPH_REF}\" == \"octopus\" ]]; then RELEASE_VER=1 ; fi ; else RELEASE_VER=1 ; REPO_URL=\"http://download.ceph.com/rpm-${CEPH_VERSION}/el8/\"; fi \u0026\u0026 rpm -Uvh \"$REPO_URL/noarch/ceph-release-1-${RELEASE_VER}.el8.noarch.rpm\" \u0026\u0026 if [[ 8 -eq 8 ]]; then yum install -y dnf-plugins-core ; yum copr enable -y tchaikov/python-scikit-learn ; yum copr enable -y tchaikov/python3-asyncssh ; fi ' \u0026\u0026 yum install -y --setopt=install_weak_deps=False ca-certificates e2fsprogs ceph-common-16.2.7 ceph-mon-16.2.7 ceph-osd-16.2.7 ceph-mds-16.2.7 cephfs-mirror-16.2.7 rbd-mirror-16.2.7 ceph-mgr-16.2.7 ceph-mgr-cephadm-16.2.7 ceph-mgr-dashboard-16.2.7 ceph-mgr-diskprediction-local-16.2.7 ceph-mgr-k8sevents-16.2.7 ceph-mgr-rook-16.2.7 python3-saml ceph-grafana-dashboards-16.2.7 kmod lvm2 gdisk smartmontools nvme-cli libstoragemgmt systemd-udev procps-ng hostname ceph-radosgw-16.2.7 libradosstriper1-16.2.7 nfs-ganesha nfs-ganesha-ceph nfs-ganesha-rgw nfs-ganesha-rados-grace nfs-ganesha-rados-urls tcmu-runner ceph-iscsi python3-rtslib attr ceph-fuse-16.2.7 rbd-nbd-16.2.7 ${CRIMSON_PACKAGES} ceph-immutable-object-cache-16.2.7 python3-scikit-learn \u0026\u0026 INITIAL_SIZE=\"$(bash -c 'sz=\"$(du -sm --exclude=/proc /)\" ; echo \"${sz%*/}\"')\" \u0026\u0026 echo 'Postinstall cleanup' \u0026\u0026 (rm -rf \"/usr/bin/hyperkube /usr/bin/etcd /usr/bin/systemd-analyze /usr/share/hwdata/{iab.txt,oui.txt} /etc/profile.d/lang.sh\" \u0026\u0026 yum clean all \u0026\u0026 rpm -q ca-certificates e2fsprogs ceph-common-16.2.7 ceph-mon-16.2.7 ceph-osd-16.2.7 ceph-mds-16.2.7 cephfs-mirror-16.2.7 rbd-mirror-16.2.7 ceph-mgr-16.2.7 ceph-mgr-cephadm-16.2.7 ceph-mgr-dashboard-16.2.7 ceph-mgr-diskprediction-local-16.2.7 ceph-mgr-k8sevents-16.2.7 ceph-mgr-rook-16.2.7 python3-saml ceph-grafana-dashboards-16.2.7 kmod lvm2 gdisk smartmontools nvme-cli libstoragemgmt systemd-udev procps-ng hostname ceph-radosgw-16.2.7 libradosstriper1-16.2.7 nfs-ganesha nfs-ganesha-ceph nfs-ganesha-rgw nfs-ganesha-rados-grace nfs-ganesha-rados-urls tcmu-runner ceph-iscsi python3-rtslib attr ceph-fuse-16.2.7 rbd-nbd-16.2.7 ${CRIMSON_PACKAGES} ceph-immutable-object-cache-16.2.7 python3-scikit-learn ) \u0026\u0026 sed -i -e 's/udev_rules = 1/udev_rules = 0/' -e 's/udev_sync = 1/udev_sync = 0/' -e 's/obtain_device_list_from_udev = 1/obtain_device_list_from_udev = 0/' /etc/lvm/lvm.conf \u0026\u0026 grep -sqo \"udev_sync = 0\" /etc/lvm/lvm.conf \u0026\u0026 grep -sqo \"udev_rules = 0\" /etc/lvm/lvm.conf \u0026\u0026 grep -sqo \"obtain_device_list_from_udev = 0\" /etc/lvm/lvm.conf \u0026\u0026 rm -rf /etc/{selinux,systemd,udev} /lib/{lsb,udev} /tmp/* /usr/lib{,64}/{locale,udev,dracut} /usr/share/{doc,info,locale,man} /usr/share/{bash-completion,pkgconfig/bash-completion.pc} /var/log/* /var/tmp/* \u0026\u0026 find / -xdev -name \"*.pyc\" -o -name \"*.pyo\" -exec rm -f {} \\; \u0026\u0026 if [ -f /usr/bin/ceph-dencoder ]; then gzip -9 /usr/bin/ceph-dencoder; fi \u0026\u0026 rm -f /usr/lib/ceph/mgr/dashboard/static/AdminLTE-*/plugins/datatables/extensions/TableTools/images/psd/* \u0026\u0026 find /var/log/ -type f -exec truncate -s 0 {} \\; \u0026\u0026 FINAL_SIZE=\"$(bash -c 'sz=\"$(du -sm --exclude=/proc /)\" ; echo \"${sz%*/}\"')\" \u0026\u0026 REMOVED_SIZE=$((INITIAL_SIZE - FINAL_SIZE)) \u0026\u0026 echo \"Cleaning process removed ${REMOVED_SIZE}MB\" \u0026\u0026 echo \"Dropped container size from ${INITIAL_SIZE}MB to ${FINAL_SIZE}MB\" \u0026\u0026 rpm -q ca-certificates e2fsprogs ceph-common-16.2.7 ceph-mon-16.2.7 ceph-osd-16.2.7 ceph-mds-16.2.7 cephfs-mirror-16.2.7 rbd-mirror-16.2.7 ceph-mgr-16.2.7 ceph-mgr-cephadm-16.2.7 ceph-mgr-dashboard-16.2.7 ceph-mgr-diskprediction-local-16.2.7 ceph-mgr-k8sevents-16.2.7 ceph-mgr-rook-16.2.7 python3-saml ceph-grafana-dashboards-16.2.7 kmod lvm2 gdisk smartmontools nvme-cli libstoragemgmt systemd-udev procps-ng hostname ceph-radosgw-16.2.7 libradosstriper1-16.2.7 nfs-ganesha nfs-ganesha-ceph nfs-ganesha-rgw nfs-ganesha-rados-grace nfs-ganesha-rados-urls tcmu-runner ceph-iscsi python3-rtslib attr ceph-fuse-16.2.7 rbd-nbd-16.2.7 ${CRIMSON_PACKAGES} ceph-immutable-object-cache-16.2.7 python3-scikit-learn \u0026\u0026 echo 'Packages verified successfully'" } ], "os": "linux", "rootfs": { "type": "layers", "diff_ids": [ "sha256:3813924f3fa4d6d1223ec6c5e8bb2f5f9acd1e49d87a9251364ff99ab52b3296", "sha256:c86122b5e4d3e8a0d438442340c75c8e35c343945b1c750b15e53b5efdcbc27d", "sha256:e9b312bba1abbd4bd396d9d543063338470cfcc884ba98ce56bbb7a40a981882", "sha256:5d9ef8a0564d47b299fb4c6f92181d876cf985d392f9426d116366b0bafd603a", "sha256:5232e08c6f588f9455b167705bfc14894a64ef2cf9ce6f72cb38733e97c55223" ] }, "config": { "Cmd": [ "/bin/bash" ], "Env": [ "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "container=oci", "I_AM_IN_A_CONTAINER=1", "CEPH_VERSION=pacific", "CEPH_POINT_RELEASE=-16.2.7", "CEPH_DEVEL=false", "CEPH_REF=pacific", "OSD_FLAVOR=default" ], "Hostname": "303559504bcc", "Image": "sha256:e95726d7b64ea8588dbe82c0fd40224ee3b8fa8c696230ca8ab74ac5622f1209", "Labels": { "CEPH_POINT_RELEASE": "-16.2.7", "GIT_BRANCH": "HEAD", "GIT_CLEAN": "True", "GIT_COMMIT": "c7639748589442d7aeaac988ada4549145ccaa4d", "GIT_REPO": "https://github.com/ceph/ceph-container.git", "RELEASE": "HEAD", "architecture": "x86_64", "build-date": "2022-04-29T11:48:11.055897", "ceph": "True", "com.redhat.build-host": "cpt-1007.osbs.prod.upshift.rdu2.redhat.com", "com.redhat.component": "centos-stream-container", "com.redhat.license_terms": "https://centos.org/legal/licensing-policy/", "description": "CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream", "distribution-scope": "public", "io.buildah.version": "1.19.8", "io.k8s.description": "The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.", "io.k8s.display-name": "CentOS Stream 8", "io.openshift.expose-services": "", "io.openshift.tags": "base centos centos-stream", "maintainer": "Guillaume Abrioux \u003cgabrioux@redhat.com\u003e", "name": "centos-stream", "release": "239.1651231664", "summary": "Provides a CentOS Stream container based on the Red Hat Universal Base Image", "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664", "vcs-ref": "3aadd00326f3dd6cfe65ee31017ab98915fddb56", "vcs-type": "git", "vendor": "Red Hat, Inc.", "version": "8" }, "ArgsEscaped": true } } }, "Results": [ { "Target": "quay.io/ceph/ceph:v16.2.7@sha256:00965b7e88c0cef116e6a47107051a4bfe952139e7b94c6cefd6607cf38a3f0f (redhat 8.5)", "Class": "os-pkgs", "Type": "redhat", "Vulnerabilities": [ { "VulnerabilityID": "CVE-2022-24903", "PkgName": "rsyslog", "InstalledVersion": "8.2102.0-9.el8", "Layer": { "Digest": "sha256:791a909c13dab49beef05668b8bce87327580b71fd9f408be535165ff0e8c425", "DiffID": "sha256:5232e08c6f588f9455b167705bfc14894a64ef2cf9ce6f72cb38733e97c55223" }, "SeveritySource": "redhat", "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2022-24903", "Title": "rsyslog: Heap-based overflow in TCP syslog server", "Description": "Rsyslog is a rocket-fast system for log processing. Modules for TCP syslog reception have a potential heap buffer overflow when octet-counted framing is used. This can result in a segfault or some other malfunction. As of our understanding, this vulnerability can not be used for remote code execution. But there may still be a slight chance for experts to do that. The bug occurs when the octet count is read. While there is a check for the maximum number of octets, digits are written to a heap buffer even when the octet count is over the maximum, This can be used to overrun the memory buffer. However, once the sequence of digits stop, no additional characters can be added to the buffer. In our opinion, this makes remote exploits impossible or at least highly complex. Octet-counted framing is one of two potential framing modes. It is relatively uncommon, but enabled by default on receivers. Modules `imtcp`, `imptcp`, `imgssapi`, and `imhttp` are used for regular syslog message reception. It is best practice not to directly expose them to the public. When this practice is followed, the risk is considerably lower. Module `imdiag` is a diagnostics module primarily intended for testbench runs. We do not expect it to be present on any production installation. Octet-counted framing is not very common. Usually, it needs to be specifically enabled at senders. If users do not need it, they can turn it off for the most important modules. This will mitigate the vulnerability.", "Severity": "HIGH", "CweIDs": [ "CWE-120" ], "CVSS": { "nvd": { "V2Vector": "AV:N/AC:M/Au:N/C:P/I:P/A:P", "V3Vector": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H", "V2Score": 6.8, "V3Score": 8.1 }, "redhat": { "V3Vector": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H", "V3Score": 8.1 } }, "References": [ "https://access.redhat.com/security/cve/CVE-2022-24903", "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-24903", "https://github.com/rsyslog/rsyslog/commit/f211042ecbb472f9d8beb4678a65d272b6f07705", "https://github.com/rsyslog/rsyslog/security/advisories/GHSA-ggw7-xr6h-mmr8", "https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/GMNNXLCU2UORRVSZO24HL4KMVPK5PHVW/", "https://ubuntu.com/security/notices/USN-5404-1" ], "PublishedDate": "2022-05-06T00:15:00Z", "LastModifiedDate": "2022-05-17T14:00:00Z" } ] }, { "Target": "Node.js", "Class": "lang-pkgs", "Type": "node-pkg" }, { "Target": "Python", "Class": "lang-pkgs", "Type": "python-pkg", "Vulnerabilities": [ { "VulnerabilityID": "CVE-2020-13092", "PkgName": "scikit-learn", "PkgPath": "usr/lib64/python3.6/site-packages/scikit_learn-0.19.1-py3.6.egg-info", "InstalledVersion": "0.19.1", "FixedVersion": "0.23.1", "Layer": { "Digest": "sha256:791a909c13dab49beef05668b8bce87327580b71fd9f408be535165ff0e8c425", "DiffID": "sha256:5232e08c6f588f9455b167705bfc14894a64ef2cf9ce6f72cb38733e97c55223" }, "SeveritySource": "nvd", "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2020-13092", "DataSource": { "ID": "osv", "Name": "Python Packaging Advisory Database", "URL": "https://github.com/pypa/advisory-db" }, "Title": "** DISPUTED ** scikit-learn (aka sklearn) through 0.23.0 can unseriali ...", "Description": "** DISPUTED ** scikit-learn (aka sklearn) through 0.23.0 can unserialize and execute commands from an untrusted file that is passed to the joblib.load() function, if __reduce__ makes an os.system call. NOTE: third parties dispute this issue because the joblib.load() function is documented as unsafe and it is the user's responsibility to use the function in a secure manner.", "Severity": "CRITICAL", "CweIDs": [ "CWE-502" ], "CVSS": { "nvd": { "V2Vector": "AV:N/AC:L/Au:N/C:P/I:P/A:P", "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H", "V2Score": 7.5, "V3Score": 9.8 } }, "References": [ "https://github.com/0FuzzingQ/vuln/blob/master/sklearn%20unserialize.md", "https://scikit-learn.org/stable/modules/model_persistence.html#security-maintainability-limitations" ], "PublishedDate": "2020-05-15T19:15:00Z", "LastModifiedDate": "2020-05-19T19:05:00Z" }, { "VulnerabilityID": "CVE-2020-28975", "PkgName": "scikit-learn", "PkgPath": "usr/lib64/python3.6/site-packages/scikit_learn-0.19.1-py3.6.egg-info", "InstalledVersion": "0.19.1", "FixedVersion": "0.24.dev0", "Layer": { "Digest": "sha256:791a909c13dab49beef05668b8bce87327580b71fd9f408be535165ff0e8c425", "DiffID": "sha256:5232e08c6f588f9455b167705bfc14894a64ef2cf9ce6f72cb38733e97c55223" }, "SeveritySource": "nvd", "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2020-28975", "DataSource": { "ID": "osv", "Name": "Python Packaging Advisory Database", "URL": "https://github.com/pypa/advisory-db" }, "Description": "** DISPUTED ** svm_predict_values in svm.cpp in Libsvm v324, as used in scikit-learn 0.23.2 and other products, allows attackers to cause a denial of service (segmentation fault) via a crafted model SVM (introduced via pickle, json, or any other model permanence standard) with a large value in the _n_support array. NOTE: the scikit-learn vendor's position is that the behavior can only occur if the library's API is violated by an application that changes a private attribute.", "Severity": "HIGH", "CVSS": { "nvd": { "V2Vector": "AV:N/AC:L/Au:N/C:N/I:N/A:P", "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H", "V2Score": 5, "V3Score": 7.5 } }, "References": [ "http://packetstormsecurity.com/files/160281/SciKit-Learn-0.23.2-Denial-Of-Service.html", "http://seclists.org/fulldisclosure/2020/Nov/44", "https://github.com/cjlin1/libsvm/blob/9a3a9708926dec87d382c43b203f2ca19c2d56a0/svm.cpp#L2501", "https://github.com/scikit-learn/scikit-learn/issues/18891" ], "PublishedDate": "2020-11-21T21:15:00Z", "LastModifiedDate": "2020-12-03T18:50:00Z" } ] } ] } ```