search

aquasecurity / starboard

Moved to https://github.com/aquasecurity/trivy-operator
https://aquasecurity.github.io/starboard/
Apache License 2.0
1.36k stars 197 forks source link

Starboard failed to scan ECR images #1252

Open JK-JIA opened 2 years ago

JK-JIA commented 2 years ago

I use starboard-operator to scan for mirroring vulnerabilities in the cluster, but when I encounter mirroring in the ECR repository, fail and report an error

I refer to https://aquasecurity.github.io/starboard/v0.15.7/vulnerability-scanning/managed-registries/ for authorization Here is my starboard-operator serviceAccount yaml

kind: ServiceAccount
metadata:
  name: starboard-operator
  namespace: starboard-system
  annotations:
    eks.amazonaws.com/role-arn: arn:aws-cn:iam::516915001847:role/trivy-ecr-role
  labels:
    app.kubernetes.io/name: starboard-operator
    app.kubernetes.io/instance: starboard-operator
    app.kubernetes.io/version: "0.15.4"
    app.kubernetes.io/managed-by: kubectl

I'm using the client/server Trivy mode ,Because I download databases very slowly in China This is how it works

trivy-server.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: trivy-server-deployment
  namespace: starboard-system
  labels:
    app: trivy-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: trivy-server
  template:
    metadata:
      labels:
        app: trivy-server
    spec:
      serviceAccountName: starboard-operator
      containers:
      - name: trivy-server
        image: docker.io/aquasec/trivy:0.25.2
        ports:
        - containerPort: 4954
        #command: ["trivy"]
        #args: ["server --listen 127.0.0.1:4954"]
        command: ["/bin/sh"]
        args: ["-c","trivy server --listen 0.0.0.0:4954"]
        #args: ["-c", "while true; do echo hello; sleep 10;done"]
        volumeMounts:
        - name: trivy-data
          mountPath: /root/.cache/trivy
          subPath: trivy
      volumes:
      - name: trivy-data
        persistentVolumeClaim:
          claimName: efs-trivy-cliam
---

apiVersion: v1
kind: Service
metadata:
  name: trivy-server
  namespace: starboard-system
spec:
  type: ClusterIP
  selector:
    app: trivy-server
  ports:
    - port: 4954
      targetPort: 4954

starboard-operator.yaml Abstract

apiVersion: v1
kind: ConfigMap
metadata:
  name: starboard-trivy-config
  namespace: starboard-system
  labels:
    app.kubernetes.io/name: starboard-operator
    app.kubernetes.io/instance: starboard-operator
    app.kubernetes.io/version: "0.15.4"
    app.kubernetes.io/managed-by: kubectl
data:
  trivy.imageRef: "docker.io/aquasec/trivy:0.25.2"
#  trivy.mode: "Standalone"
  trivy.mode: "ClientServer"
  trivy.serverURL: "http://trivy-server:4954"
  trivy.severity: "UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL"
  trivy.timeout: "90m0s"
  trivy.dbRepository: "ghcr.io/aquasecurity/trivy-db"
  trivy.resources.requests.cpu: 100m
  trivy.resources.requests.memory: 100M
  trivy.resources.limits.cpu: 1000m
  trivy.resources.limits.memory: 1000M

The following is the error log for the starboard-operator

{"level":"error","ts":1661158616.6022856,"logger":"reconciler.vulnerabilityreport","msg":"Scan job container","job":"starboard-system/scan-vulnerabilityreport-7c87d58f58","container":"jenkins-agent01","status.reason":"Error","status.message":"2022-08-22T08:56:55.119Z\t\u001b[31mFATAL\u001b[0m\tscan error: unable to initialize a scanner: unable to initialize the docker scanner: 3 errors occurred:\n\t* unable to inspect the image (516915001847.dkr.ecr.cn-northwest-1.amazonaws.com.cn/jenkins-slave:v7): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n\t* unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory\n\t* GET https://516915001847.dkr.ecr.cn-northwest-1.amazonaws.com.cn/v2/jenkins-slave/manifests/v7: unexpected status code 401 Unauthorized: Not Authorized\n\n\n\n","stacktrace":"github.com/aquasecurity/starboard/pkg/vulnerabilityreport.(*WorkloadController).reconcileJobs.func1\n\t/home/runner/work/starboard/starboard/pkg/vulnerabilityreport/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile/reconcile.go:102\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1661158617.44241,"logger":"reconciler.vulnerabilityreport","msg":"Scan job container","job":"starboard-system/scan-vulnerabilityreport-84d4d648c6","container":"nginx","status.reason":"Error","status.message":"2022-08-22T08:56:55.496Z\t\u001b[31mFATAL\u001b[0m\tscan error: unable to initialize a scanner: unable to initialize the docker scanner: 3 errors occurred:\n\t* unable to inspect the image (516915001847.dkr.ecr.cn-northwest-1.amazonaws.com.cn/ixtra-frontend:3.3-rc-3): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n\t* unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory\n\t* GET https://516915001847.dkr.ecr.cn-northwest-1.amazonaws.com.cn/v2/ixtra-frontend/manifests/3.3-rc-3: unexpected status code 401 Unauthorized: Not Authorized\n\n\n\n","stacktrace":"github.com/aquasecurity/starboard/pkg/vulnerabilityreport.(*WorkloadController).reconcileJobs.func1\n\t/home/runner/work/starboard/starboard/pkg/vulnerabilityreport/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile/reconcile.go:102\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1661158618.2136166,"logger":"reconciler.vulnerabilityreport","msg":"Scan job container","job":"starboard-system/scan-vulnerabilityreport-5fdc98694","container":"nginx","status.reason":"Error","status.message":"2022-08-22T08:56:55.142Z\t\u001b[31mFATAL\u001b[0m\tscan error: unable to initialize a scanner: unable to initialize the docker scanner: 3 errors occurred:\n\t* unable to inspect the image (516915001847.dkr.ecr.cn-northwest-1.amazonaws.com.cn/ixtra-frontend:3.3-rc-3): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n\t* unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory\n\t* GET https://516915001847.dkr.ecr.cn-northwest-1.amazonaws.com.cn/v2/ixtra-frontend/manifests/3.3-rc-3: unexpected status code 401 Unauthorized: Not Authorized\n\n\n\n","stacktrace":"github.com/aquasecurity/starboard/pkg/vulnerabilityreport.(*WorkloadController).reconcileJobs.func1\n\t/home/runner/work/starboard/starboard/pkg/vulnerabilityreport/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile/reconcile.go:102\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1661158619.0005393,"logger":"reconciler.vulnerabilityreport","msg":"Scan job container","job":"starboard-system/scan-vulnerabilityreport-5f58c95b49","container":"nginx","status.reason":"Error","status.message":"2022-08-22T08:56:55.952Z\t\u001b[31mFATAL\u001b[0m\tscan error: unable to initialize a scanner: unable to initialize the docker scanner: 3 errors occurred:\n\t* unable to inspect the image (516915001847.dkr.ecr.cn-northwest-1.amazonaws.com.cn/ixtra-frontend:3.4.5): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n\t* unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory\n\t* GET https://516915001847.dkr.ecr.cn-northwest-1.amazonaws.com.cn/v2/ixtra-frontend/manifests/3.4.5: unexpected status code 401 Unauthorized: Not Authorized\n\n\n\n","stacktrace":"github.com/aquasecurity/starboard/pkg/vulnerabilityreport.(*WorkloadController).reconcileJobs.func1\n\t/home/runner/work/starboard/starboard/pkg/vulnerabilityreport/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile/reconcile.go:102\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1661158622.0122895,"logger":"reconciler.vulnerabilityreport","msg":"Scan job container","job":"starboard-system/scan-vulnerabilityreport-fd959ddfb","container":"kubernetes-dashboard","status.reason":"Error","status.message":"2022-08-22T08:56:58.253Z\t\u001b[31mFATAL\u001b[0m\tscan error: unable to initialize a scanner: unable to initialize the docker scanner: 3 errors occurred:\n\t* unable to inspect the image (kubernetesui/dashboard:v2.5.1): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n\t* unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory\n\t* GET https://index.docker.io/v2/kubernetesui/dashboard/manifests/sha256:6614c53fcdb9df9cb920c701c6a418e398be9b5ee147e5231ad6669fd2b76862: TOOMANYREQUESTS: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit\n\n\n","stacktrace":"github.com/aquasecurity/starboard/pkg/vulnerabilityreport.(*WorkloadController).reconcileJobs.func1\n\t/home/runner/work/starboard/starboard/pkg/vulnerabilityreport/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile/reconcile.go:102\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1661158622.4091635,"logger":"reconciler.vulnerabilityreport","msg":"Scan job container","job":"starboard-system/scan-vulnerabilityreport-6d459d7c7c","container":"dashboards","status.reason":"Error","status.message":"2022-08-22T08:56:58.310Z\t\u001b[31mFATAL\u001b[0m\tscan error: unable to initialize a scanner: unable to initialize the docker scanner: 3 errors occurred:\n\t* unable to inspect the image (docker.io/opensearchproject/opensearch-dashboards:2.0.0): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n\t* unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory\n\t* GET https://index.docker.io/v2/opensearchproject/opensearch-dashboards/manifests/sha256:fda49bc2f3f3317d58d63fbcbcfb7ad1fcd7958dc528941511d2dcf2da078b72: TOOMANYREQUESTS: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit\n\n\n","stacktrace":"github.com/aquasecurity/starboard/pkg/vulnerabilityreport.(*WorkloadController).reconcileJobs.func1\n\t/home/runner/work/starboard/starboard/pkg/vulnerabilityreport/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile/reconcile.go:102\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1661158630.1363208,"logger":"reconciler.vulnerabilityreport","msg":"Scan job container","job":"starboard-system/scan-vulnerabilityreport-855dd745b7","container":"module-configmap-reloader","status.reason":"Error","status.message":"2022-08-22T08:56:58.145Z\t\u001b[31mFATAL\u001b[0m\tscan error: unable to initialize a scanner: unable to initialize the docker scanner: 3 errors occurred:\n\t* unable to inspect the image (jimmidyson/configmap-reload:v0.5.0): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n\t* unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory\n\t* GET https://index.docker.io/v2/jimmidyson/configmap-reload/manifests/v0.5.0: TOOMANYREQUESTS: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit\n\n\n","stacktrace":"github.com/aquasecurity/starboard/pkg/vulnerabilityreport.(*WorkloadController).reconcileJobs.func1\n\t/home/runner/work/starboard/starboard/pkg/vulnerabilityreport/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile/reconcile.go:102\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:227"}
{"level":"error","ts":1661158646.4274912,"logger":"reconciler.vulnerabilityreport","msg":"Scan job container","job":"starboard-system/scan-vulnerabilityreport-79667547d8","container":"kube-rbac-proxy","status.reason":"Error","status.message":"2022-08-22T08:57:25.281Z\t\u001b[31mFATAL\u001b[0m\tscan error: unable to initialize a scanner: unable to initialize the docker scanner: 3 errors occurred:\n\t* unable to inspect the image (gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n\t* unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory\n\t* Get \"https://gcr.io/v2/\": dial tcp 64.233.188.82:443: i/o timeout\n\n\n","stacktrace":"github.com/aquasecurity/starboard/pkg/vulnerabilityreport.(*WorkloadController).reconcileJobs.func1\n\t/home/runner/work/starboard/starboard/pkg/vulnerabilityreport/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile/reconcile.go:102\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:227"}
.......

Please help me, any reply will be helpful to me, thank you😀

skymoore commented 1 year ago

@JK-JIA for future reference, it's a good idea to redact the aws account id.