k8s scan fails to retrieve container image information

[mariuskimmina]

Describe the bug I'm seeing the following error when running cnquery scan k8s

Asset: kube-system/kube-apiserver-minikube
------------------------------------------

Retrieve Pod Security Context:
k8s.pod: {
  containers: [
    0: {
      securityContext: {}
    }
  ]
  ephemeralContainers: []
  initContainers: []
}

Retrieve Pod information:
k8s.pod: k8s.pod namespace="kube-system" name="kube-apiserver-minikube" created=2023-10-07 08:23:25 +0200 CEST

Retrieve container image information:
error: 1 error occurred:
        * 1 error occurred:
        * rpc error: code = Unknown desc = cannot find connection type k8s
k8s.pod: {}

This is only 1 example error, I got one of these for every asset.

To Reproduce Setup a k8s cluster, in my case it was a fresh install of minikube and run cnquery scan k8s

Desktop (please complete the following information):

• Minikube: Kubernetes v1.27.4 on Docker 24.0.4

[czunker]

This could be related to an issue I'm currently looking at for the gcp snapshot scan:

FTL failed to run query error="provider type does not match"

In both cases we switch the provider. In this issue from k8s to os. And for the gcp snapshot from gcp to os.

[czunker]

@mariuskimmina Could you please try, wether https://github.com/mondoohq/cnquery/pull/2127 solves your issue?

[mariuskimmina]

@mariuskimmina Could you please try, wether #2127 solves your issue?

No, I'm still seeing the error

[czunker]

Strange thing is, this works for run:

cnquery run k8s --namespaces cert-manager --discover container-images -c "container.image{ * }"                                                                                                                                               
! using builtin provider for k8s
→ no Mondoo configuration file provided, using defaults
! using builtin provider for os
! using builtin provider for os
! using builtin provider for os
! using builtin provider for k8s
container.image: {
  identifier: "sha256:c5644d09c6cfce8059f6b8979fb43f14ca326921a87b571a62ce9ee6dcdf014c"
  identifierType: "digest"
  reference: "quay.io/jetstack/cert-manager-webhook@sha256:c5644d09c6cfce8059f6b8979fb43f14ca326921a87b571a62ce9ee6dcdf014c"
  name: "quay.io/jetstack/cert-manager-webhook@sha256:c5644d09c6cfce8059f6b8979fb43f14ca326921a87b571a62ce9ee6dcdf014c"
  repository: container.repository id = quay.io/jetstack/cert-manager-webhook
}
....

[czunker]

This works, when you add the container-images to the discovery options:

cnquery scan k8s --namespaces cert-manager --discover clusters,deployments,container-images                                                                                                                                                      ✔ │ 13s │ 14:44:36 
→ loaded configuration from /etc/opt/mondoo/mondoo.yml using source default
→ using service account credentials
→ discover related assets for 1 asset(s)
→ synchronize assets

 cert-manager                                          ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
 cert-manager/cert-manager                             ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
 cert-manager/cert-manager-cainjector                  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
 cert-manager/cert-manager-webhook                     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
 quay.io/jetstack/cert-manager-controller@2642e7f41545 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
 quay.io/jetstack/cert-manager-cainjector@da7e239ee264 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
 quay.io/jetstack/cert-manager-webhook@a3205d026246    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%

 7/7 scanned                                           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%

Data (7 assets)
===============
...
Asset: quay.io/jetstack/cert-manager-controller@2642e7f41545
------------------------------------------------------------

asset-overview-assetversion:
asset.version: "11.7"

asset-overview-assetarch:
asset.arch: "amd64"

asset-overview-assettitle:
asset.title: "Distroless, Docker Image"

Client version:
mondoo.version: "9.0.2"

Cloud:
if: "Unknown"
...

Discovery of container-images is not enabled by default, because it can take a long time to fetch all the images in a large cluster.

[mariuskimmina]

Okay, makes sense, I'll verify later that it works for me with the --discover option but I think we should probably not show it at all then instead of showing it with errors

[mariuskimmina]

I can confirm that the error is gone when adding --discover

[mariuskimmina]

The error also doesn't occur if you use cnquery scan k8s --discover clusters. So using --discover with any option will work fine. Only if we run cnquery scan k8s without any discover argument I see the error

[czunker]

The error slightly changed in the meantime:

Retrieve container information:
error: 1 error occurred:
    * rpc error: code = Unknown desc = incorrect provider for asset, not adding
k8s.replicaset.containers: []

The error also vanishes when we run it with --discover clusters because it then does not fetch the k8s objects, which include the containers.

But it still fails, when I execute this command:

cnquery scan k8s --discover deployments,container-images
....
Asset: kube-system/coredns
--------------------------

asset-overview-assettitle:
asset.title: "Kubernetes Deployment, Kubernetes Cluster"

Mondoo client version:
mondoo.version: "9.5.1"

Retrieve container information:
error: 1 error occurred:
    * rpc error: code = Unknown desc = incorrect provider for asset, not adding
k8s.deployment.containers: []

Retrieve deployment information:
k8s.deployments: [
  0: k8s.deployment namespace="kube-system" name="coredns" created=2023-11-05 17:18:49 +0100 CET
]

Asset: registry.k8s.io/coredns/coredns@be7652ce0b43
---------------------------------------------------

asset-overview-assettitle:
asset.title: "Docker Image"

Mondoo client version:
mondoo.version: "9.5.1"

...

It fetches the image, but does not provide the information to the deployment.

[imilchev]

this should no longer be an issue