Kubernetes KubeConfig ConfigException[214]: Service host/port is not set.

[mirisbowring]

Steps to Reproduce

For testing the kubernetes provider, i've setup a Kubernetes on a MacBook and exposed it to the local network. I am trying to scan the cluster from a different linux based device.

I am able to execute kubectl --kubeconfig kube.conf get pods to test cluster access and permissions.

Now, if i execute: docker run --rm toniblyx/prowler:stable kubernetes --kubeconfig-file $pwd/kube.conf I get the following error: CRITICAL: ConfigException[214]: Service host/port is not set.

Expected behavior

I would expect the scan to execute since connection to the cluster is obviously possible.

Actual Result with Screenshots or Logs

2024-10-21 14:52:35,277 [File: provider.py:228]     [Module: provider]   CRITICAL: ConfigException[214]: Service host/port is not set.
                         _
 _ __  _ __ _____      _| | ___ _ __
| '_ \| '__/ _ \ \ /\ / / |/ _ \ '__|
| |_) | | | (_) \ V  V /| |  __/ |
| .__/|_|  \___/ \_/\_/ |_|\___|_|v4.4.1
|_| the handy multi-cloud security tool

Date: 2024-10-21 14:52:34

How did you install Prowler?

Docker (docker pull toniblyx/prowler)

Environment Resource

Prowler is being executed as docker command from an linux host on the same local network

Kubernetes is exposed to the local network via: kubectl proxy --address='0.0.0.0' --port=8080 --disable-filter=true (This is just for testing)

OS used

MacOS and Linux

Prowler version

4.4.1

Pip version

the one bundled in the latest stable docker image

Context

This is the working kube.config

apiVersion: v1
clusters:
- cluster:
    server: http://10.100.3.38:8080
  name: docker-desktop
contexts:
- context:
    cluster: docker-desktop
    user: docker-desktop
  name: docker-desktop
current-context: docker-desktop
kind: Config
preferences: {}
users:
- name: prowler-service-acc
  user:
    token: eyJhbGciOiJSUzI1NiIsImtpZCI6IlhBMHBI...

[jfagoagas]

Hello @mirisbowring, as per the command you executed I see that you are not mapping your local filesystem to the container , thus Prowler is not being able to get the content of $pwd/kube.conf. You will need to use the mount volume -v flag https://docs.docker.com/reference/cli/docker/container/run/#volume.

Also, you should do a quick test to see if the kubernetes cluster is accessible from the running container.

Please let me know if this helps you fix the issue. Thanks!

[mirisbowring]

Ah you were right, that was the first issue I was able to solve by running:

docker run --rm -v /mnt/folder:/folder toniblyx/prowler:stable kubernetes --kubeconfig-file /folder/kube.conf

But the "real" issue was this:

                         _
 _ __  _ __ _____      _| | ___ _ __
| '_ \| '__/ _ \ \ /\ / / |/ _ \ '__|
| |_) | | | (_) \ V  V /| |  __/ |
| .__/|_|  \___/ \_/\_/ |_|\___|_|v4.4.1
|_| the handy multi-cloud security tool

Date: 2024-10-21 17:22:29

2024-10-21 17:22:29,649 [File: provider.py:228]     [Module: provider]   CRITICAL: ConfigException[214]: Invalid kube-config file. No configuration found.

I also verified that the file is present within the container and that it can be read by setting the correct UID and making it world readable/writable (chmod 777)

[sergargar]

Hi @mirisbowring, can you verify if the docker image can reach your Kubernetes cluster? Also, how is your kube config looks like? Do you have the current_context set?

[mirisbowring]

Hi @sergargar I added the used kubeconfig to the issue description. I also installed kubectl into the container and have been able to contact my cluster from there.

I also tried to set the --context and --namespace option for prowler kubernetes but it did not solve the problem either :)

[mirisbowring]

Ah, an information that is maybe missing: If I execute Prowler directly on my Kubernetes Machine and point to this kubefile, it works.

[mirisbowring]

Seems like it tries to fall back to a local kubeconf. I installed another kubernetes on another host and tried to scan with prowler from my MacOs and get this result:

prowler kubernetes --kubeconfig-file $(pwd)/lab.kube.conf
                         _
 _ __  _ __ _____      _| | ___ _ __
| '_ \| '__/ _ \ \ /\ / / |/ _ \ '__|
| |_) | | | (_) \ V  V /| |  __/ |
| .__/|_|  \___/ \_/\_/ |_|\___|_|v4.4.0
|_| the handy multi-cloud security tool

Date: 2024-10-22 09:01:29

-> Using the Kubernetes credentials below:
  · Kubernetes Cluster: docker-desktop
  · User: docker-desktop
  · Namespaces: big-monolith, default, kube-node-lease, kube-public, kube-system, secure-middleware
  · Roles: No associated Roles

-> Using the following configuration:
  · Config File: /usr/local/Cellar/prowler/4.4.0/libexec/lib/python3.12/site-packages/prowler/config/config.yaml

Executing 83 checks, please wait...
-> Scan completed! |▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉| 83/83 [100%] in 3.4s 

Though the lab.kube.config points to a completely different cluster. Instead, my local test cluster on the macbook has been scanned (that cannot be scanned from a remote host - see previous messages).

[mirisbowring]

I was able to track down the problem!

The issue seems to be with the --kubeconfig-file option. As i discovered in the last message, the local file is picked up instead. So i went ahead and replaced the ~/.kube/configwith the one from the remote cluster. After starting the scan (with the --kubeconfig-file option) it worked as expected! The remote context, namespaces, etc. are being used!

After, I retried this with the prowler docker container too. Instead of mounting the config anywhere into the container, i mounted it into /home/prowler/.kube/. It was important to name it config in this case (anything else like kube.conf was not working).

So instead of the No configuration founderror, it picks up the files placed in the .kube folder and is able to scan the remote cluster!

So the bug is not with the kube provider itself but with the way, the --kubeconfig-file option is handled! 😊

[sergargar]

Thanks for all the testing @mirisbowring ! I will take a look at how the --kubeconfig-file option is handled and come back with a solution 😄

[sergargar]

I found the error @mirisbowring, we were not passing the config file as an argument in the list_kube_config_contexts function, therefore always the default config file (~/.kube/config) was used.

[mirisbowring]

That's great! Sounds like a "small" problem and not too complex 😃