Deployed Kubecost and its in container creating state

[nidiculageorge]

Kubecost Version

1.106.4

Kubernetes Version

1.25.11

Kubernetes Platform

AKS

Description

We deployed kubecost using the following helm command

helm install kubecost cost-analyzer --repo https://kubecost.github.io/cost-analyzer/ --namespace kubecost --create-namespace --set kubecostToken="bmlkaWN1bGFnZW9yZ2VAaW4ucm0uY29txm343yadf98"

As per the following link

https://www.kubecost.com/install#show-instructions

Steps to reproduce

1. Install kubecost using the following link - https://www.kubecost.com/install#show-instructions
2. Command - helm install kubecost cost-analyzer --repo https://kubecost.github.io/cost-analyzer/ --namespace kubecost --create-namespace --set kubecostToken="bmlkaWN1bGFnZW9yZ2VAaW4ucm0uY29txm343yadf98"

Expected behavior

The container should be running but the pods are in container creating state

PFB below

Impact

The pods are not in running state ,its in container creating state and unable to access the kubecost

Screenshots

Logs

PS /home/nithin> kubectl logs kubecost-cost-analyzer-7596f84b9d-tv5w2 -n kubecost                     
Defaulted container "cost-model" out of: cost-model, cost-analyzer-frontend
Error from server (BadRequest): container "cost-model" in pod "kubecost-cost-analyzer-7596f84b9d-tv5w2" is waiting to start: ContainerCreating

Slack discussion

No response

Troubleshooting

• [X] I have read and followed the issue guidelines and this is a bug impacting only the Kubecost application.
• [X] I have searched other issues in this repository and mine is not recorded.

[nidiculageorge]

Is these pods required to run in Linux Nodes,as our cluster has windows nodes we can see the container creating pods are getting scheduled in windows Nodes

---

Normal Scheduled 36m default-scheduler Successfully assigned kubecost/kubecost-kube-state-metrics-68459c8d5f-5wx2l to aksnpwin000002

Warning FailedMount 36m kubelet MountVolume.SetUp failed for volume "kube-api-access-87hs6" : chown c:\var\lib\kubelet\pods\c995c7e0-b23e-4554-98bf-0d081cc938f9\volumes\kubernetes.io~projected\kube-api-access-87hs6..2023_11_01_02_55_46.4289131879\token: not supported by windows

[AjayTripathy]

Yes, we recommend setting your taints/tolerations to run kubecost on linux nodes.

[nidiculageorge]

@AjayTripathy Currently the linux nodepool dont have any taints added as other pods are running in the same nodepool ,which will have an effect

Is there a way other than adding taints and toleeration to the nodepool. we used the below commands also to schedule the pods in Linux Nodes

helm install kubecost cost-analyzer --repo https://kubecost.github.io/cost-analyzer/ --namespace kubecost --create-namespace --set kubecostToken="bmlkaWN1bGFnZW9yZ2VAaW4ucm0uY29txm343yadf98" --set nodeSelector."kubernetes.io/os"=linux

But only the kubecostanalyser pod is getting schduled there,rest are getting schduled in windows nodes

[AjayTripathy]

You should similarly be able to set nodeselectors for the other pods by referencing the chart subvariables, etc --set prometheus.nodeSelector."kubernetes.io/os"=linux

[nidiculageorge]

@AjayTripathy is this you are mentioning on how we set for each pods

The command is below when i run getting the below error

helm install kubecost cost-analyzer --repo https://kubecost.github.io/cost-analyzer/ --namespace kubecost --create-namespace --set kubecostToken="bmlkaWN1bGFnZW9yZ2VAaW4ucm0uY29txm343yadf98" --set prometheus.nodeSelector."kubernetes.io/os"=linux --set cost-analyser.nodeSelector."kubernetes.io/os"=linux --set grafana.nodeSelector."kubernetes.io/os"=linux --set kube-state-metrics.nodeSelector."kubernetes.io/os"=linux

Error: unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(Deployment.spec.template.spec.nodeSelector.kubernetes): invalid type for io.k8s.api.core.v1.PodSpec.nodeSelector: got "map", expected "string"

[nidiculageorge]

You should similarly be able to set nodeselectors for the other pods by referencing the chart subvariables, etc --set prometheus.nodeSelector."kubernetes.io/os"=linux

@AjayTripathy is this you are mentioning on how we set for each pods

The command is below when i run getting the below error

helm install kubecost cost-analyzer --repo https://kubecost.github.io/cost-analyzer/ --namespace kubecost --create-namespace --set kubecostToken="bmlkaWN1bGFnZW9yZ2VAaW4ucm0uY29txm343yadf98" --set prometheus.nodeSelector."kubernetes.io/os"=linux --set cost-analyser.nodeSelector."kubernetes.io/os"=linux --set grafana.nodeSelector."kubernetes.io/os"=linux --set kube-state-metrics.nodeSelector."kubernetes.io/os"=linux

Error: unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(Deployment.spec.template.spec.nodeSelector.kubernetes): invalid type for io.k8s.api.core.v1.PodSpec.nodeSelector: got "map", expected "string"

[AjayTripathy]

@nidiculageorge more detailed instructions on how to do this available here: https://docs.kubecost.com/install-and-configure/advanced-configuration/windows-node-support