Available memroy in kubecost-cost-analyzer PVC is gradually decreasing

[MageshSrinivasulu]

Kubecost Helm Chart Version

2.4.0

Kubernetes Version

1.29

Kubernetes Platform

EKS

Description

Here is the graph that shows the available memory in the disk kubecost-cost-analyzer the trend in gradually trending lower. The sudden spike in availability is because I adjusted the etlDailyStoreDurationDays but it is not helping much still the memory slowly decreasing

is

Steps to reproduce

Deploy the latest helm. Integrate the Kubecost with your billing account and run it for some time. you will see the memory in PVC getting reduced gradually

Expected behavior

memory should be either getting freed up or available memory should not reduce gradually

Impact

Available memory in PVC keeps on getting reduced will soon run out of free space

Screenshots

Logs

No response

Slack discussion

https://kubecost.slack.com/archives/CLFV60Y90/p1728277157577009

Troubleshooting

• [X] I have read and followed the issue guidelines and this is a bug impacting only the Helm chart.
• [X] I have searched other issues in this repository and mine is not recorded.

[AjayTripathy]

@MageshSrinivasulu we'd expect this then to stabilize when you've hit your number of retention days. What's your retention number and how many days are you seeing in the UI now?

[MageshSrinivasulu]

@AjayTripathy I have set the etlDailyStoreDurationDays to 30 from its default configuration of 91 days. Can you please specify the helm configuration you are referring to?

[MageshSrinivasulu]

Below are the folders that have very high memory usage

kubecost-cost-analyzer is attached to the path /var/configs

[MageshSrinivasulu]

I had to clear some older files in the path /var/configs/localBucket/billing_integrations/<AWS-ACCOUNT>-s3:--<S3-BUCKET>/cloud_costs/bingen/1d on the pod to solve the issue temporarily. This will not be a sustainable solution in the long run. How to configure this routine clean up using helm deployment?

[MageshSrinivasulu]

@MageshSrinivasulu we'd expect this then to stabilize when you've hit your number of retention days. What's your retention number and how many days are you seeing in the UI now?

@AjayTripathy Is this the one you are speaking about?

[AjayTripathy]

yes. It will gradually increase until that time window then should stabilize.

[MageshSrinivasulu]

It has been more than 30 days since I made the changes. It keeps decreasing gradually. It is not getting stabilized