search

kubernetes / client-go

Go client for Kubernetes.
Apache License 2.0
8.78k stars 2.9k forks source link

List function consuming high chunk of memory #1272

Closed kusolank closed 2 weeks ago

kusolank commented 1 year ago

k8sClientSet.CoreV1().Pods("").List(context.Background(), metav1.ListOptions{})

and NewFilteredEventInformer.ListFunc is consuming high chunk of memory when there are large number of pods and pod churn in cluster.

Using them to get resources in every minute same issue was reported on #871

alexandnpu commented 7 months ago

same issue here, below is my heap profile and my client-go version is v0.25.3.

and my use case here is that prometheus uses k8s client to do service discovery

image
k8s-triage-robot commented 3 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

gnuletik commented 2 months ago

Same issue here:

(pprof) top
Showing nodes accounting for 91.72GB, 99.67% of 92.03GB total
Dropped 584 nodes (cum <= 0.46GB)
Showing top 10 nodes out of 15
      flat  flat%   sum%        cum   cum%
   91.72GB 99.67% 99.67%    91.91GB 99.87%  io.ReadAll
         0     0% 99.67%    91.91GB 99.87%  k8s.io/client-go/informers/core/v1.NewFilteredPodInformer.func1
         0     0% 99.67%    91.91GB 99.87%  k8s.io/client-go/kubernetes/typed/core/v1.(*pods).List
         0     0% 99.67%    91.91GB 99.88%  k8s.io/client-go/rest.(*Request).Do
         0     0% 99.67%    91.91GB 99.87%  k8s.io/client-go/rest.(*Request).Do.func1
         0     0% 99.67%    91.91GB 99.88%  k8s.io/client-go/rest.(*Request).request
         0     0% 99.67%    91.91GB 99.87%  k8s.io/client-go/rest.(*Request).request.func3
         0     0% 99.67%    91.91GB 99.87%  k8s.io/client-go/rest.(*Request).request.func3.1 (inline)
         0     0% 99.67%    91.91GB 99.87%  k8s.io/client-go/rest.(*Request).transformResponse
         0     0% 99.67%    91.91GB 99.87%  k8s.io/client-go/tools/cache.(*ListWatch).List
typ431127 commented 2 months ago

+1 I have the same problem

gnuletik commented 2 months ago

I was able to mitigate this issue by using grpc as content type.

kuldeepsolanki04 commented 2 months ago

@gnuletik Can you please help with the example?

gnuletik commented 2 months ago

You can set the ContentType value of the config you provide when creating the clientset.

kubeConfig.ContentType = "application/vnd.kubernetes.protobuf"

// creates the clientset
kubeClient, err := kubernetes.NewForConfig(kubeConfig)
if err != nil {
        return nil, err
}
kuldeepsolanki04 commented 2 months ago

@gnuletik Thanks, let me try this as well

k8s-triage-robot commented 1 month ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 2 weeks ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 2 weeks ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes/client-go/issues/1272#issuecomment-2173081242): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.