kubernetes-sigs / prometheus-adapter

An implementation of the custom.metrics.k8s.io API using Prometheus
Apache License 2.0
1.9k stars 551 forks source link

unable to get node metrics #610

Open vijaychandranimmakuri opened 11 months ago

vijaychandranimmakuri commented 11 months ago

Hi Team, I was cloned repo and installed on from master Repo. When I use Kubectl top pods able to get metrics like CPU and Memory usages But, When I use Kubectl top nodes unable to scrap node metrics even through adapter pod not throwing any error.

when i use kubectl top node it has to show.

I used default adapter configuration which was in the GitHub repo

Environment:

dashpole commented 11 months ago

/assign @dgrisonnet /triage accepted

dgrisonnet commented 10 months ago

Hi @vijaychandranimmakuri could you perhaps share the logs of prometheus-adapter?

pasztorl commented 7 months ago

Hi!

Just installed the chart version 4.9.0 and metrics for pods works but for node not. Here is my log:

I0125 15:52:51.763849       1 adapter.go:114] successfully using in-cluster auth
I0125 15:52:52.542521       1 serving.go:342] Generated self-signed cert (/tmp/cert/apiserver.crt, /tmp/cert/apiserver.key)
I0125 15:52:52.543609       1 dynamic_serving_content.go:113] "Loaded a new cert/key pair" name="serving-cert::/tmp/cert/apiserver.crt::/tmp/cert/apiserver.key"
I0125 15:52:53.570927       1 requestheader_controller.go:244] Loaded a new request header values for RequestHeaderAuthRequestController
I0125 15:52:53.573181       1 maxinflight.go:140] "Initialized nonMutatingChan" len=400
I0125 15:52:53.573361       1 maxinflight.go:146] "Initialized mutatingChan" len=200
I0125 15:52:53.573525       1 timing_ratio_histogram.go:203] "TimingRatioHistogramVec.NewForLabelValuesSafe hit the inefficient case" fqName="apiserver_flowcontrol_read_vs_write_current_requests" labelValues=[executing readOnly]
I0125 15:52:53.573737       1 timing_ratio_histogram.go:203] "TimingRatioHistogramVec.NewForLabelValuesSafe hit the inefficient case" fqName="apiserver_flowcontrol_read_vs_write_current_requests" labelValues=[executing mutating]
I0125 15:52:53.573859       1 maxinflight.go:117] "Set denominator for readonly requests" limit=400
I0125 15:52:53.573978       1 maxinflight.go:121] "Set denominator for mutating requests" limit=200
I0125 15:52:53.574594       1 config.go:814] Not requested to run hook priority-and-fairness-config-consumer
I0125 15:52:53.581911       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager
I0125 15:52:53.582381       1 reflector.go:287] Starting reflector *v1.PartialObjectMetadata (0s) from pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231
I0125 15:52:53.582397       1 reflector.go:323] Listing and watching *v1.PartialObjectMetadata from pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231
I0125 15:52:53.612004       1 genericapiserver.go:524] MuxAndDiscoveryComplete has all endpoints registered and discovery information is complete
I0125 15:52:53.615678       1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController
I0125 15:52:53.615781       1 shared_informer.go:311] Waiting for caches to sync for RequestHeaderAuthRequestController
I0125 15:52:53.615871       1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"
I0125 15:52:53.615923       1 shared_informer.go:311] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0125 15:52:53.615883       1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
I0125 15:52:53.616067       1 shared_informer.go:311] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0125 15:52:53.616020       1 reflector.go:287] Starting reflector *v1.ConfigMap (12h0m0s) from pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231
I0125 15:52:53.616245       1 reflector.go:323] Listing and watching *v1.ConfigMap from pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231
I0125 15:52:53.616058       1 reflector.go:287] Starting reflector *v1.ConfigMap (12h0m0s) from pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231
I0125 15:52:53.616937       1 reflector.go:323] Listing and watching *v1.ConfigMap from pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231
I0125 15:52:53.617331       1 reflector.go:287] Starting reflector *v1.ConfigMap (12h0m0s) from pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231
I0125 15:52:53.617427       1 reflector.go:323] Listing and watching *v1.ConfigMap from pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231
I0125 15:52:53.617765       1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/tmp/cert/apiserver.crt::/tmp/cert/apiserver.key"
I0125 15:52:53.623033       1 tlsconfig.go:200] "Loaded serving cert" certName="serving-cert::/tmp/cert/apiserver.crt::/tmp/cert/apiserver.key" certDetail="\"localhost@1706197972\" [serving] validServingFor=[127.0.0.1,localhost,localhost] issuer=\"localhost-ca@1706197972\" (2024-01-25 14:52:51 +0000 UTC to 2025-01-24 14:52:51 +0000 UTC (now=2024-01-25 15:52:53.622936043 +0000 UTC))"
I0125 15:52:53.625440       1 named_certificates.go:53] "Loaded SNI cert" index=0 certName="self-signed loopback" certDetail="\"apiserver-loopback-client@1706197973\" [serving] validServingFor=[apiserver-loopback-client] issuer=\"apiserver-loopback-client-ca@1706197972\" (2024-01-25 14:52:52 +0000 UTC to 2025-01-24 14:52:52 +0000 UTC (now=2024-01-25 15:52:53.625386613 +0000 UTC))"
I0125 15:52:53.625476       1 secure_serving.go:210] Serving securely on [::]:6443
I0125 15:52:53.625537       1 genericapiserver.go:672] [graceful-termination] waiting for shutdown to be initiated
I0125 15:52:53.625573       1 tlsconfig.go:240] "Starting DynamicServingCertificateController"
I0125 15:52:53.625698       1 reflector.go:287] Starting reflector *v1.Node (0s) from pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231
I0125 15:52:53.625710       1 reflector.go:323] Listing and watching *v1.Node from pkg/mod/k8s.io/client-go@v0.27.2/tools/cache/reflector.go:231
I0125 15:52:53.716969       1 shared_informer.go:341] caches populated
I0125 15:52:53.717056       1 shared_informer.go:318] Caches are synced for RequestHeaderAuthRequestController
I0125 15:52:53.716970       1 shared_informer.go:341] caches populated
I0125 15:52:53.717136       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0125 15:52:53.716978       1 shared_informer.go:341] caches populated
I0125 15:52:53.717157       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0125 15:52:53.717586       1 tlsconfig.go:178] "Loaded client CA" index=0 certName="client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" certDetail="\"Wonderline sktrend.net K8S CA\" [] groups=[Wonderline Europe zRt.] issuer=\"SK Trend Kft. sktrend.net ROOT CA\" (2024-01-07 19:26:27 +0000 UTC to 2034-01-04 19:26:27 +0000 UTC (now=2024-01-25 15:52:53.717557624 +0000 UTC))"
I0125 15:52:53.717656       1 tlsconfig.go:178] "Loaded client CA" index=1 certName="client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" certDetail="\"Wonderline sktrend.net K8S CA\" [] groups=[Wonderline Europe zRt.] issuer=\"SK Trend Kft. sktrend.net ROOT CA\" (2024-01-07 19:46:53 +0000 UTC to 2034-01-04 19:46:53 +0000 UTC (now=2024-01-25 15:52:53.717603783 +0000 UTC))"
I0125 15:52:53.717706       1 tlsconfig.go:178] "Loaded client CA" index=2 certName="client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" certDetail="\"sktrend.net K8S CA\" [] groups=[SK Trend Kereskedelmi es Szolgaltato Informatikai Kft.] issuer=\"SK Trend Kft. sktrend.net ROOT CA\" (2024-01-10 18:21:46 +0000 UTC to 2034-01-07 18:21:46 +0000 UTC (now=2024-01-25 15:52:53.717686405 +0000 UTC))"
I0125 15:52:53.717745       1 tlsconfig.go:178] "Loaded client CA" index=3 certName="client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" certDetail="\"front-proxy-ca\" [] validServingFor=[front-proxy-ca] issuer=\"<self>\" (2024-01-07 19:22:14 +0000 UTC to 2034-01-04 19:27:14 +0000 UTC (now=2024-01-25 15:52:53.717722006 +0000 UTC))"
I0125 15:52:53.717778       1 tlsconfig.go:178] "Loaded client CA" index=4 certName="client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" certDetail="\"front-proxy-ca\" [] validServingFor=[front-proxy-ca] issuer=\"<self>\" (2024-01-10 18:18:22 +0000 UTC to 2034-01-07 18:23:22 +0000 UTC (now=2024-01-25 15:52:53.717761016 +0000 UTC))"
I0125 15:52:53.717804       1 tlsconfig.go:178] "Loaded client CA" index=5 certName="client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" certDetail="\"front-proxy-ca\" [] validServingFor=[front-proxy-ca] issuer=\"<self>\" (2024-01-10 19:51:02 +0000 UTC to 2034-01-07 19:56:02 +0000 UTC (now=2024-01-25 15:52:53.71778919 +0000 UTC))"
I0125 15:52:53.717833       1 tlsconfig.go:178] "Loaded client CA" index=6 certName="client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" certDetail="\"front-proxy-ca\" [] validServingFor=[front-proxy-ca] issuer=\"<self>\" (2024-01-11 04:52:16 +0000 UTC to 2034-01-08 04:57:16 +0000 UTC (now=2024-01-25 15:52:53.717814263 +0000 UTC))"
I0125 15:52:53.718481       1 tlsconfig.go:200] "Loaded serving cert" certName="serving-cert::/tmp/cert/apiserver.crt::/tmp/cert/apiserver.key" certDetail="\"localhost@1706197972\" [serving] validServingFor=[127.0.0.1,localhost,localhost] issuer=\"localhost-ca@1706197972\" (2024-01-25 14:52:51 +0000 UTC to 2025-01-24 14:52:51 +0000 UTC (now=2024-01-25 15:52:53.718459142 +0000 UTC))"
I0125 15:52:53.719189       1 named_certificates.go:53] "Loaded SNI cert" index=0 certName="self-signed loopback" certDetail="\"apiserver-loopback-client@1706197973\" [serving] validServingFor=[apiserver-loopback-client] issuer=\"apiserver-loopback-client-ca@1706197972\" (2024-01-25 14:52:52 +0000 UTC to 2025-01-24 14:52:52 +0000 UTC (now=2024-01-25 15:52:53.719158438 +0000 UTC))"
I0125 15:53:27.293055       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="380.656µs" userAgent="kube-probe/1.27" audit-ID="2ffc6cdb-ebd5-4d37-b531-66ee0acd7b18" srcIP="172.31.0.147:39022" resp=200
I0125 15:53:27.312369       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="273.619µs" userAgent="kube-probe/1.27" audit-ID="0e1cc929-e935-48b3-8833-d443c4264e96" srcIP="172.31.0.147:39012" resp=200
I0125 15:53:27.509106       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="10.044416ms" userAgent="Go-http-client/2.0" audit-ID="52f79340-e556-4461-944b-f259ef529f2c" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:27.514622       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.285338ms" userAgent="Go-http-client/2.0" audit-ID="60ef946a-220b-4b84-8a69-a4ddec4659d3" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:27.515109       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="18.258994ms" userAgent="Go-http-client/2.0" audit-ID="790bf25f-9393-454f-9b8b-4e70acb152c5" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:27.517922       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.754812ms" userAgent="Go-http-client/2.0" audit-ID="82b8d206-578f-41d2-af1e-018985e5c1aa" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:27.528296       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="11.802893ms" userAgent="Go-http-client/2.0" audit-ID="c11bff26-03e9-47b8-b6f8-8e75f4376a1f" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:27.535034       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="20.360087ms" userAgent="Go-http-client/2.0" audit-ID="f3ab613b-2d2a-428e-acc3-f080730bb08a" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:27.536854       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="20.55261ms" userAgent="Go-http-client/2.0" audit-ID="e8b13703-b75e-47d5-aca3-8a7bc1e1b319" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:27.567581       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="68.439716ms" userAgent="Go-http-client/2.0" audit-ID="c54a1bb3-811b-476f-bbe4-b04d19cc62a4" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:27.568734       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="70.667743ms" userAgent="Go-http-client/2.0" audit-ID="4c159ffd-e3e2-4e4d-b36b-c41e8ab9d6d4" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:27.573986       1 writers.go:122] apiserver was unable to write a JSON response: http2: stream closed
E0125 15:53:27.574021       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http2: stream closed"}: http2: stream closed
E0125 15:53:27.576600       1 writers.go:135] apiserver was unable to write a fallback JSON response: http2: stream closed
I0125 15:53:27.578972       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.166003ms" userAgent="Go-http-client/2.0" audit-ID="fb54b347-6142-46ea-bfda-1bb054c8fdcf" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:27.579128       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.321623ms" userAgent="Go-http-client/2.0" audit-ID="18a38ed7-cc11-4fc4-a982-9a7f67d279c2" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:27.579133       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="50.54454ms" userAgent="Go-http-client/2.0" audit-ID="0cd07cd5-b557-4788-b6d9-a65235b218b8" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:27.579157       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="0cd07cd5-b557-4788-b6d9-a65235b218b8"
E0125 15:53:27.579228       1 writers.go:122] apiserver was unable to write a JSON response: http2: stream closed
I0125 15:53:27.579253       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="10.066524ms" userAgent="Go-http-client/2.0" audit-ID="2e6b7371-4af3-4896-b03e-5e7ce8e9bb87" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:27.579273       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="2e6b7371-4af3-4896-b03e-5e7ce8e9bb87"
I0125 15:53:27.581224       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.014136ms" userAgent="Go-http-client/2.0" audit-ID="875b148f-e028-4a86-8ace-95ef6ac1eaea" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:27.589047       1 httplog.go:132] "HTTP" verb="GET" URI="/apis" latency="2.150576ms" userAgent="" audit-ID="e16cdd95-5456-47b1-b94d-0d2bfc97ed8d" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:27.590420       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="8.913604ms" userAgent="Go-http-client/2.0" audit-ID="883f960d-fcc7-4de5-8fa8-c233cec0fcc6" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:27.590667       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="883f960d-fcc7-4de5-8fa8-c233cec0fcc6"
E0125 15:53:27.590458       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
I0125 15:53:27.590494       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="16.425575ms" userAgent="Go-http-client/2.0" audit-ID="e463c7a6-4f4e-4be6-894f-b44403fe0d41" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:27.590541       1 timeout.go:142] post-timeout activity - time-elapsed: 11.369407ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
I0125 15:53:27.592646       1 httplog.go:132] "HTTP" verb="GET" URI="/apis" latency="10.644799ms" userAgent="" audit-ID="44166f45-5fe0-404a-b030-0f2f4debffc5" srcIP="172.31.0.147:39118" resp=200
E0125 15:53:27.592718       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http2: stream closed"}: http2: stream closed
E0125 15:53:27.594125       1 writers.go:122] apiserver was unable to write a JSON response: http2: stream closed
I0125 15:53:27.594177       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="12.651129ms" userAgent="Go-http-client/2.0" audit-ID="48973ee6-83de-4acb-b176-0f667f02bffc" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:27.594197       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="48973ee6-83de-4acb-b176-0f667f02bffc"
E0125 15:53:27.594232       1 writers.go:135] apiserver was unable to write a fallback JSON response: http2: stream closed
I0125 15:53:27.594295       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="14.933708ms" userAgent="Go-http-client/2.0" audit-ID="316145fa-2022-4d82-9e6b-ea02bb118fad" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:27.598447       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="10.152365ms" userAgent="Go-http-client/2.0" audit-ID="6e51de54-3263-411b-adcf-a3973b2f39fe" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:27.599187       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="6e51de54-3263-411b-adcf-a3973b2f39fe"
E0125 15:53:27.598546       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0125 15:53:27.599407       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:53:27.602799       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http2: stream closed"}: http2: stream closed
I0125 15:53:27.605297       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="28.628251ms" userAgent="Go-http-client/2.0" audit-ID="d2efc411-cb2a-444d-9c04-6272db2aa11b" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:27.605347       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="d2efc411-cb2a-444d-9c04-6272db2aa11b"
E0125 15:53:27.605478       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0125 15:53:27.605546       1 timeout.go:142] post-timeout activity - time-elapsed: 26.240797ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:53:27.608521       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:53:27.609917       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:53:27.611019       1 writers.go:135] apiserver was unable to write a fallback JSON response: http2: stream closed
E0125 15:53:27.613195       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:53:27.615462       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:53:27.616622       1 timeout.go:142] post-timeout activity - time-elapsed: 26.051794ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:53:27.617816       1 timeout.go:142] post-timeout activity - time-elapsed: 23.600051ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:53:27.618908       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:53:27.619995       1 timeout.go:142] post-timeout activity - time-elapsed: 21.419269ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:53:27.623508       1 timeout.go:142] post-timeout activity - time-elapsed: 18.135122ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
I0125 15:53:28.390041       1 httplog.go:132] "HTTP" verb="GET" URI="/openapi/v2" latency="14.026816ms" userAgent="" audit-ID="dc63eec3-652e-4e40-9c59-78557ef03d2a" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:28.391807       1 httplog.go:132] "HTTP" verb="GET" URI="/openapi/v3" latency="15.862639ms" userAgent="" audit-ID="b8b39dee-a725-4341-bf3a-d90894cf9b6a" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:28.393506       1 httplog.go:132] "HTTP" verb="GET" URI="/openapi/v3" latency="1.226362ms" userAgent="" audit-ID="c75269df-cb12-41c2-8770-12d5aef156a2" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:28.635873       1 httplog.go:132] "HTTP" verb="GET" URI="/openapi/v2" latency="5.371542ms" userAgent="" audit-ID="78a1f757-aa6a-49d9-9036-a902d887b01e" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:29.869892       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dsidekiq%2Cqueue-pod-name%3Dall-in-1%2Crelease%3Dgitlab" latency="4.384673ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="3b46ae18-63a7-4e21-91d2-8eb710b925a2" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:29.871764       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dkas%2Crelease%3Dgitlab" latency="6.231109ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="a5b9539e-baf1-413a-9150-dcf57d345698" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:29.872057       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dwebservice%2Cgitlab.com%2Fwebservice-name%3Ddefault%2Crelease%3Dgitlab" latency="6.008698ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="5f3727bf-fda2-4b6e-bc88-7397029ef8dd" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:36.329146       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/default/pods" latency="2.311455ms" userAgent="kubectl/v1.27.9 (linux/amd64) kubernetes/d15213f" audit-ID="13bececd-472b-4786-bc67-c5632f1f0879" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:37.184823       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="190.447µs" userAgent="kube-probe/1.27" audit-ID="43242a44-c809-4068-8b11-f11477f707ab" srcIP="172.31.0.147:36852" resp=200
I0125 15:53:37.185719       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="165.547µs" userAgent="kube-probe/1.27" audit-ID="768d58f7-e90b-43be-9b3f-f6922a365446" srcIP="172.31.0.147:36854" resp=200
I0125 15:53:37.871522       1 httplog.go:132] "HTTP" verb="GET" URI="/apis" latency="2.554397ms" userAgent="" audit-ID="6b488398-b951-4ff6-a8c0-ae69c1a4606b" srcIP="172.31.0.147:39118" resp=304
I0125 15:53:37.871592       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="1.518899ms" userAgent="Go-http-client/2.0" audit-ID="420ae107-af1b-4262-b8f6-6641196a5a8a" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:37.871999       1 httplog.go:132] "HTTP" verb="GET" URI="/apis" latency="1.894278ms" userAgent="" audit-ID="98c412e4-5d79-491b-92cc-0f1fd2aab01f" srcIP="172.31.0.147:39118" resp=304
I0125 15:53:37.872243       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.158425ms" userAgent="Go-http-client/2.0" audit-ID="7398f209-e04f-414b-b8ac-adaa90a007b8" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:37.873980       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.206634ms" userAgent="Go-http-client/2.0" audit-ID="e47952d9-2da6-49c4-85d1-b43dd54d2d7d" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:37.874874       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.244705ms" userAgent="Go-http-client/2.0" audit-ID="89ad037b-8648-4b69-a662-4b79c5c61b8e" srcIP="172.31.0.147:39088" resp=200
I0125 15:53:37.876593       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="7.516239ms" userAgent="Go-http-client/2.0" audit-ID="4f8e5540-ffd6-45fb-85e6-c2eb8d2631f7" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:37.880751       1 writers.go:122] apiserver was unable to write a JSON response: http2: stream closed
E0125 15:53:37.880906       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http2: stream closed"}: http2: stream closed
I0125 15:53:37.880863       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="8.323387ms" userAgent="Go-http-client/2.0" audit-ID="5e3e2162-635b-41d4-aaa4-4857de7533cf" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:37.881002       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="5e3e2162-635b-41d4-aaa4-4857de7533cf"
I0125 15:53:37.881538       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="9.813882ms" userAgent="Go-http-client/2.0" audit-ID="973744b4-5794-4949-8eff-ec8634c3514d" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:37.881649       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0125 15:53:37.881726       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="973744b4-5794-4949-8eff-ec8634c3514d"
E0125 15:53:37.882653       1 writers.go:135] apiserver was unable to write a fallback JSON response: http2: stream closed
I0125 15:53:37.883179       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="10.425706ms" userAgent="Go-http-client/2.0" audit-ID="70739985-a137-414c-a398-7e64e7f8e0b3" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:37.883204       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="70739985-a137-414c-a398-7e64e7f8e0b3"
E0125 15:53:37.883400       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0125 15:53:37.884271       1 timeout.go:142] post-timeout activity - time-elapsed: 3.387412ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
I0125 15:53:37.884643       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="13.474427ms" userAgent="Go-http-client/2.0" audit-ID="97f4a209-d6e3-4d18-87de-b6f79c1a1931" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:37.884673       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="97f4a209-d6e3-4d18-87de-b6f79c1a1931"
E0125 15:53:37.884808       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
I0125 15:53:37.885364       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="13.250099ms" userAgent="Go-http-client/2.0" audit-ID="d710da00-564e-4527-be69-f25e4b3b8391" srcIP="172.31.0.147:39088" resp=200
E0125 15:53:37.885369       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0125 15:53:37.885399       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="d710da00-564e-4527-be69-f25e4b3b8391"
E0125 15:53:37.886409       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:53:37.889723       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:53:37.892976       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:53:37.894043       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:53:37.896242       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:53:37.897293       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:53:37.898376       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:53:37.899451       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:53:37.900535       1 timeout.go:142] post-timeout activity - time-elapsed: 18.968326ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:53:37.901604       1 timeout.go:142] post-timeout activity - time-elapsed: 18.384727ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:53:37.902675       1 timeout.go:142] post-timeout activity - time-elapsed: 17.985535ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:53:37.903764       1 timeout.go:142] post-timeout activity - time-elapsed: 18.336431ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:53:38.772538       1 provider.go:191] unable to fetch CPU metrics for pod &PartialObjectMetadata{ObjectMeta:ObjectMeta{Name:prometheus-adapter-79d48545c4-cncrt,GenerateName:prometheus-adapter-79d48545c4-,Namespace:monitoring,SelfLink:,UID:d88bbef5-7b82-473b-89f1-2dce69cfd758,ResourceVersion:11642517,Generation:0,CreationTimestamp:2024-01-25 15:52:46 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{app.kubernetes.io/component: metrics,app.kubernetes.io/instance: prometheus-adapter,app.kubernetes.io/managed-by: Helm,app.kubernetes.io/name: prometheus-adapter,app.kubernetes.io/part-of: prometheus-adapter,app.kubernetes.io/version: v0.11.2,helm.sh/chart: prometheus-adapter-4.9.0,pod-template-hash: 79d48545c4,},Annotations:map[string]string{checksum/config: 7cc936be6ecd1bebc2b9693f8787d4cc36ad2a238c6a7e55babfcf1fcaa0f2bd,k8s.v1.cni.cncf.io/network-status: [{
    "name": "cilium",
    "interface": "eth0",
    "ips": [
        "172.31.0.165"
    ],
    "mac": "92:26:15:8d:ec:fc",
    "default": true,
    "dns": {},
    "gateway": [
        "172.31.0.147"
    ]
}],},OwnerReferences:[]OwnerReference{OwnerReference{Kind:ReplicaSet,Name:prometheus-adapter-79d48545c4,UID:f40a3f73-2914-477b-9fb4-1ce7fb1ed1fe,APIVersion:apps/v1,Controller:*true,BlockOwnerDeletion:*true,},},Finalizers:[],ManagedFields:[]ManagedFieldsEntry{ManagedFieldsEntry{Manager:kube-controller-manager,Operation:Update,APIVersion:v1,Time:2024-01-25 15:52:46 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:metadata":{"f:annotations":{".":{},"f:checksum/config":{}},"f:generateName":{},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{},"f:helm.sh/chart":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"f40a3f73-2914-477b-9fb4-1ce7fb1ed1fe\"}":{}}},"f:spec":{"f:affinity":{},"f:containers":{"k:{\"name\":\"prometheus-adapter\"}":{".":{},"f:args":{},"f:image":{},"f:imagePullPolicy":{},"f:livenessProbe":{".":{},"f:failureThreshold":{},"f:httpGet":{".":{},"f:path":{},"f:port":{},"f:scheme":{}},"f:initialDelaySeconds":{},"f:periodSeconds":{},"f:successThreshold":{},"f:timeoutSeconds":{}},"f:name":{},"f:ports":{".":{},"k:{\"containerPort\":6443,\"protocol\":\"TCP\"}":{".":{},"f:containerPort":{},"f:name":{},"f:protocol":{}}},"f:readinessProbe":{".":{},"f:failureThreshold":{},"f:httpGet":{".":{},"f:path":{},"f:port":{},"f:scheme":{}},"f:initialDelaySeconds":{},"f:periodSeconds":{},"f:successThreshold":{},"f:timeoutSeconds":{}},"f:resources":{},"f:securityContext":{".":{},"f:allowPrivilegeEscalation":{},"f:capabilities":{".":{},"f:drop":{}},"f:readOnlyRootFilesystem":{},"f:runAsNonRoot":{},"f:runAsUser":{},"f:seccompProfile":{".":{},"f:type":{}}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/etc/adapter/\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/tmp\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{".":{},"f:fsGroup":{}},"f:serviceAccount":{},"f:serviceAccountName":{},"f:terminationGracePeriodSeconds":{},"f:volumes":{".":{},"k:{\"name\":\"config\"}":{".":{},"f:configMap":{".":{},"f:defaultMode":{},"f:name":{}},"f:name":{}},"k:{\"name\":\"tmp\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}},Subresource:,},ManagedFieldsEntry{Manager:multus-daemon,Operation:Update,APIVersion:v1,Time:2024-01-25 15:52:47 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{}}}},Subresource:status,},ManagedFieldsEntry{Manager:kubelet,Operation:Update,APIVersion:v1,Time:2024-01-25 15:53:27 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:phase":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"172.31.0.165\"}":{".":{},"f:ip":{}}},"f:startTime":{}}},Subresource:status,},},},}, skipping
I0125 15:53:38.773972       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/pods" latency="20.458969ms" userAgent="kubectl/v1.27.9 (linux/amd64) kubernetes/d15213f" audit-ID="ce040c34-1b5f-4363-83b8-c44114cd63d0" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:42.096764       1 provider.go:291] missing CPU for node "cp1.int.hu1.sktrend.net", skipping
I0125 15:53:42.096994       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/nodes" latency="2.936282ms" userAgent="kubectl/v1.27.9 (linux/amd64) kubernetes/d15213f" audit-ID="d70a4018-0a0d-4040-9730-90a2137e9095" srcIP="172.31.0.147:39118" resp=200
E0125 15:53:44.832768       1 provider.go:191] unable to fetch CPU metrics for pod &PartialObjectMetadata{ObjectMeta:ObjectMeta{Name:prometheus-adapter-79d48545c4-cncrt,GenerateName:prometheus-adapter-79d48545c4-,Namespace:monitoring,SelfLink:,UID:d88bbef5-7b82-473b-89f1-2dce69cfd758,ResourceVersion:11642517,Generation:0,CreationTimestamp:2024-01-25 15:52:46 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{app.kubernetes.io/component: metrics,app.kubernetes.io/instance: prometheus-adapter,app.kubernetes.io/managed-by: Helm,app.kubernetes.io/name: prometheus-adapter,app.kubernetes.io/part-of: prometheus-adapter,app.kubernetes.io/version: v0.11.2,helm.sh/chart: prometheus-adapter-4.9.0,pod-template-hash: 79d48545c4,},Annotations:map[string]string{checksum/config: 7cc936be6ecd1bebc2b9693f8787d4cc36ad2a238c6a7e55babfcf1fcaa0f2bd,k8s.v1.cni.cncf.io/network-status: [{
    "name": "cilium",
    "interface": "eth0",
    "ips": [
        "172.31.0.165"
    ],
    "mac": "92:26:15:8d:ec:fc",
    "default": true,
    "dns": {},
    "gateway": [
        "172.31.0.147"
    ]
}],},OwnerReferences:[]OwnerReference{OwnerReference{Kind:ReplicaSet,Name:prometheus-adapter-79d48545c4,UID:f40a3f73-2914-477b-9fb4-1ce7fb1ed1fe,APIVersion:apps/v1,Controller:*true,BlockOwnerDeletion:*true,},},Finalizers:[],ManagedFields:[]ManagedFieldsEntry{ManagedFieldsEntry{Manager:kube-controller-manager,Operation:Update,APIVersion:v1,Time:2024-01-25 15:52:46 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:metadata":{"f:annotations":{".":{},"f:checksum/config":{}},"f:generateName":{},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{},"f:helm.sh/chart":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"f40a3f73-2914-477b-9fb4-1ce7fb1ed1fe\"}":{}}},"f:spec":{"f:affinity":{},"f:containers":{"k:{\"name\":\"prometheus-adapter\"}":{".":{},"f:args":{},"f:image":{},"f:imagePullPolicy":{},"f:livenessProbe":{".":{},"f:failureThreshold":{},"f:httpGet":{".":{},"f:path":{},"f:port":{},"f:scheme":{}},"f:initialDelaySeconds":{},"f:periodSeconds":{},"f:successThreshold":{},"f:timeoutSeconds":{}},"f:name":{},"f:ports":{".":{},"k:{\"containerPort\":6443,\"protocol\":\"TCP\"}":{".":{},"f:containerPort":{},"f:name":{},"f:protocol":{}}},"f:readinessProbe":{".":{},"f:failureThreshold":{},"f:httpGet":{".":{},"f:path":{},"f:port":{},"f:scheme":{}},"f:initialDelaySeconds":{},"f:periodSeconds":{},"f:successThreshold":{},"f:timeoutSeconds":{}},"f:resources":{},"f:securityContext":{".":{},"f:allowPrivilegeEscalation":{},"f:capabilities":{".":{},"f:drop":{}},"f:readOnlyRootFilesystem":{},"f:runAsNonRoot":{},"f:runAsUser":{},"f:seccompProfile":{".":{},"f:type":{}}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/etc/adapter/\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/tmp\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{".":{},"f:fsGroup":{}},"f:serviceAccount":{},"f:serviceAccountName":{},"f:terminationGracePeriodSeconds":{},"f:volumes":{".":{},"k:{\"name\":\"config\"}":{".":{},"f:configMap":{".":{},"f:defaultMode":{},"f:name":{}},"f:name":{}},"k:{\"name\":\"tmp\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}},Subresource:,},ManagedFieldsEntry{Manager:multus-daemon,Operation:Update,APIVersion:v1,Time:2024-01-25 15:52:47 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{}}}},Subresource:status,},ManagedFieldsEntry{Manager:kubelet,Operation:Update,APIVersion:v1,Time:2024-01-25 15:53:27 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:phase":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"172.31.0.165\"}":{".":{},"f:ip":{}}},"f:startTime":{}}},Subresource:status,},},},}, skipping
I0125 15:53:44.833283       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/monitoring/pods" latency="4.743063ms" userAgent="node-fetch" audit-ID="9cda6aa6-38f0-4f01-b945-4cbcba3cb96a" srcIP="172.31.0.147:39118" resp=200
E0125 15:53:44.834515       1 provider.go:191] unable to fetch CPU metrics for pod &PartialObjectMetadata{ObjectMeta:ObjectMeta{Name:prometheus-adapter-79d48545c4-cncrt,GenerateName:prometheus-adapter-79d48545c4-,Namespace:monitoring,SelfLink:,UID:d88bbef5-7b82-473b-89f1-2dce69cfd758,ResourceVersion:11642517,Generation:0,CreationTimestamp:2024-01-25 15:52:46 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{app.kubernetes.io/component: metrics,app.kubernetes.io/instance: prometheus-adapter,app.kubernetes.io/managed-by: Helm,app.kubernetes.io/name: prometheus-adapter,app.kubernetes.io/part-of: prometheus-adapter,app.kubernetes.io/version: v0.11.2,helm.sh/chart: prometheus-adapter-4.9.0,pod-template-hash: 79d48545c4,},Annotations:map[string]string{checksum/config: 7cc936be6ecd1bebc2b9693f8787d4cc36ad2a238c6a7e55babfcf1fcaa0f2bd,k8s.v1.cni.cncf.io/network-status: [{
    "name": "cilium",
    "interface": "eth0",
    "ips": [
        "172.31.0.165"
    ],
    "mac": "92:26:15:8d:ec:fc",
    "default": true,
    "dns": {},
    "gateway": [
        "172.31.0.147"
    ]
}],},OwnerReferences:[]OwnerReference{OwnerReference{Kind:ReplicaSet,Name:prometheus-adapter-79d48545c4,UID:f40a3f73-2914-477b-9fb4-1ce7fb1ed1fe,APIVersion:apps/v1,Controller:*true,BlockOwnerDeletion:*true,},},Finalizers:[],ManagedFields:[]ManagedFieldsEntry{ManagedFieldsEntry{Manager:kube-controller-manager,Operation:Update,APIVersion:v1,Time:2024-01-25 15:52:46 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:metadata":{"f:annotations":{".":{},"f:checksum/config":{}},"f:generateName":{},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{},"f:helm.sh/chart":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"f40a3f73-2914-477b-9fb4-1ce7fb1ed1fe\"}":{}}},"f:spec":{"f:affinity":{},"f:containers":{"k:{\"name\":\"prometheus-adapter\"}":{".":{},"f:args":{},"f:image":{},"f:imagePullPolicy":{},"f:livenessProbe":{".":{},"f:failureThreshold":{},"f:httpGet":{".":{},"f:path":{},"f:port":{},"f:scheme":{}},"f:initialDelaySeconds":{},"f:periodSeconds":{},"f:successThreshold":{},"f:timeoutSeconds":{}},"f:name":{},"f:ports":{".":{},"k:{\"containerPort\":6443,\"protocol\":\"TCP\"}":{".":{},"f:containerPort":{},"f:name":{},"f:protocol":{}}},"f:readinessProbe":{".":{},"f:failureThreshold":{},"f:httpGet":{".":{},"f:path":{},"f:port":{},"f:scheme":{}},"f:initialDelaySeconds":{},"f:periodSeconds":{},"f:successThreshold":{},"f:timeoutSeconds":{}},"f:resources":{},"f:securityContext":{".":{},"f:allowPrivilegeEscalation":{},"f:capabilities":{".":{},"f:drop":{}},"f:readOnlyRootFilesystem":{},"f:runAsNonRoot":{},"f:runAsUser":{},"f:seccompProfile":{".":{},"f:type":{}}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/etc/adapter/\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/tmp\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{".":{},"f:fsGroup":{}},"f:serviceAccount":{},"f:serviceAccountName":{},"f:terminationGracePeriodSeconds":{},"f:volumes":{".":{},"k:{\"name\":\"config\"}":{".":{},"f:configMap":{".":{},"f:defaultMode":{},"f:name":{}},"f:name":{}},"k:{\"name\":\"tmp\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}},Subresource:,},ManagedFieldsEntry{Manager:multus-daemon,Operation:Update,APIVersion:v1,Time:2024-01-25 15:52:47 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{}}}},Subresource:status,},ManagedFieldsEntry{Manager:kubelet,Operation:Update,APIVersion:v1,Time:2024-01-25 15:53:27 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:phase":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"172.31.0.165\"}":{".":{},"f:ip":{}}},"f:startTime":{}}},Subresource:status,},},},}, skipping
I0125 15:53:44.834861       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/monitoring/pods" latency="5.857717ms" userAgent="node-fetch" audit-ID="e1357a9f-72a3-4698-8b9f-9a14834efe17" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:44.885735       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dkas%2Crelease%3Dgitlab" latency="5.351532ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="6dd69691-fe3f-471f-b3c9-b8d9b23fc36e" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:44.886919       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dwebservice%2Cgitlab.com%2Fwebservice-name%3Ddefault%2Crelease%3Dgitlab" latency="6.446717ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="7fede069-1f9b-4774-81c4-6edc38568552" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:44.886919       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dsidekiq%2Cqueue-pod-name%3Dall-in-1%2Crelease%3Dgitlab" latency="6.508599ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="9784e3cd-52f2-4d49-9e64-17453d2a616b" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:47.183640       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="227.605µs" userAgent="kube-probe/1.27" audit-ID="2c631754-2e17-4f46-a94c-60a48f9b869c" srcIP="172.31.0.147:39090" resp=200
I0125 15:53:47.184176       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="137.825µs" userAgent="kube-probe/1.27" audit-ID="c4f39d8c-81a0-4e60-81eb-719bb4c9b7f0" srcIP="172.31.0.147:39092" resp=200
I0125 15:53:57.186445       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="219.083µs" userAgent="kube-probe/1.27" audit-ID="5a4b4326-55d2-4125-a771-ac2781bbb033" srcIP="172.31.0.147:35006" resp=200
I0125 15:53:57.186786       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="125.786µs" userAgent="kube-probe/1.27" audit-ID="9cda84cf-c25d-4127-94ce-b52fbb69abe7" srcIP="172.31.0.147:35004" resp=200
I0125 15:53:59.926059       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dkas%2Crelease%3Dgitlab" latency="16.600826ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="74fe56f2-c492-43a5-b560-d10d2260bf6c" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:59.926120       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dsidekiq%2Cqueue-pod-name%3Dall-in-1%2Crelease%3Dgitlab" latency="16.651277ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="fdae7602-d084-4ada-8e18-b856870be771" srcIP="172.31.0.147:39118" resp=200
I0125 15:53:59.927962       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dwebservice%2Cgitlab.com%2Fwebservice-name%3Ddefault%2Crelease%3Dgitlab" latency="18.473861ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="550b7279-1539-4687-8c48-e5142e656735" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:07.185467       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="175.583µs" userAgent="kube-probe/1.27" audit-ID="bdde94f7-a996-472c-b70e-3588bc93e21c" srcIP="172.31.0.147:59076" resp=200
I0125 15:54:07.186261       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="163.334µs" userAgent="kube-probe/1.27" audit-ID="a1124b91-bba2-4b94-b514-0e3bab5cbd66" srcIP="172.31.0.147:59074" resp=200
I0125 15:54:07.883323       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="8.984916ms" userAgent="Go-http-client/2.0" audit-ID="0fc9c5af-7f82-4f60-a128-020bc9a2d02c" srcIP="172.31.0.147:39088" resp=200
I0125 15:54:07.885626       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.110903ms" userAgent="Go-http-client/2.0" audit-ID="360fc762-bbf7-43ca-a1a5-33553617c8e5" srcIP="172.31.0.147:39088" resp=200
I0125 15:54:07.887634       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="1.835989ms" userAgent="Go-http-client/2.0" audit-ID="82ab221a-2bad-4079-ab8b-d23f6f82f95a" srcIP="172.31.0.147:39088" resp=200
I0125 15:54:07.889954       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.006849ms" userAgent="Go-http-client/2.0" audit-ID="b5dbd713-914c-4d3a-9f21-f4157a65b6cf" srcIP="172.31.0.147:39088" resp=200
I0125 15:54:07.890879       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.64482ms" userAgent="Go-http-client/2.0" audit-ID="217b372b-53d9-4215-b1e7-590a20ed1f3e" srcIP="172.31.0.147:39088" resp=200
E0125 15:54:07.897310       1 writers.go:122] apiserver was unable to write a JSON response: http2: stream closed
E0125 15:54:07.897333       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http2: stream closed"}: http2: stream closed
I0125 15:54:07.897382       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="7.370358ms" userAgent="Go-http-client/2.0" audit-ID="bbd8049f-001e-45c7-a325-0dc861e3801c" srcIP="172.31.0.147:39088" resp=200
E0125 15:54:07.897444       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="bbd8049f-001e-45c7-a325-0dc861e3801c"
E0125 15:54:07.903970       1 writers.go:135] apiserver was unable to write a fallback JSON response: http2: stream closed
I0125 15:54:07.905310       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="17.092742ms" userAgent="Go-http-client/2.0" audit-ID="ebf86fee-a857-4c74-aabf-d62db2a302d4" srcIP="172.31.0.147:39088" resp=200
E0125 15:54:07.905493       1 timeout.go:142] post-timeout activity - time-elapsed: 8.029221ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:54:07.906197       1 writers.go:122] apiserver was unable to write a JSON response: http2: stream closed
I0125 15:54:07.906199       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="15.233934ms" userAgent="Go-http-client/2.0" audit-ID="86dc9d3d-46a3-4dcc-b365-69a7117235bc" srcIP="172.31.0.147:39088" resp=200
E0125 15:54:07.906399       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="86dc9d3d-46a3-4dcc-b365-69a7117235bc"
E0125 15:54:07.906210       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
I0125 15:54:07.906249       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="18.021337ms" userAgent="Go-http-client/2.0" audit-ID="644420e1-3d8b-4de3-8f53-cd935c7f66d8" srcIP="172.31.0.147:39088" resp=200
E0125 15:54:07.906816       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="644420e1-3d8b-4de3-8f53-cd935c7f66d8"
E0125 15:54:07.908775       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http2: stream closed"}: http2: stream closed
E0125 15:54:07.911183       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:54:07.912186       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
I0125 15:54:07.912205       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="21.788154ms" userAgent="Go-http-client/2.0" audit-ID="efbe200f-5571-48bf-b4b4-12025838e102" srcIP="172.31.0.147:39088" resp=200
E0125 15:54:07.912365       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="efbe200f-5571-48bf-b4b4-12025838e102"
E0125 15:54:07.913429       1 writers.go:135] apiserver was unable to write a fallback JSON response: http2: stream closed
E0125 15:54:07.914624       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:54:07.915777       1 timeout.go:142] post-timeout activity - time-elapsed: 9.559396ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:54:07.916854       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:54:07.919322       1 timeout.go:142] post-timeout activity - time-elapsed: 13.012223ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:54:07.921752       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:54:07.923968       1 timeout.go:142] post-timeout activity - time-elapsed: 11.742522ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
I0125 15:54:14.936383       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dwebservice%2Cgitlab.com%2Fwebservice-name%3Ddefault%2Crelease%3Dgitlab" latency="4.908545ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="6a11ebe3-7f92-465e-89e9-01f766402a27" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:14.937940       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dsidekiq%2Cqueue-pod-name%3Dall-in-1%2Crelease%3Dgitlab" latency="6.481505ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="96183dea-7004-4971-ba8e-ef484fd8f51c" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:14.938103       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dkas%2Crelease%3Dgitlab" latency="6.739369ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="1c88f212-3975-4303-8d02-b5e678c24319" srcIP="172.31.0.147:39118" resp=200
E0125 15:54:15.756153       1 provider.go:191] unable to fetch CPU metrics for pod &PartialObjectMetadata{ObjectMeta:ObjectMeta{Name:prometheus-adapter-79d48545c4-cncrt,GenerateName:prometheus-adapter-79d48545c4-,Namespace:monitoring,SelfLink:,UID:d88bbef5-7b82-473b-89f1-2dce69cfd758,ResourceVersion:11642517,Generation:0,CreationTimestamp:2024-01-25 15:52:46 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{app.kubernetes.io/component: metrics,app.kubernetes.io/instance: prometheus-adapter,app.kubernetes.io/managed-by: Helm,app.kubernetes.io/name: prometheus-adapter,app.kubernetes.io/part-of: prometheus-adapter,app.kubernetes.io/version: v0.11.2,helm.sh/chart: prometheus-adapter-4.9.0,pod-template-hash: 79d48545c4,},Annotations:map[string]string{checksum/config: 7cc936be6ecd1bebc2b9693f8787d4cc36ad2a238c6a7e55babfcf1fcaa0f2bd,k8s.v1.cni.cncf.io/network-status: [{
    "name": "cilium",
    "interface": "eth0",
    "ips": [
        "172.31.0.165"
    ],
    "mac": "92:26:15:8d:ec:fc",
    "default": true,
    "dns": {},
    "gateway": [
        "172.31.0.147"
    ]
}],},OwnerReferences:[]OwnerReference{OwnerReference{Kind:ReplicaSet,Name:prometheus-adapter-79d48545c4,UID:f40a3f73-2914-477b-9fb4-1ce7fb1ed1fe,APIVersion:apps/v1,Controller:*true,BlockOwnerDeletion:*true,},},Finalizers:[],ManagedFields:[]ManagedFieldsEntry{ManagedFieldsEntry{Manager:kube-controller-manager,Operation:Update,APIVersion:v1,Time:2024-01-25 15:52:46 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:metadata":{"f:annotations":{".":{},"f:checksum/config":{}},"f:generateName":{},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{},"f:helm.sh/chart":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"f40a3f73-2914-477b-9fb4-1ce7fb1ed1fe\"}":{}}},"f:spec":{"f:affinity":{},"f:containers":{"k:{\"name\":\"prometheus-adapter\"}":{".":{},"f:args":{},"f:image":{},"f:imagePullPolicy":{},"f:livenessProbe":{".":{},"f:failureThreshold":{},"f:httpGet":{".":{},"f:path":{},"f:port":{},"f:scheme":{}},"f:initialDelaySeconds":{},"f:periodSeconds":{},"f:successThreshold":{},"f:timeoutSeconds":{}},"f:name":{},"f:ports":{".":{},"k:{\"containerPort\":6443,\"protocol\":\"TCP\"}":{".":{},"f:containerPort":{},"f:name":{},"f:protocol":{}}},"f:readinessProbe":{".":{},"f:failureThreshold":{},"f:httpGet":{".":{},"f:path":{},"f:port":{},"f:scheme":{}},"f:initialDelaySeconds":{},"f:periodSeconds":{},"f:successThreshold":{},"f:timeoutSeconds":{}},"f:resources":{},"f:securityContext":{".":{},"f:allowPrivilegeEscalation":{},"f:capabilities":{".":{},"f:drop":{}},"f:readOnlyRootFilesystem":{},"f:runAsNonRoot":{},"f:runAsUser":{},"f:seccompProfile":{".":{},"f:type":{}}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/etc/adapter/\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/tmp\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{".":{},"f:fsGroup":{}},"f:serviceAccount":{},"f:serviceAccountName":{},"f:terminationGracePeriodSeconds":{},"f:volumes":{".":{},"k:{\"name\":\"config\"}":{".":{},"f:configMap":{".":{},"f:defaultMode":{},"f:name":{}},"f:name":{}},"k:{\"name\":\"tmp\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}},Subresource:,},ManagedFieldsEntry{Manager:multus-daemon,Operation:Update,APIVersion:v1,Time:2024-01-25 15:52:47 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{}}}},Subresource:status,},ManagedFieldsEntry{Manager:kubelet,Operation:Update,APIVersion:v1,Time:2024-01-25 15:53:27 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:phase":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"172.31.0.165\"}":{".":{},"f:ip":{}}},"f:startTime":{}}},Subresource:status,},},},}, skipping
I0125 15:54:15.756599       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/monitoring/pods" latency="4.548434ms" userAgent="node-fetch" audit-ID="f5ca8c45-8d24-43dd-8947-1e076300324a" srcIP="172.31.0.147:39118" resp=200
E0125 15:54:15.763444       1 provider.go:191] unable to fetch CPU metrics for pod &PartialObjectMetadata{ObjectMeta:ObjectMeta{Name:prometheus-adapter-79d48545c4-cncrt,GenerateName:prometheus-adapter-79d48545c4-,Namespace:monitoring,SelfLink:,UID:d88bbef5-7b82-473b-89f1-2dce69cfd758,ResourceVersion:11642517,Generation:0,CreationTimestamp:2024-01-25 15:52:46 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{app.kubernetes.io/component: metrics,app.kubernetes.io/instance: prometheus-adapter,app.kubernetes.io/managed-by: Helm,app.kubernetes.io/name: prometheus-adapter,app.kubernetes.io/part-of: prometheus-adapter,app.kubernetes.io/version: v0.11.2,helm.sh/chart: prometheus-adapter-4.9.0,pod-template-hash: 79d48545c4,},Annotations:map[string]string{checksum/config: 7cc936be6ecd1bebc2b9693f8787d4cc36ad2a238c6a7e55babfcf1fcaa0f2bd,k8s.v1.cni.cncf.io/network-status: [{
    "name": "cilium",
    "interface": "eth0",
    "ips": [
        "172.31.0.165"
    ],
    "mac": "92:26:15:8d:ec:fc",
    "default": true,
    "dns": {},
    "gateway": [
        "172.31.0.147"
    ]
}],},OwnerReferences:[]OwnerReference{OwnerReference{Kind:ReplicaSet,Name:prometheus-adapter-79d48545c4,UID:f40a3f73-2914-477b-9fb4-1ce7fb1ed1fe,APIVersion:apps/v1,Controller:*true,BlockOwnerDeletion:*true,},},Finalizers:[],ManagedFields:[]ManagedFieldsEntry{ManagedFieldsEntry{Manager:kube-controller-manager,Operation:Update,APIVersion:v1,Time:2024-01-25 15:52:46 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:metadata":{"f:annotations":{".":{},"f:checksum/config":{}},"f:generateName":{},"f:labels":{".":{},"f:app.kubernetes.io/component":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{},"f:app.kubernetes.io/version":{},"f:helm.sh/chart":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"f40a3f73-2914-477b-9fb4-1ce7fb1ed1fe\"}":{}}},"f:spec":{"f:affinity":{},"f:containers":{"k:{\"name\":\"prometheus-adapter\"}":{".":{},"f:args":{},"f:image":{},"f:imagePullPolicy":{},"f:livenessProbe":{".":{},"f:failureThreshold":{},"f:httpGet":{".":{},"f:path":{},"f:port":{},"f:scheme":{}},"f:initialDelaySeconds":{},"f:periodSeconds":{},"f:successThreshold":{},"f:timeoutSeconds":{}},"f:name":{},"f:ports":{".":{},"k:{\"containerPort\":6443,\"protocol\":\"TCP\"}":{".":{},"f:containerPort":{},"f:name":{},"f:protocol":{}}},"f:readinessProbe":{".":{},"f:failureThreshold":{},"f:httpGet":{".":{},"f:path":{},"f:port":{},"f:scheme":{}},"f:initialDelaySeconds":{},"f:periodSeconds":{},"f:successThreshold":{},"f:timeoutSeconds":{}},"f:resources":{},"f:securityContext":{".":{},"f:allowPrivilegeEscalation":{},"f:capabilities":{".":{},"f:drop":{}},"f:readOnlyRootFilesystem":{},"f:runAsNonRoot":{},"f:runAsUser":{},"f:seccompProfile":{".":{},"f:type":{}}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/etc/adapter/\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/tmp\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{".":{},"f:fsGroup":{}},"f:serviceAccount":{},"f:serviceAccountName":{},"f:terminationGracePeriodSeconds":{},"f:volumes":{".":{},"k:{\"name\":\"config\"}":{".":{},"f:configMap":{".":{},"f:defaultMode":{},"f:name":{}},"f:name":{}},"k:{\"name\":\"tmp\"}":{".":{},"f:emptyDir":{},"f:name":{}}}}},Subresource:,},ManagedFieldsEntry{Manager:multus-daemon,Operation:Update,APIVersion:v1,Time:2024-01-25 15:52:47 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:metadata":{"f:annotations":{"f:k8s.v1.cni.cncf.io/network-status":{}}}},Subresource:status,},ManagedFieldsEntry{Manager:kubelet,Operation:Update,APIVersion:v1,Time:2024-01-25 15:53:27 +0000 UTC,FieldsType:FieldsV1,FieldsV1:{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:phase":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"172.31.0.165\"}":{".":{},"f:ip":{}}},"f:startTime":{}}},Subresource:status,},},},}, skipping
I0125 15:54:15.763840       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/monitoring/pods" latency="4.696205ms" userAgent="node-fetch" audit-ID="7672cf06-3ba0-44e7-88cb-00ce6e6b2941" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:17.186755       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="160.855µs" userAgent="kube-probe/1.27" audit-ID="5ab55feb-a98a-4b27-9481-67572ac150af" srcIP="172.31.0.147:40736" resp=200
I0125 15:54:17.187436       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="115.337µs" userAgent="kube-probe/1.27" audit-ID="fb6f092a-b957-4d99-8168-9f617cd0d2f9" srcIP="172.31.0.147:40734" resp=200
I0125 15:54:25.571705       1 httplog.go:132] "HTTP" verb="GET" URI="/apis" latency="2.005475ms" userAgent="" audit-ID="784f9e8c-5ee4-4d83-8510-c3b2c49c0bb6" srcIP="172.31.0.147:39118" resp=304
I0125 15:54:25.572321       1 httplog.go:132] "HTTP" verb="GET" URI="/apis" latency="2.615981ms" userAgent="" audit-ID="e717215b-e4bc-4494-8a48-cccb229257c7" srcIP="172.31.0.147:39118" resp=304
I0125 15:54:27.185038       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="148.014µs" userAgent="kube-probe/1.27" audit-ID="5caf41f2-161b-4fe1-97fe-1fe12fe22acb" srcIP="172.31.0.147:53178" resp=200
I0125 15:54:27.185736       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="158.468µs" userAgent="kube-probe/1.27" audit-ID="c623a43a-db12-412e-9df1-a57aa85e86d8" srcIP="172.31.0.147:53176" resp=200
I0125 15:54:28.397354       1 httplog.go:132] "HTTP" verb="GET" URI="/openapi/v3" latency="4.236226ms" userAgent="" audit-ID="5bed9ccf-4dcd-41e1-b6cd-16c05d295640" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:28.398898       1 httplog.go:132] "HTTP" verb="GET" URI="/openapi/v3" latency="1.097208ms" userAgent="" audit-ID="a54cd85c-faae-447c-9ed8-85f99fd1e7da" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:28.633430       1 httplog.go:132] "HTTP" verb="GET" URI="/openapi/v2" latency="2.841657ms" userAgent="" audit-ID="36759ea1-6b17-46e8-b2ab-874159b3fb03" srcIP="172.31.0.147:39118" resp=304
I0125 15:54:28.708580       1 httplog.go:132] "HTTP" verb="GET" URI="/openapi/v2" latency="1.169494ms" userAgent="" audit-ID="01a4043d-41b7-4db6-bbbb-103b85f4a7b4" srcIP="172.31.0.147:39118" resp=304
I0125 15:54:29.961414       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dkas%2Crelease%3Dgitlab" latency="11.29106ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="296a9370-34af-4b32-9960-e2f850023d92" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:29.961841       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dwebservice%2Cgitlab.com%2Fwebservice-name%3Ddefault%2Crelease%3Dgitlab" latency="11.648951ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="74d2f515-ebeb-48e8-bbc1-f29e4db473c3" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:29.964723       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dsidekiq%2Cqueue-pod-name%3Dall-in-1%2Crelease%3Dgitlab" latency="14.533499ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="0e3bdfb0-cd34-4c86-833b-89f387c8cfe6" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:37.184446       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="171.709µs" userAgent="kube-probe/1.27" audit-ID="fa9d1fb2-25bc-406f-8085-f4a6404b7ef3" srcIP="172.31.0.147:49198" resp=200
I0125 15:54:37.184953       1 httplog.go:132] "HTTP" verb="GET" URI="/healthz" latency="112.56µs" userAgent="kube-probe/1.27" audit-ID="e2611a94-f726-4c8b-a4e0-407f84f7f083" srcIP="172.31.0.147:49196" resp=200
I0125 15:54:37.872057       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.194302ms" userAgent="Go-http-client/2.0" audit-ID="a760c34c-4327-415d-9248-bce5eb59421b" srcIP="172.31.0.147:39088" resp=200
I0125 15:54:37.873454       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="1.826919ms" userAgent="Go-http-client/2.0" audit-ID="c89f25d3-ff45-4fdb-8ad6-df97a8646a16" srcIP="172.31.0.147:39088" resp=200
I0125 15:54:37.873679       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.002338ms" userAgent="Go-http-client/2.0" audit-ID="82e9b9c7-fe3e-4933-acdf-1c64479b1992" srcIP="172.31.0.147:39088" resp=200
I0125 15:54:37.873955       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.34153ms" userAgent="Go-http-client/2.0" audit-ID="bfe706df-0d38-4ddc-86eb-540e92254c9d" srcIP="172.31.0.147:39088" resp=200
I0125 15:54:37.874106       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/metrics.k8s.io/v1beta1" latency="2.494759ms" userAgent="Go-http-client/2.0" audit-ID="23baee42-bbec-4c9a-918e-1ebf6fd65358" srcIP="172.31.0.147:39088" resp=200
I0125 15:54:37.880696       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="8.4685ms" userAgent="Go-http-client/2.0" audit-ID="e35ce2df-d949-4e57-acca-958b04f4c480" srcIP="172.31.0.147:39088" resp=200
E0125 15:54:37.880888       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="e35ce2df-d949-4e57-acca-958b04f4c480"
E0125 15:54:37.880767       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
I0125 15:54:37.881203       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="7.432418ms" userAgent="Go-http-client/2.0" audit-ID="5f3ac487-7157-414e-9636-d42516b631b5" srcIP="172.31.0.147:39088" resp=200
E0125 15:54:37.881429       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="5f3ac487-7157-414e-9636-d42516b631b5"
E0125 15:54:37.881342       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
I0125 15:54:37.881575       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="7.941627ms" userAgent="Go-http-client/2.0" audit-ID="6f24c905-7bb1-4e6f-b06b-b6dac9a0ca05" srcIP="172.31.0.147:39088" resp=200
E0125 15:54:37.881596       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="6f24c905-7bb1-4e6f-b06b-b6dac9a0ca05"
I0125 15:54:37.881663       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="9.341819ms" userAgent="Go-http-client/2.0" audit-ID="1fee0309-f3f2-40a2-ad7a-db5ce0a1db16" srcIP="172.31.0.147:39088" resp=200
E0125 15:54:37.881683       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="1fee0309-f3f2-40a2-ad7a-db5ce0a1db16"
E0125 15:54:37.881729       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0125 15:54:37.881802       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0125 15:54:37.882075       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
I0125 15:54:37.882157       1 panic.go:884] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta1" latency="7.84409ms" userAgent="Go-http-client/2.0" audit-ID="e905d9ad-6d49-4077-951f-c4a877c51a50" srcIP="172.31.0.147:39088" resp=200
E0125 15:54:37.882178       1 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/custom.metrics.k8s.io/v1beta1" audit-ID="e905d9ad-6d49-4077-951f-c4a877c51a50"
E0125 15:54:37.882350       1 writers.go:122] apiserver was unable to write a JSON response: http: Handler timeout
E0125 15:54:37.883215       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:54:37.884314       1 timeout.go:142] post-timeout activity - time-elapsed: 3.509254ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:54:37.886465       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:54:37.889723       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:54:37.890784       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:54:37.892957       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
E0125 15:54:37.895170       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:54:37.896235       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:54:37.897329       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:54:37.898400       1 writers.go:135] apiserver was unable to write a fallback JSON response: http: Handler timeout
E0125 15:54:37.899498       1 timeout.go:142] post-timeout activity - time-elapsed: 18.282327ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:54:37.900610       1 timeout.go:142] post-timeout activity - time-elapsed: 19.003247ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:54:37.901689       1 timeout.go:142] post-timeout activity - time-elapsed: 19.996499ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
E0125 15:54:37.902821       1 timeout.go:142] post-timeout activity - time-elapsed: 20.61369ms, GET "/apis/custom.metrics.k8s.io/v1beta1" result: <nil>
I0125 15:54:44.832141       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/monitoring/pods" latency="4.640097ms" userAgent="node-fetch" audit-ID="6d0db585-2334-420e-b9e2-bcc69c27097c" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:44.833377       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/monitoring/pods" latency="5.879391ms" userAgent="node-fetch" audit-ID="4898dc7f-3774-458b-86f4-1918ee14617c" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:44.971383       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dkas%2Crelease%3Dgitlab" latency="5.047627ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="dd44b1d1-3bb5-4376-9d00-5ade94048dac" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:44.971716       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dwebservice%2Cgitlab.com%2Fwebservice-name%3Ddefault%2Crelease%3Dgitlab" latency="5.3291ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="be4fafec-f73e-462b-aa38-232260e9ce35" srcIP="172.31.0.147:39118" resp=200
I0125 15:54:44.984218       1 httplog.go:132] "HTTP" verb="LIST" URI="/apis/metrics.k8s.io/v1beta1/namespaces/devel/pods?labelSelector=app%3Dsidekiq%2Cqueue-pod-name%3Dall-in-1%2Crelease%3Dgitlab" latency="2.795892ms" userAgent="kube-controller-manager/v1.27.9 (linux/amd64) kubernetes/d15213f/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="953d00c0-bcef-4fb3-ae90-78849ca15560" srcIP="172.31.0.147:39118" resp=200