elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
194 stars 421 forks source link

[Kubernetes]: Visualization Display errors for Kubernetes Jobs and Cronjobs Dashboards #11060

Open Danouchka opened 1 week ago

Danouchka commented 1 week ago

Integration Name

Kubernetes [kubernetes]

Dataset Name

No response

Integration Version

1.67.0

Agent Version

8.15.0

Agent Output Type

elasticsearch

Elasticsearch Version

8.15.1

OS Version and Architecture

Elastic Cloud GCP Belgium

Software/API Version

No response

Error Message

Visualizations below not rendered properly

Image

Error: `The above error occurred in l:

at l (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:450:80089)
at S (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:365:754987)
at div
at div
at t.default (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:376:230815)
at R (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:397:20810)
at div
at https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:360:46261
at div
at https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:360:46261
at x (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/plugin/expressionMetricVis/1.0.0/expressionMetricVis.chunk.1.js:1:1408)
at div
at https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:360:46261
at x (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-src/kbn-ui-shared-deps-src.js:43:313212)
at L (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-src/kbn-ui-shared-deps-src.js:43:314013)
at p (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-src/kbn-ui-shared-deps-src.js:43:308941)
at c (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:365:190149)
at t (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-src/kbn-ui-shared-deps-src.js:43:223282)
at ee (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-src/kbn-ui-shared-deps-src.js:43:223788)
at Context (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/core/core.entry.js:16:207034)
at p (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:365:594284)
at d (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:365:150723)
at v (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:360:46705)
at a (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:388:11593)
at f (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:6:59856)
at x (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:365:620368)
at s (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:376:203784)
at a (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:365:965943)
at m (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:381:1072272)
at T (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/plugin/expressionMetricVis/1.0.0/expressionMetricVis.plugin.js:1:11438)
at D (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/plugin/expressionMetricVis/1.0.0/expressionMetricVis.plugin.js:1:11763)
at C (https://sa-da-ec-gcp-clus-02.kb.europe-west1.gcp.cloud.es.io:9243/f66ec5b0ddd9/bundles/plugin/expressionMetricVis/1.0.0/expressionMetricVis.plugin.js:1:12243)`

Event Original

No response

What did you do?

Nothing

What did you see?

Image

What did you expect to see?

Properly rendered visualization

Anything else?

No response

Danouchka commented 1 week ago

cc @mlunadia

gizas commented 1 week ago

Assigned @constanca-m that currently is on sdh support.

From first look seems a visualisation problem. This visualisation is based on kubernetes.job.pods.* metics. Can you please provide us the discovery info for those metrics? Have you waited a little until agent sent some data and then refreshed the page?

Additionally:

Danouchka commented 1 week ago

Hi

my agents are running for a while ( 2 years) and are healthy

I have no jobs but the visualizations would render well in previous intégrations versions.

It's not because I have 0 results that i should get these visualization errors.

Thank you

Danouchka commented 1 week ago

Hi

I also cleaned my browser cache but it's the same

Best, Dan

gizas commented 1 week ago

I was able to reproduce it with 1.67.0

See 1.66.4: Image

See 1.67.0: Image

Seems that during export of last version something broke. We will investigate

gizas commented 1 week ago

I did some investigation and seems that just the rendering of the visualisations just changed from 8.14 to 8.15

See below 8.14.0 Image

In my case even in 8.15.1 when I had the above red error pages, when data came in, the visualisation worked.

Image

@Danouchka can I request once more the following:

kubectl get jobs -A
watch "kubectl get jobs -A"
kubectl get pods -A | grep -i elastic
gizas commented 1 week ago

FYI I added this issue here for kibana team: https://github.com/elastic/kibana/issues/192569

Danouchka commented 6 days ago

Hi @gizas

I get the following:

#watch "kubectl get jobs -A"
Every 2.0s: kubectl get jobs -A                                                                                                                                                                                                                                   

No resources found

#cloudshell:~ (elastic-sa)$ kubectl get pods -A | grep -i elastic
kube-system   elastic-agent-jzmhl                                     1/1     Running   0               2d19h
kube-system   elastic-agent-ldkl7                                     1/1     Running   0               2d19h
kube-system   elastic-agent-lmxf5                                     1/1     Running   0               2d19h
m-a-leclercq commented 1 day ago

Having the same issue with EDOT.