search

moxious / triage

testing triage actions for issues
0 stars 1 forks source link

Dashboards: not visible after upgarding to 10.3.0 #321

Open tonypowa opened 5 months ago

tonypowa commented 5 months ago

What happened?

We are trying to upgrade grafana version from 10.2.2 to 10.3.0 and we are not able to view any dashboards over grafana. Along with that I am getting these error logs in grafana pod:

logger=cleanup t=2024-03-07T08:52:20.607034908Z level=info msg="Completed cleanup jobs" duration=20.919916ms

logger=grafana.update.checker t=2024-03-07T08:52:20.681237965Z level=error msg="Update check failed" error="failed to get stable version from grafana.com: Get \"https://grafana.com/api/grafana/versions/stable\": EOF" duration=72.929456ms

logger=cleanup t=2024-03-07T09:02:20.612760994Z level=info msg="Completed cleanup jobs" duration=26.036921ms

logger=grafana.update.checker t=2024-03-07T09:02:20.646409821Z level=error msg="Update check failed" error="failed to get stable version from grafana.com: Get \"https://grafana.com/api/grafana/versions/stable\": EOF" duration=37.93563ms

logger=cleanup t=2024-03-07T09:12:20.597345966Z level=info msg="Completed cleanup jobs" duration=11.160309ms

logger=grafana.update.checker t=2024-03-07T09:12:20.616170181Z level=error msg="Update check failed" error="failed to get stable version from grafana.com: Get \"https://grafana.com/api/grafana/versions/stable\": EOF" duration=7.484706ms

logger=infra.usagestats t=2024-03-07T09:12:58.595035116Z level=info msg="Usage stats are ready to report"

logger=cleanup t=2024-03-07T09:22:20.612306142Z level=info msg="Completed cleanup jobs" duration=26.013134ms

logger=grafana.update.checker t=2024-03-07T09:22:20.642918382Z level=error msg="Update check failed" error="failed to get stable version from grafana.com: Get \"https://grafana.com/api/grafana/versions/stable\": EOF" duration=34.838946ms

Attaching full logs here as well.

grafana_error_logs.txt

image

What did you expect to happen?

We are expecting successfull upgrade to newer versions of grafana starting from 10.3.x versions to resolve our CVEs and other issues as well. But we are not able to get dashbaords after that and somehow its complaining that stable version isnot present here: https://grafana.com/api/grafana/versions/stable.

Did this work before?

No

How do we reproduce it?

  1. Change the grafana old version to 10.3.x version

  2. Deploy it into some k8s cluster and check the grafana pod logs if they are getting simialr issues aboev attached of stable version not present.

3.Check grafana dashbaords if dahsboards are visible or not.

Is the bug inside a dashboard panel?

No response

Environment (with versions)?

Grafana: https://github.com/grafana/grafana/releases/tag/v10.3.0

OS: Mariner

Browser:

Grafana platform?

Kubernetes

Datasource(s)?

No response

tonypowa commented 5 months ago

This issue is a test copy of an issue in another repo. Original issue: https://github.com/grafana/grafana/issues/#84056

moxious commented 5 months ago

Summary: After upgrading Grafana from version 10.2.2 to 10.3.0, dashboards became invisible and logs indicate a failure in checking for updates against grafana.com's API, pointing to a potential issue with the new version's update checker or connectivity. This was unexpected as the upgrade was intended to resolve CVEs and other issues.

moxious commented 5 months ago

Hello @tonypowa, it seems you've encountered an issue after upgrading to Grafana 10.3.0 and can no longer see dashboards. This appears related to the Grafana Backend Platform as it involves the upgrade process and potential issues with Grafana backend services after the upgrade. I suggest providing the issue details in the Backend Platform Project to get the correct assistance. Make sure to include the error logs and steps you've taken for reproducing the issue.

moxious commented 5 months ago

Elaboration:

Hello @tonypowa,

Thank you for bringing this issue to our attention. To help us investigate this issue further, could you please provide the following additional information?

  1. Are there any relevant messages in the Grafana web UI, such as error popups or banners? Screenshots of the UI might be helpful.
  2. Could you provide us with the configuration settings that you are using for your Grafana deployment? Please omit any sensitive information.
  3. Did you observe any errors or warnings during the upgrade process itself that might be related to the dashboard issue?
  4. Are there any proxy or network policies in place that might be preventing access to https://grafana.com/api/grafana/versions/stable from within your Kubernetes cluster?
  5. Confirm if you are able to reach https://grafana.com/api/grafana/versions/stable successfully from outside the cluster environment.
  6. Could you also check and share the browser's developer console logs for any errors while accessing the dashboards?

Any additional context around the time the issue started occurring or steps taken before the upgrade could be valuable. The more details we have, the better we can assist.

Looking forward to your response!

Kind regards, The GitHub Community Support Team