search

doitintl / bigquery-grafana

Google BigQuery Datasource Plugin for Grafana. (NO LONGER MAINTAINED)
https://www.doit-intl.com/labs/
MIT License
242 stars 76 forks source link

"google: could not find default credentials" error on testing alerts only #369

Closed alexbiediger closed 2 years ago

alexbiediger commented 3 years ago

Bug Report

I've managed to successfully get several panels working, but I've been stuck on alerts for several days. My suspicion is alerts do not use my configured JWT authentication and instead force use GCE even though it is not configured/authorized/present.

Expected Behavior

When panel queries run successfully I would expect alert queries to do so as well.

Actual Behavior

Even though panel queries run successfully, when testing out alerts I get the following error messages:

  "firing": true,
  "state": "pending",
  "conditionEvals": " = true",
  "timeMs": "13.489ms",
  "error": "request handler error: failed to query data: Failed to query data: rpc error: code = Unavailable desc = error reading from server: EOF",
  "logs": [
    {
      "message": "Condition[0]: Query",
      "data": {
        "from": 1630983473329,
        "queries": [
          {
            "refId": "A",
            "model": {
              "dataset": "",
              "format": "time_series",
              "group": [],
              "location": "asia-southeast1",
              "metricColumn": "none",
              "orderByCol": "1",
              "orderBySort": "1",
              "project": "co-169315",
              "rawQuery": true,
              "rawSql": "WITH test AS (\n  SELECT *\n  FROM (SELECT\n          TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 0 DAY)) AS time, 'metric1' as metric, 5000 as value\n        UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY)), 'metric1', 4000\n        UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 2 DAY)), 'metric1', 3000\n        UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 3 DAY)), 'metric1', 2000\n        UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 4 DAY)), 'metric1', 1000\n\n        UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 0 DAY)), 'metric2', 5500\n        UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY)), 'metric2', 4500\n        UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 2 DAY)), 'metric2', 3500\n        UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 3 DAY)), 'metric2', 2500\n        UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 4 DAY)), 'metric2', 1500\n  )\n) SELECT * from test \n\n",
              "refId": "A",
              "select": [
                [
                  {
                    "params": [
                      "-- value --"
                    ],
                    "type": "column"
                  }
                ]
              ],
              "timeColumn": "-- time --",
              "timeColumnType": "TIMESTAMP",
              "where": [
                {
                  "name": "$__timeFilter",
                  "params": [],
                  "type": "macro"
                }
              ]
            },
            "datasource": {
              "id": 1,
              "name": "Google BigQuery"
            },
            "maxDataPoints": 0,
            "intervalMs": 0
          }
        ],
        "to": 1630983773329
      }
    }
  ]
}
t=2021-09-07T03:00:48+0000 lvl=eror msg="query BigQueryRun error" logger=plugins.backend pluginId=doitintl-bigquery-datasource Error="bigquery.NewClient: bigquery: constructing client: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information."
t=2021-09-07T03:00:48+0000 lvl=eror msg="Alert Rule Result Error" logger=alerting.evalContext ruleId=0 name="Panel Title alert" error="request handler error: failed to query data: Failed to query data: rpc error: code = Unavailable desc = error reading from server: EOF" changing state to=alerting

Steps to Reproduce the Problem

  1. docker run -e "GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=doitintl-bigquery-datasource" -d -p 3000:3000 grafana/grafana
  2. docker exec [...] grafana-cli --pluginUrl https://github.com/doitintl/bigquery-grafana/archive/2.0.2.zip plugins install doitintl-bigquery-datasource
  3. docker start/stop [...]
  4. Add JWT token

  5. Create new panel using test query 

    WITH test AS (
SELECT *
FROM (SELECT
      TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 0 DAY)) AS time, 'metric1' as metric, 5000 as value
    UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY)), 'metric1', 4000
    UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 2 DAY)), 'metric1', 3000
    UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 3 DAY)), 'metric1', 2000
    UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 4 DAY)), 'metric1', 1000

    UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 0 DAY)), 'metric2', 5500
    UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY)), 'metric2', 4500
    UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 2 DAY)), 'metric2', 3500
    UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 3 DAY)), 'metric2', 2500
    UNION ALL SELECT TIMESTAMP(DATE_SUB(CURRENT_DATE(), INTERVAL 4 DAY)), 'metric2', 1500
)
) SELECT * from test
  6. Panel query goes through, graph/table view work well
  7. Create basic alert, error comes in.

Specifications

kooinam-ridebeam commented 2 years ago

I've run into similar issue as well. Do we have any update regarding this?

ofir5300 commented 2 years ago

Please update to v2.0.3 (Available on Grafana cloud) and let me know if it persist

ofir5300 commented 2 years ago

We are retiring the DoiT International BigQuery DataSource plugin for Grafana. We recommend that you migrate to the official Google BigQuery data source maintained by Grafana. In August 2022, we will cease all development and archive the GitHub project.

We are sorry for any inconvenience.