blackcowmoo / grafana-google-analytics-datasource

Grafana Google Analytics datasource
https://grafana.com/grafana/plugins/blackcowmoo-googleanalytics-datasource/
Apache License 2.0
35 stars 15 forks source link

Plugin health check failed #81

Closed asil-x closed 10 months ago

asil-x commented 10 months ago

I have follow and double check all introduction step but still receive plugin health check failed

Grafana a v10.1.0 & Google Analytics v0.2.0

here is my log: 2023-08-31 09:23:13 logger=context userId=1 orgId=1 uname=admin t=2023-08-31T07:23:13.716521425Z level=error msg="Plugin unavailable" error="plugin unavailable" remote_addr=172.17.0.1 traceID= 2023-08-31 09:23:13 logger=context userId=1 orgId=1 uname=admin t=2023-08-31T07:23:13.716696498Z level=error msg="Request Completed" method=GET path=/api/datasources/uid/e7ffe070-f3c7-458f-9eb2-4fed4471972e/health status=503 remote_addr=172.17.0.1 time_ms=1 duration=1.327593ms size=45 referer=http://localhost:3000/connections/datasources/edit/e7ffe070-f3c7-458f-9eb2-4fed4471972e handler=/api/datasources/uid/:uid/health

image

Google Analytics API and Google Analytics Reporting API are both installed:

image
lcc3108 commented 10 months ago

You need to attach the debug log.

It is estimated that there is a problem with the following link. https://github.com/blackcowmoo/grafana-google-analytics-datasource/issues/51#issuecomment-1691386878

asil-x commented 10 months ago

You need to attach the debug log.

It is estimated that there is a problem with the following link. #51 (comment)

I tried but still same result, here is my log 2023-08-31 10:02:52 logger=context userId=1 orgId=1 uname=admin t=2023-08-31T08:02:52.530606107Z level=error msg="Request Completed" method=GET path=/api/datasources/uid/d4721ac7-2d74-4bad-bf05-3071c2f5a1e5/health status=500 remote_addr=172.17.0.1 time_ms=12930 duration=12.930048263s size=53 referer=http://localhost:3000/connections/datasources/edit/d4721ac7-2d74-4bad-bf05-3071c2f5a1e5 handler=/api/datasources/uid/:uid/health 2023-08-31 10:02:52 logger=plugin.blackcowmoo-googleanalytics-datasource t=2023-08-31T08:02:52.530451579Z level=error msg="plugin process exited" path=/var/lib/grafana/plugins/blackcowmoo-googleanalytics-datasource/gpx_blackcowmoo-googleanalytics-datasource_linux_amd64 pid=54 error="exit status 2" 2023-08-31 10:02:52 logger=plugin.blackcowmoo-googleanalytics-datasource t=2023-08-31T08:02:52.879560581Z level=info msg=Profiler enabled=false

lcc3108 commented 10 months ago

The log you attached is not a "debug" log. However, it seems to be the same issue as #51.

I'm releasing version 0.2.1 to patch this issue and I'm waiting for your Grafana approval.

moovs commented 10 months ago

@lcc3108 maybe you remember on verion 0.2.0 I gor error Plugin Health Check Failed, after you updated to 0.2.1 version I tried this version and this error gone, but I received a new error:

Screenshot 2023-09-01 at 10 24 02

Whay do you think about it? Is it problem with my profile or something else?

lcc3108 commented 10 months ago

@moovs Which version of health check did you do, UA(GA3) or GA4?

Please attach a debug log to check the problem.

moovs commented 10 months ago

Yes, it's UA profile here is my debug logs:

logger=accesscontrol.service t=2023-09-01T08:36:25.439017523Z level=debug msg="fetch permissions from store" key=rbac-permissions-1-user-1
logger=accesscontrol.service t=2023-09-01T08:36:25.453395326Z level=debug msg="cache permissions" key=rbac-permissions-1-user-1
logger=accesscontrol.evaluator t=2023-09-01T08:36:25.45366309Z level=debug msg="matched scope" userscope=datasources:* targetscope=datasources:uid:Jbdfjksdf
logger=datasources t=2023-09-01T08:36:25.453886174Z level=debug msg="Received command to update data source" url=
logger=datasource t=2023-09-01T08:36:25.453904195Z level=debug msg="Applying default URL parsing for this data source type" type=blackcowmoo-googleanalytics-datasource url=
logger=datasource t=2023-09-01T08:36:25.453912569Z level=debug msg="Data source URL doesn't specify protocol, so prepending it with http:// in order to make it unambiguous" type=blackcowmoo-googleanalytics-datasource url=
logger=sqlstore.session t=2023-09-01T08:36:25.465154704Z level=debug msg="reusing existing session" transaction=true
logger=sqlstore.session t=2023-09-01T08:36:25.4763378Z level=debug msg="reusing existing session" transaction=true
logger=secrets.kvstore t=2023-09-01T08:36:25.493462945Z level=debug msg="got secret value" orgId=1 type=datasource namespace=" Google Analytics"
logger=sqlstore.session t=2023-09-01T08:36:25.499768181Z level=debug msg="reusing existing session" transaction=true
logger=sqlstore.session t=2023-09-01T08:36:25.50735888Z level=debug msg="reusing existing session" transaction=true
logger=secrets.kvstore t=2023-09-01T08:36:25.507834119Z level=debug msg="secret value updated" orgId=1 type=datasource namespace=" Google Analytics"
logger=sqlstore.transactions t=2023-09-01T08:36:25.50797674Z level=debug msg="skip committing the transaction because it belongs to a session created in the outer scope"
logger=sqlstore.transactions t=2023-09-01T08:36:25.507989836Z level=debug msg="skip committing the transaction because it belongs to a session created in the outer scope"
logger=secrets.kvstore t=2023-09-01T08:36:25.522798376Z level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=" Google Analytics"
logger=accesscontrol.service t=2023-09-01T08:36:25.728561784Z level=debug msg="using cached permissions" key=rbac-permissions-1-user-1
logger=accesscontrol.service t=2023-09-01T08:36:26.324817819Z level=debug msg="using cached permissions" key=rbac-permissions-1-user-1
logger=accesscontrol.evaluator t=2023-09-01T08:36:26.325106851Z level=debug msg="matched scope" userscope=datasources:* targetscope=datasources:uid:JbvB1dkIk
logger=secrets.kvstore t=2023-09-01T08:36:26.336598247Z level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=" Google Analytics"
logger=accesscontrol.service t=2023-09-01T08:36:26.546011835Z level=debug msg="using cached permissions" key=rbac-permissions-1-user-1
logger=datasources t=2023-09-01T08:36:26.546206285Z level=debug msg="Querying for data source via SQL store" uid=Jbljijf orgId=1
logger=secrets.kvstore t=2023-09-01T08:36:26.557896215Z level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=" Google Analytics"
logger=plugin.blackcowmoo-googleanalytics-datasource t=2023-09-01T08:36:26.667933897Z level=debug msg="UA GetAccountSummaries raw accounts" debug="[map[id:225841363 kind:analytics#accountSummary name:Default Account for Firebase]]"
logger=plugin.blackcowmoo-googleanalytics-datasource t=2023-09-01T08:36:26.668014326Z level=debug msg="UA GetAccountSummaries accounts" debug=null
logger=plugin.blackcowmoo-googleanalytics-datasource t=2023-09-01T08:36:26.668042791Z level=error msg="CheckHealth: Not Exist Valid Profile"
logger=context userId=1 orgId=1 uname=superadmin t=2023-09-01T08:36:26.668317754Z level=info msg="Request Completed" method=GET path=/api/datasources/uid/JbvB1dkIk/health status=400 remote_addr=172.25.0.1 time_ms=133 duration=133.068223ms size=67 referer=http://domain/datasources/edit/Jbdfkjk handler=/api/datasources/uid/:uid/health
logger=ngalert.scheduler t=2023-09-01T08:36:30.012143913Z level=debug msg="No changes detected. Skip updating"
lcc3108 commented 10 months ago

@moovs UA has a structure of account -> webproperty -> profile. You must have a profile to query the data.

logger=plugin.blackcowmoo-googleanalytics-datasource t=2023-09-01T08:36:26.667933897Z level=debug msg="UA GetAccountSummaries raw accounts" debug="[map[id:225841363 kind:analytics#accountSummary name:Default Account for Firebase]]"
logger=plugin.blackcowmoo-googleanalytics-datasource t=2023-09-01T08:36:26.668014326Z level=debug msg="UA GetAccountSummaries accounts" debug=null

According to the log, only one account without webproperty and profile(items[0]) The error message "Not Exist Valid Profile" was responded accordingly.

Based on the past post (https://github.com/blackcowmoo/grafana-google-analytics-datasource/issues/51#issuecomment-1689817760), all accounts are as follows.

{
  "kind": "analytics#accountSummaries",
  "username": "****",
  "totalResults": 4,
  "startIndex": 1,
  "itemsPerPage": 10,
  "items": [
    {
      "id": "2*****",
      "kind": "analytics#accountSummary",
      "name": "Default Account for Firebase",
      "webProperties": []
    },
    {
      "id": "2******",
      "kind": "analytics#accountSummary",
      "name": "***",
      "webProperties": []
    },
    {
      "id": "2***",
      "kind": "analytics#accountSummary",
      "webProperties": [
        {
          "kind": "analytics#webPropertySummary",
          "id": "UA-2******",
          "name": "******",
          "internalWebPropertyId": "2******",
          "level": "STANDARD",
          "websiteUrl": "http://1******",
          "profiles": [
            {
              "kind": "analytics#profileSummary",
              "id": "2*******",
              "name": "*",
              "type": "WEB"
            }
          ]
        }
      ]
    },
  ]
}

It seems that the permission of the service account placed in grafana can only be inquired about the first item. You must give permission to the Graffana service account on the Google Analytics page in the array of items[2].

moovs commented 10 months ago

I did all from your manual)) maybe do you need update your installation instruction?

Screenshot 2023-09-01 at 14 07 46
lcc3108 commented 10 months ago

Yes, you proceeded according to the manual. However, in the Google Analytics Setting section, only some accounts (GA accounts, not Google accounts) were conducted.

moovs commented 10 months ago

Do you mean that I should setup all of my google analytics accounts according your manual? Not only one account that I want to use in my grafana integration?

lcc3108 commented 10 months ago

@moovs Generate JWT file can be done only once. In the process above, one service account will be created.

You must give View permission to your service account by account in Google Analytics. However, you only need to do an account with a profile.(This is because you need a profile to accumulate data.) In your case, there are three UA accounts, but only one profile exists.

In other words, currently, data is accumulated only in the account below, so you only need to do the account below.

    {
      "id": "2***",
      "kind": "analytics#accountSummary",
      "webProperties": [
        {
          "kind": "analytics#webPropertySummary",
          "id": "UA-2******",
          "name": "******",
          "internalWebPropertyId": "2******",
          "level": "STANDARD",
          "websiteUrl": "http://1******",
          "profiles": [
            {
              "kind": "analytics#profileSummary",
              "id": "2*******",
              "name": "*",
              "type": "WEB"
            }
          ]
        }
      ]
    },

In my case, I have two Google Analytics accounts

image

You must grant permission to each account. It's like the following picture. image

image

Then you can look it up on Grafana as follows.

image

In your case, only 'Default Account for Firebase' has permissions that do not currently have a profile created. (Checked by log)

From version 0.2.1, you will get an error saying that you don't have the right account because you have excluded accounts that don't have a profile (data can't be retrieved) from the list.

PS Please note that data will not be accumulated in the UA (GA3) from 2023-07-01.

This is a change on Google Analytics.

lcc3108 commented 10 months ago

@asil-x 0.2.1 has been released.

If the same problem occurs in that version, please open this issue again