search

divinity666 / ruby-grafana-reporter

Reporting Service for Grafana
MIT License
66 stars 5 forks source link

when the panel is a table,the resulting image is not complete in width and height. #21

Closed myselfghost closed 2 years ago

myselfghost commented 2 years ago

Describe the issue

when the panel is a table,the resulting image is not complete in width and height. I set the width value in the template and it doesn't seem to work.

I want to report a table to pdf ,Is there any good solution？ Template file (if applicable)

grafana_panel_image:11[dashboard="wVCbu81nz",width="1200",height="500",var-DS_PROMETHEUS="xxxxxx]

Environment (please complete the following information):

divinity666 commented 2 years ago

Thanks for using the reporter.

You are trying to add a table as image. Please note, that you can also add it as a "real" table by using the other grafana_panel function calls.

To fix your issue, try using the render-width and render-height options.

myselfghost commented 2 years ago

Thanks for using the reporter.

You are trying to add a table as image. Please note, that you can also add it as a "real" table by using the other grafana_panel function calls.

To fix your issue, try using the render-width and render-height options.

can not find function grafana_panel, can you show me how to use ,thank you .

I use 

include::grafana_panel_query_table:2[query="A",dashboard="dH2etRfnk"]
D, [2022-02-18T13:51:52.051967 #765] DEBUG -- : Document attributes: {"convert-backend"=>"pdf", "imagesdir"=>"./images", "var-template"=>"a.adoc"}
D, [2022-02-18T13:51:52.052121 #765] DEBUG -- : Template ./templates/a.adoc contains 1 calls of grafana reporter functions.
D, [2022-02-18T13:51:52.054217 #765] DEBUG -- : Processing PanelQueryTableIncludeProcessor (instance: default, dashboard: dH2etRfnk, panel: 2, query: A)
D, [2022-02-18T13:51:52.054418 #765] DEBUG -- : Requesting http://10.156.128.21:3000/api/frontend/settings with '' and timeout '60'
D, [2022-02-18T13:51:52.213294 #765] DEBUG -- : Received response #<Net::HTTPOK:0x00005590eff5f6d8>
D, [2022-02-18T13:51:52.220095 #765] DEBUG -- : Requesting http://10.156.128.21:3000/api/dashboards/uid/dH2etRfnk with '' and timeout '60'
D, [2022-02-18T13:51:52.309884 #765] DEBUG -- : Received response #<Net::HTTPOK:0x00005590f0542a28>
D, [2022-02-18T13:51:52.312874 #765] DEBUG -- : Requesting http://10.156.128.21:3000/api/datasources/proxy/1/api/v1/query?time=1645192311000&query=100-+100%2A+min_over_time%28node_memory_MemAvailable_bytes%7Bidc%3D%22tencent-cloud%22%2Csystem%3D%22k8s-2b%22%7D+%5B30d%3A5m%5D%29+%2Fnode_memory_MemTotal_bytes%7Bidc%3D%22tencent-cloud%22%2Csystem%3D%22k8s-2b%22%7D with '' and timeout '60'
D, [2022-02-18T13:51:52.399402 #765] DEBUG -- : Received response #<Net::HTTPOK:0x00005590effccc10>
E, [2022-02-18T13:51:52.400131 #765] ERROR -- : GrafanaReporterError: The datasource request to 'Prometheus-10.156.128.93' (Grafana::PrometheusDatasource) failed with an internal error: undefined method `[]' for nil:NilClass
divinity666 commented 2 years ago

Looks like there is an internal error, i.e. An issue in the reporter.

currently I have no chance to check it. Maybe go ahead with the image processor, as specified at first.

divinity666 commented 2 years ago

Maybe a few more things to check in the meantime:

1) Could you provide a screenshot of the panel in the dashboard, where the query succeeds? 2) What happens if you try using grafana_panel_query_value with the same panel request? 3) Try to call the last http request from your debug log in the browser? Can you share the results (x-out for potentially private information)? 4) If you still want to render it as a image, try using render-width and render-height.

myselfghost commented 2 years ago

Maybe a few more things to check in the meantime:

  1. Could you provide a screenshot of the panel in the dashboard, where the query succeeds?
  2. What happens if you try using grafana_panel_query_value with the same panel request?
  3. Try to call the last http request from your debug log in the browser? Can you share the results (x-out for potentially private information)?
  4. If you still want to render it as a image, try using render-width and render-height.

render-width and render-height work fine

screenshot of the panel in the dashboard： https://img.400511.net/2022/02/21/s6moLtLk.png

=== 1:

include::grafana_panel_query_value:11[query="A",dashboard="wVCbu81nz",job="consul-node-exporter",idc="tencent-cloud",system="westmoney",environment="pro"]

=== 2:

include::grafana_panel_query_table:11[query="A",dashboard="wVCbu81nz",job="consul-node-exporter",idc="tencent-cloud",system="westmoney",environment="pro"]

=== 3:

grafana_panel_image:11[dashboard="wVCbu81nz",from="now-30d",to="now",render-width="1200",render-height="500",var-DS_PROMETHEUS="Prometheus-10.156.128.93",var-job="consul-node-exporter",var-idc="tencent-cloud",var-system="quote",var-environment="pro"]
I, [2022-02-21T09:36:27.757273 #2516]  INFO -- : Report started at 2022-02-21 09:36:27 +0800
I, [2022-02-21T09:36:27.757459 #2516]  INFO -- : You are running ruby-grafana-reporter version 0.5.1.
D, [2022-02-21T09:36:27.757573 #2516] DEBUG -- : Document attributes: {"convert-backend"=>"pdf", "imagesdir"=>"./images", "var-template"=>"b"}
D, [2022-02-21T09:36:27.757783 #2516] DEBUG -- : Template ./templates/b.adoc contains 3 calls of grafana reporter functions.
D, [2022-02-21T09:36:27.758832 #2516] DEBUG -- : Processing PanelQueryTableIncludeProcessor (instance: default, dashboard: wVCbu81nz, panel: 11, query: A)
D, [2022-02-21T09:36:27.759037 #2516] DEBUG -- : Requesting https://monitor.sxftwork.com/api/frontend/settings with '' and timeout '60'
D, [2022-02-21T09:36:29.589283 #2516] DEBUG -- : Received response #<Net::HTTPOK:0x00000000083d9db8>
D, [2022-02-21T09:36:29.592692 #2516] DEBUG -- : Requesting https://monitor.sxftwork.com/api/dashboards/uid/wVCbu81nz with '' and timeout '60'
D, [2022-02-21T09:36:29.870558 #2516] DEBUG -- : Received response #<Net::HTTPOK:0x00000000082b06d0>
D, [2022-02-21T09:36:29.874595 #2516] DEBUG -- : Requesting https://monitor.sxftwork.com/api/datasources/proxy/1/api/v1/query?time=1645407386000&query=100-+100%2A+min_over_time%28node_memory_MemAvailable_bytes%7Bidc%3D%22tencent-cloud%22%2Csystem%3D%22devops%22%7D+%5B30d%3A5m%5D%29+%2Fnode_memory_MemTotal_bytes%7Bidc%3D%22tencent-cloud%22%2Csystem%3D%22devops%22%7D with '' and timeout '60'
D, [2022-02-21T09:36:30.089998 #2516] DEBUG -- : Received response #<Net::HTTPOK:0x000000000821d600>
E, [2022-02-21T09:36:30.092252 #2516] ERROR -- : GrafanaReporterError: The datasource request to 'Prometheus-10.156.128.93' (Grafana::PrometheusDatasource) failed with an internal error: undefined method `[]' for nil:NilClass
D, [2022-02-21T09:36:30.144786 #2516] DEBUG -- : Processing PanelQueryValueInlineMacro (instance: default, dashboard: , panel: 11, query: )
D, [2022-02-21T09:36:30.145665 #2516] DEBUG -- : Requesting https://monitor.sxftwork.com/api/dashboards/uid/ with '' and timeout '60'
D, [2022-02-21T09:36:30.402191 #2516] DEBUG -- : Received response #<Net::HTTPNotFound:0x0000000008d57480>
D, [2022-02-21T09:36:30.402919 #2516] DEBUG -- : HTTP response body: {
  "message": "Not found"
}

E, [2022-02-21T09:36:30.403345 #2516] ERROR -- : GrafanaError: The specified dashboard '' does not exist. (Grafana::DashboardDoesNotExistError)
D, [2022-02-21T09:36:30.409063 #2516] DEBUG -- : Processing PanelImageInlineMacro (instance: default, dashboard: wVCbu81nz, panel: 11)
D, [2022-02-21T09:36:30.410224 #2516] DEBUG -- : Requesting https://monitor.sxftwork.com/render/d-solo/wVCbu81nz?panelId=11&fullscreen=true&theme=light&timeout=60&var-DS_PROMETHEUS=Prometheus-10.156.128.93&var-job=consul-node-exporter&var-idc=tencent-cloud&var-environment=pro&var-system=quote&var-node=10.156.160.3%3A9100&var-template=b&width=1200&height=500&from=1642815387000&to=1645407386000 with '' and timeout '60'
D, [2022-02-21T09:36:36.901169 #2516] DEBUG -- : Received response #<Net::HTTPOK:0x0000000008c90088>
I, [2022-02-21T09:36:37.077916 #2516]  INFO -- : Report creation ended after 10 seconds with status 'finished'

http request result：

{"status":"success","data":{"resultType":"vector","result":[]}}
divinity666 commented 2 years ago

Thanks for your detailed feedback. Let's go through it step by step:

1) render-width and render-height are working as expected, which solves your initial case, right? 2) The test case with grafana_panel_query_value is unsuccessfull, due to a malformed request. Try the following in your template file: grafana_panel_query_value::11[query="A",dashboard="wVCbu81nz",job="consul-node-exporter",idc="tencent-cloud",system="westmoney",environment="pro"]

(no include and double colons) 3) The test case with grafana_panel_query_table is unsuccessfull, due to an internal error:

E, [2022-02-21T09:36:30.092252 #2516] ERROR -- : GrafanaReporterError: The datasource request to 'Prometheus-10.156.128.93' (Grafana::PrometheusDatasource) failed with an internal error: undefined method `[]' for nil:NilClass

Considering the response of the HTML request, it looks like, there is an issue in building the HTTP url in the reporter. That would have to be checked in detailed. If you see any chance to provide the URL that is generated and called by grafana, feel free to post it here.

If you are able to reproduce the issue e.g. on play.grafana.org, let me know how, and I can dig deeper.

divinity666 commented 2 years ago

Another thing you may try:

Toggle the Instant setting of your configuration and save the dashboard. Execute the report again and post the results.

myselfghost commented 2 years ago

Another thing you may try:

Toggle the Instant setting of your configuration and save the dashboard. Execute the report again and post the results.

grafana: Format: table , Instant: true, the same issue.

grafana_panel_query_value::11[query="A",dashboard="wVCbu81nz"]
 [2022-03-01T09:47:27.208015 #13808] DEBUG -- : Processing PanelQueryValueInlineMacro (instance: default, dashboard: wVCbu81nz, panel: :11, query: A)
E, [2022-03-01T09:47:27.208850 #13808] ERROR -- : GrafanaError: The specified panel id ':11' does not exist on the dashboard 'wVCbu81nz'. (Grafana::PanelDoesNotExistError)
D, [2022-03-01T09:47:27.214867 #13808] DEBUG -- : Processing PanelImageInlineMacro (instance: default, dashboard: wVCbu81nz, panel: 11)
https://monitor.sxftwork.com/api/datasources/proxy/1/api/v1/query?time=1646099241000&query=100-+100%2A+min_over_time%28node_memory_MemAvailable_bytes%7Bidc%3D%22tencent-cloud%22%2Csystem%3D%22devops%22%7D+%5B30d%3A5m%5D%29+%2Fnode_memory_MemTotal_bytes%7Bidc%3D%22tencent-cloud%22%2Csystem%3D%22devops%22%7D

http request result is null, so there is an issue in building the HTTP url in the reporter.

divinity666 commented 2 years ago

Thanks for your patience. I currently try to hunt this one down, but I'm still having issues to reproduce it.

Could you maybe post the panel JSON from grafana? This should help me to understand your panel configuration in detail.

myselfghost commented 2 years ago

Thanks for your patience. I currently try to hunt this one down, but I'm still having issues to reproduce it.

Could you maybe post the panel JSON from grafana? This should help me to understand your panel configuration in detail.

{
  "id": 11,
  "gridPos": {
    "h": 18,
    "w": 24,
    "x": 0,
    "y": 0
  },
  "type": "table",
  "title": "Host Reporter",
  "transformations": [
    {
      "id": "seriesToColumns",
      "options": {
        "byField": "instance"
      }
    },
    {
      "id": "organize",
      "options": {
        "excludeByName": {
          "InstanceId": true,
          "InstanceId 2": true,
          "InstanceName": true,
          "InstanceName 2": true,
          "OsType": true,
          "OsType 2": true,
          "Time": true,
          "Time 2": true,
          "Time 3": true,
          "Time 4": true,
          "alertsource": true,
          "alertsource 2": true,
          "environment": true,
          "environment 2": true,
          "environment 3": true,
          "environment 4": true,
          "idc 1": true,
          "idc 2": true,
          "idc 3": true,
          "idc 4": true,
          "important": true,
          "important 2": true,
          "instanceid": true,
          "instanceid 2": true,
          "instancename 2": true,
          "instancename 3": true,
          "instancename 4": true,
          "job": true,
          "job 2": true,
          "monitor_group": true,
          "monitor_group 2": true,
          "no_mem_warn 1": true,
          "no_mem_warn 2": true,
          "ostype": true,
          "ostype 2": true,
          "system 2": true,
          "system 3": true,
          "system 4": true
        },
        "indexByName": {
          "InstanceId": 1,
          "InstanceName": 2,
          "OsType": 3,
          "Time": 0,
          "Value": 13,
          "alertsource": 4,
          "environment": 6,
          "idc": 7,
          "instance": 5,
          "instanceid": 8,
          "instancename": 9,
          "job": 10,
          "ostype": 11,
          "system": 12
        },
        "renameByName": {
          "Value": "mem_max",
          "Value #A": "mem_max(%)",
          "Value #B": "mem_avg(%)",
          "Value #C": "cpu_max(%)",
          "Value #D": "cpu_avg(%)",
          "instance": "IP"
        }
      }
    }
  ],
  "pluginVersion": "8.3.5",
  "fieldConfig": {
    "defaults": {
      "custom": {
        "align": "center",
        "displayMode": "auto"
      },
      "thresholds": {
        "mode": "absolute",
        "steps": [
          {
            "color": "green",
            "value": null
          }
        ]
      },
      "mappings": [],
      "color": {
        "mode": "thresholds"
      },
      "max": 100,
      "min": 0
    },
    "overrides": [
      {
        "matcher": {
          "id": "byName",
          "options": "mem_avg(%)"
        },
        "properties": [
          {
            "id": "thresholds",
            "value": {
              "mode": "percentage",
              "steps": [
                {
                  "color": "green",
                  "value": null
                },
                {
                  "color": "yellow",
                  "value": 80
                }
              ]
            }
          },
          {
            "id": "custom.displayMode",
            "value": "color-background"
          }
        ]
      },
      {
        "matcher": {
          "id": "byName",
          "options": "cpu_avg(%)"
        },
        "properties": [
          {
            "id": "thresholds",
            "value": {
              "mode": "percentage",
              "steps": [
                {
                  "color": "green",
                  "value": null
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            }
          },
          {
            "id": "custom.displayMode",
            "value": "color-background"
          }
        ]
      }
    ]
  },
  "options": {
    "showHeader": true,
    "footer": {
      "show": false,
      "reducer": [
        "sum"
      ],
      "fields": ""
    }
  },
  "targets": [
    {
      "datasource": {
        "type": "prometheus",
        "uid": "k3SQ5N_Gk"
      },
      "exemplar": false,
      "expr": "100- 100* min_over_time(node_memory_MemAvailable_bytes{idc=\"$idc\",system=\"$system\"} [30d:5m]) /node_memory_MemTotal_bytes{idc=\"$idc\",system=\"$system\"}",
      "format": "table",
      "instant": true,
      "interval": "",
      "legendFormat": "",
      "refId": "A"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "k3SQ5N_Gk"
      },
      "exemplar": false,
      "expr": "100- 100* avg_over_time(node_memory_MemAvailable_bytes{idc=\"$idc\",system=\"$system\"} [30d:10m]) /node_memory_MemTotal_bytes{idc=\"$idc\",system=\"$system\"}",
      "format": "table",
      "hide": false,
      "instant": true,
      "interval": "",
      "legendFormat": "",
      "refId": "B"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "k3SQ5N_Gk"
      },
      "exemplar": false,
      "expr": "100 - min_over_time(avg by(instance,system,idc,environment,instancename) (rate(node_cpu_seconds_total{mode=\"idle\",idc=\"$idc\",system=\"$system\"}[2m]))[30d:5m]) * 100",
      "format": "table",
      "hide": false,
      "instant": true,
      "interval": "",
      "legendFormat": "",
      "refId": "C"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "k3SQ5N_Gk"
      },
      "exemplar": false,
      "expr": "100 - avg_over_time(avg by(instance,system,idc,environment,instancename) (rate(node_cpu_seconds_total{mode=\"idle\",idc=\"$idc\",system=\"$system\"}[2m]))[30d:5m]) * 100",
      "format": "table",
      "hide": false,
      "instant": true,
      "interval": "",
      "legendFormat": "",
      "refId": "D"
    }
  ],
  "datasource": null
}
divinity666 commented 2 years ago

I just recognized, that Grafana changed the way of storing the JSON format and also of returning values (at least for some datasources).

I tried to fix these issues in release 0.5.2. Please checkout if your issue persists with that version.

If not, please feel free to reopen a new issue (we already mixed 2 issues in this report).

Thanks for your continuing support in finding the issues - your JSON helped a lot there.