grafana-toolbox / grafana-snapshots-tool

Python tool using Grafana API to generate fully autonomous snapshots that can be imported or exported as JSON files to external Grafana.
Apache License 2.0
13 stars 5 forks source link

unhashable type: 'dict' #2

Open aerohygenx opened 1 year ago

aerohygenx commented 1 year ago

Hi, I got the following error `grafana-snapshots -d "dash-sandbox"

datasources OK.

panel: {'datasource': {'type': 'influxdb', 'uid': '5HyLKpank'}, 'description': 'the business id: ${business}\nthe account is: ${location}', 'fieldConfig': {'defaults': {'color': {'mode': 'palette-classic'}, 'custom': {'axisCenteredZero': False, 'axisColorMode': 'text', 'axisLabel': '', 'axisPlacement': 'auto', 'barAlignment': 0, 'drawStyle': 'line', 'fillOpacity': 0, 'gradientMode': 'none', 'hideFrom': {'legend': False, 'tooltip': False, 'viz': False}, 'lineInterpolation': 'linear', 'lineWidth': 1, 'pointSize': 5, 'scaleDistribution': {'type': 'linear'}, 'showPoints': 'auto', 'spanNulls': False, 'stacking': {'group': 'A', 'mode': 'none'}, 'thresholdsStyle': {'mode': 'off'}}, 'mappings': [], 'thresholds': {'mode': 'absolute', 'steps': [{'color': 'green', 'value': None}, {'color': 'red', 'value': 80}]}, 'unit': 's'}, 'overrides': []}, 'gridPos': {'h': 8, 'w': 14, 'x': 0, 'y': 0}, 'id': 6, 'options': {'legend': {'calcs': ['count'], 'displayMode': 'list', 'placement': 'bottom', 'showLegend': True}, 'tooltip': {'mode': 'single', 'sort': 'none'}}, 'pluginVersion': '9.1.0-76166', 'targets': [{'datasource': {'type': 'influxdb', 'uid': '5HyLKpank'}, 'query': 'from(bucket: "testbucket")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r["_measurement"] == "rrm2")\n |> filter(fn: (r) => r["business"] == "${business}")\n |> filter(fn: (r) => r["location"] == "${location}")\n |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n |> group(columns: [], mode:"by") \n |> drop(columns: ["_measurement", "topic", "location", "business", "uuid", "_start", "_stop"])\n |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}], 'title': 'test sanitization duration for: ${space}', 'type': 'timeseries'}

dtsrc: {'type': 'influxdb', 'uid': '5HyLKpank'}

target: [{'datasource': {'type': 'influxdb', 'uid': '5HyLKpank'}, 'query': 'from(bucket: "testbucket")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r["_measurement"] == "rrm2")\n |> filter(fn: (r) => r["business"] == "${business}")\n |> filter(fn: (r) => r["location"] == "${location}")\n |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n |> group(columns: [], mode:"by") \n |> drop(columns: ["_measurement", "topic", "location", "business", "uuid", "_start", "_stop"])\n |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}]

Traceback (most recent call last): File "/Users/aerojuan/.pyenv/versions/3.9.10/bin/grafana-snapshots", line 8, in sys.exit(main()) File "/Users/aerojuan/.pyenv/versions/3.9.10/lib/python3.9/site-packages/grafana_snapshots/cli.py", line 268, in main res = data_api.get_dashboard_data() File "/Users/aerojuan/.pyenv/versions/3.9.10/lib/python3.9/site-packages/grafana_snapshots/grafanaData.py", line 313, in get_dashboard_data if dtsrc in self.datasources and targets is not None:

TypeError: unhashable type: 'dict'`

The configuration yaml is:

` general: debug: true snapshotsuffix: "%Y%m%d%H%M" output_path: snapshots grafana: default: protocol: https host: my.grafana.net port: 443 token: "74777474783929" search_api_limit: 5000 verify_ssl: true contexts:

"dash-sandbox": time_from: "now-7d" time_to: "now"

vars:
  business: "UNKNOWN"
  location: "UNKNOWN"
  space: "ALL"

`

Thanks in advance for your help.

amotl commented 1 year ago

Hi @aerohygenx and @peekjef72,

apologies if this might be the wrong guess, but I believe the reason could be that more recent versions of Grafana populate the datasource field with a dictionary instead of a bare name, like

'datasource': {'type': 'influxdb', 'uid': '5HyLKpank'}

We had to fix grafana-wtf correspondingly with https://github.com/panodata/grafana-wtf/commit/3d7dc3c8 and https://github.com/panodata/grafana-wtf/commit/6e3b440efb1.

With kind regards, Andreas.

aerohygenx commented 1 year ago

Hi @amotl , Thank you for your feedback. I Hope @peekjef72 can confirm your assessment and fix soon. Regards, Juan

peekjef72 commented 1 year ago

Hi @amotl , @aerohygenx

Andreas was right ! the pb was with datasources return type. I haven't already upgrade to Grafana v 9 (because I'm still using Grafana 7.5 due to some panels that can't upgrade), so this problem has not occurred for me. I've just released v 0.2.1 that have fixed the datasource pb. It fixes too when the dashboard global datasource it set to "mixed", a use case that I'd never checked before. It seems that the format of the snapshot has changed with new version of Grafana: the tool generates valid snapshots but the rendering for some panels is not exactly the same that it is for the "live" view : I've have observed that the axis-y autoscale is not working. I'll create a separate issue for this. I hope that I can fix that part very soon.

peekjef72 commented 1 year ago

@aerohygenx when you have tested, tell me it is ok to close this issue.

aerohygenx commented 1 year ago

Hi Jean-Francois, Thanks for the update. I plan to run the test tomorrow Monday; I’ll let you know my results as soon as possible. Regards, Juan

From: Pik Jean-Francois @.> Date: Sunday, September 11, 2022 at 10:24 AM To: peekjef72/grafana-snapshots-tool @.> Cc: Juan Ore @.>, Mention @.> Subject: Re: [peekjef72/grafana-snapshots-tool] unhashable type: 'dict' (Issue #2)

@aerohygenxhttps://github.com/aerohygenx when you have tested, tell me it is ok to close this issue.

— Reply to this email directly, view it on GitHubhttps://github.com/peekjef72/grafana-snapshots-tool/issues/2#issuecomment-1242976179, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A26SITJGVRVBADUQNVMPFUDV5XTRLANCNFSM6AAAAAAQHB7RSA. You are receiving this because you were mentioned.Message ID: @.***>

aerohygenx commented 1 year ago

Good Day Jean_Francois, I’m getting the following error:

grafana-snapshots -d "RRM2-sandbox" -F "General"
datasources OK.
panel: {'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'description': 'the business id: ${business}\nthe account is:  ${location}', 'fieldConfig': {'defaults': {'color': {'mode': 'palette-classic'}, 'custom': {'axisCenteredZero': False, 'axisColorMode': 'text', 'axisLabel': '', 'axisPlacement': 'auto', 'barAlignment': 0, 'drawStyle': 'line', 'fillOpacity': 0, 'gradientMode': 'none', 'hideFrom': {'legend': False, 'tooltip': False, 'viz': False}, 'lineInterpolation': 'linear', 'lineWidth': 1, 'pointSize': 5, 'scaleDistribution': {'type': 'linear'}, 'showPoints': 'auto', 'spanNulls': False, 'stacking': {'group': 'A', 'mode': 'none'}, 'thresholdsStyle': {'mode': 'off'}}, 'mappings': [], 'thresholds': {'mode': 'absolute', 'steps': [{'color': 'green', 'value': None}, {'color': 'red', 'value': 80}]}, 'unit': 's'}, 'overrides': []}, 'gridPos': {'h': 8, 'w': 14, 'x': 0, 'y': 0}, 'id': 6, 'options': {'legend': {'calcs': ['count'], 'displayMode': 'list', 'placement': 'bottom', 'showLegend': True}, 'tooltip': {'mode': 'single', 'sort': 'none'}}, 'pluginVersion': '9.1.0-76166', 'targets': [{'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'query': 'from(bucket: "testjson2")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r["_measurement"] == "rrm2")\n  |> filter(fn: (r) => r["business"] == "${business}")\n  |> filter(fn: (r) => r["location"] == "${location}")\n  |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n  |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n  |> group(columns: [], mode:"by")  \n  |> drop(columns: ["_measurement", "topic", "location", "business", "uuid", "_start", "_stop"])\n  |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}], 'title': 'test sanitization duration for:  ${space}', 'type': 'timeseries'}
dtsrc: {'type': 'influxdb', 'uid': '5HyKKpank'}
target: [{'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'query': 'from(bucket: "testjson2")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r["_measurement"] == "rrm2")\n  |> filter(fn: (r) => r["business"] == "${business}")\n  |> filter(fn: (r) => r["location"] == "${location}")\n  |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n  |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n  |> group(columns: [], mode:"by")  \n  |> drop(columns: ["_measurement", "topic", "location", "business", "uuid", "_start", "_stop"])\n  |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}]

target expr is not defined: skipped!

panel: {'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'description': 'the business id: ${business}\nthe account is:  ${location}', 'fieldConfig': {'defaults': {'color': {'mode': 'thresholds'}, 'custom': {'align': 'center', 'displayMode': 'color-text', 'inspect': False}, 'mappings': [], 'thresholds': {'mode': 'absolute', 'steps': [{'color': 'green', 'value': None}, {'color': 'red', 'value': 80}]}}, 'overrides': [{'matcher': {'id': 'byName', 'options': '_start'}, 'properties': [{'id': 'custom.hidden', 'value': True}]}, {'matcher': {'id': 'byName', 'options': '_stop'}, 'properties': [{'id': 'custom.hidden', 'value': True}]}, {'matcher': {'id': 'byName', 'options': 'space'}, 'properties': [{'id': 'custom.hidden', 'value': True}]}, {'matcher': {'id': 'byName', 'options': 'runDuration'}, 'properties': [{'id': 'displayName', 'value': 'duration'}, {'id': 'unit', 'value': 's'}]}, {'matcher': {'id': 'byName', 'options': '_time'}, 'properties': [{'id': 'displayName', 'value': 'time'}]}]}, 'gridPos': {'h': 9, 'w': 12, 'x': 0, 'y': 8}, 'id': 2, 'options': {'footer': {'fields': '', 'reducer': ['count'], 'show': False}, 'showHeader': True, 'sortBy': [{'desc': True, 'displayName': 'time'}]}, 'pluginVersion': '9.1.3-5f01ebf', 'targets': [{'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'query': 'from(bucket: "testjson2")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r["_measurement"] == "rrm2")\n  |> filter(fn: (r) => r["business"] == "${business}")\n  |> filter(fn: (r) => r["location"] == "${location}")\n  |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n  |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n  |> group(columns: [], mode:"by")  \n  |> drop(columns: ["_measurement", "topic", "account", "business", "uuid", "_start", "_stop"])\n  |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}], 'title': 'test sanitization dates:  ${space}', 'type': 'table'}

dtsrc: {'type': 'influxdb', 'uid': '5HyKKpank'}

target: [{'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'query': 'from(bucket: "testjson2")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r["_measurement"] == "rrm2")\n  |> filter(fn: (r) => r["business"] == "${business}")\n  |> filter(fn: (r) => r["location"] == "${location}")\n  |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n  |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n  |> group(columns: [], mode:"by")  \n  |> drop(columns: ["_measurement", "topic", "account", "business", "uuid", "_start", "_stop"])\n  |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}]

target expr is not defined: skipped!

Traceback (most recent call last):
  File "/Users/aerojuan/.pyenv/versions/3.9.10/bin/grafana-snapshots", line 8, in <module>
    sys.exit(main())
  File "/Users/aerojuan/.pyenv/versions/3.9.10/lib/python3.9/site-packages/grafana_snapshots/cli.py", line 275, in main
    res = data_api.get_dashboard_data()
  File "/Users/aerojuan/.pyenv/versions/3.9.10/lib/python3.9/site-packages/grafana_snapshots/grafanaData.py", line 426, in get_dashboard_data
    , 'transformations': panel['transformations']
KeyError: 'transformations'

From: Juan Ore @.> Date: Sunday, September 11, 2022 at 3:49 PM To: peekjef72/grafana-snapshots-tool @.>, peekjef72/grafana-snapshots-tool @.> Cc: Mention @.> Subject: Re: [peekjef72/grafana-snapshots-tool] unhashable type: 'dict' (Issue #2) Hi Jean-Francois, Thanks for the update. I plan to run the test tomorrow Monday; I’ll let you know my results as soon as possible. Regards, Juan

From: Pik Jean-Francois @.> Date: Sunday, September 11, 2022 at 10:24 AM To: peekjef72/grafana-snapshots-tool @.> Cc: Juan Ore @.>, Mention @.> Subject: Re: [peekjef72/grafana-snapshots-tool] unhashable type: 'dict' (Issue #2)

@aerohygenxhttps://github.com/aerohygenx when you have tested, tell me it is ok to close this issue.

— Reply to this email directly, view it on GitHubhttps://github.com/peekjef72/grafana-snapshots-tool/issues/2#issuecomment-1242976179, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A26SITJGVRVBADUQNVMPFUDV5XTRLANCNFSM6AAAAAAQHB7RSA. You are receiving this because you were mentioned.Message ID: @.***>

aerohygenx commented 1 year ago

Hi Jean-Francois, Adding the result of the built of grafana-snapshots: Successfully built grafana-snapshots Installing collected packages: setuptools, grafana-snapshots Attempting uninstall: setuptools Found existing installation: setuptools 65.3.0 Uninstalling setuptools-65.3.0: Successfully uninstalled setuptools-65.3.0 Successfully installed grafana-snapshots-0.2.1 setuptools-39.2.0

aerohygenx commented 1 year ago

Hi Jean-Francois, My dashboard consists of two panels: first is a "timeseries" and second is a "panel". This time my dashboard contains only one "timeseries" panel and here is the result:

grafana-snapshots -d "RRM2-onepanel" -F "General" -f "now-30d" datasources OK.

panel: {'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'description': 'the business id: ${business}\nthe account is: ${location}', 'fieldConfig': {'defaults': {'color': {'mode': 'palette-classic'}, 'custom': {'axisCenteredZero': False, 'axisColorMode': 'text', 'axisLabel': '', 'axisPlacement': 'auto', 'barAlignment': 0, 'drawStyle': 'line', 'fillOpacity': 0, 'gradientMode': 'none', 'hideFrom': {'legend': False, 'tooltip': False, 'viz': False}, 'lineInterpolation': 'linear', 'lineWidth': 1, 'pointSize': 5, 'scaleDistribution': {'type': 'linear'}, 'showPoints': 'auto', 'spanNulls': False, 'stacking': {'group': 'A', 'mode': 'none'}, 'thresholdsStyle': {'mode': 'off'}}, 'mappings': [], 'thresholds': {'mode': 'absolute', 'steps': [{'color': 'green', 'value': None}, {'color': 'red', 'value': 80}]}, 'unit': 's'}, 'overrides': []}, 'gridPos': {'h': 8, 'w': 14, 'x': 0, 'y': 0}, 'id': 6, 'options': {'legend': {'calcs': ['count'], 'displayMode': 'list', 'placement': 'bottom', 'showLegend': True}, 'tooltip': {'mode': 'single', 'sort': 'none'}}, 'pluginVersion': '9.1.0-76166', 'targets': [{'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'query': 'from(bucket: "testjson2")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r["_measurement"] == "rrm2")\n |> filter(fn: (r) => r["business"] == "${business}")\n |> filter(fn: (r) => r["location"] == "${location}")\n |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n |> group(columns: [], mode:"by") \n |> drop(columns: ["_measurement", "topic", "location", "business", "uuid", "_start", "_stop"])\n |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}], 'title': 'test sanitization duration for: ${space}', 'type': 'timeseries'} dtsrc: {'type': 'influxdb', 'uid': '5HyKKpank'} target: [{'datasource': {'type': 'influxdb', 'uid': '5HyKKpank'}, 'query': 'from(bucket: "testjson2")\n\t|> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r["_measurement"] == "rrm2")\n |> filter(fn: (r) => r["business"] == "${business}")\n |> filter(fn: (r) => r["location"] == "${location}")\n |> filter(fn: (r) => r["_field"] == "space" or r["_field"] == "runDuration")\n |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") \n |> group(columns: [], mode:"by") \n |> drop(columns: ["_measurement", "topic", "location", "business", "uuid", "_start", "_stop"])\n |> filter(fn: (r) => if "${space}" != "ALL" then r["space"] == "${space}" else true)\n', 'refId': 'A'}] target expr is not defined: skipped! time_from = 2022-08-13T12:50:57 - time_to = 2022-09-12T12:50:57

OK: new snapshot 'RRM2-onepanel_202209121250' created.



I see an entry 'RRM2-onepanel_202209121250' in the snapshot but no data gets displayed in the snapshot link.
aerohygenx commented 1 year ago

here is the json file generated from the export: grafana-snapshots -d "RRM2-onepanel" -F "General" -f "now-30d" export

{ "dashboard": { "annotations": { "list": [ { "builtIn": 1, "enable": true, "hide": true, "iconColor": "rgba(0, 211, 255, 1)", "name": "Annotations & Alerts", "target": { "limit": 100, "matchAny": false, "tags": [], "type": "dashboard" }, "type": "dashboard" } ] }, "editable": true, "fiscalYearStartMonth": 0, "graphTooltip": 0, "id": 33, "links": [], "liveNow": false, "panels": [ { "datasource": { "type": "influxdb", "uid": "5HyKKpank" }, "description": "the business id: ${business}\nthe account is: ${location}", "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "drawStyle": "line", "fillOpacity": 0, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null }, { "color": "red", "value": 80 } ] }, "unit": "s" }, "overrides": [] }, "gridPos": { "h": 8, "w": 14, "x": 0, "y": 0 }, "id": 6, "options": { "legend": { "calcs": [ "count" ], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "mode": "single", "sort": "none" } }, "pluginVersion": "9.1.0-76166", "title": "test sanitization duration for: ${space}", "type": "timeseries" } ], "refresh": "", "schemaVersion": 37, "style": "dark", "tags": [], "templating": { "list": [ { "current": { "selected": true, "text": "UNKNOWN", "value": "UNKNOWN" }, "description": "Business code", "hide": 0, "includeAll": false, "multi": false, "name": "business", "options": [ { "selected": true, "text": "UNKNOWN", "value": "UNKNOWN" } ], "query": "UNKNOWN", "queryValue": "UNKNOWN", "skipUrlSync": false, "type": "custom" }, { "current": { "selected": false, "text": "UNKNOWN", "value": "UNKNOWN" }, "description": "One business may have multiple accounts; default value is \"main\"", "hide": 0, "includeAll": false, "multi": false, "name": "location", "options": [ { "selected": true, "text": "UNKNOWN", "value": "UNKNOWN" } ], "query": "UNKNOWN", "queryValue": "", "skipUrlSync": false, "type": "custom" }, { "current": { "selected": true, "text": "ALL", "value": "ALL" }, "description": "type space in upper case", "hide": 0, "includeAll": false, "multi": false, "name": "space", "options": [ { "selected": true, "text": "ALL", "value": "ALL" } ], "query": "ALL", "queryValue": "ALL", "skipUrlSync": false, "type": "custom" } ] }, "time": { "from": "2022-08-13T13:05:08", "to": "2022-09-12T13:05:08", "raw": { "from": "now-30d", "to": "now" } }, "timepicker": {}, "timezone": "", "title": "RRM2-onepanel", "uid": "l7Prk9GVk", "version": 2, "weekStart": "", "snapshot": { "originalUrl": "https://aerohygenxops.grafana.net:443/api/d/l7Prk9GVk/rrm2-onepanel?from=1660410308000&to=1663002308000", "timestamp": "2022-09-12T13:05:08.309823" } }, "name": "RRM2-onepanel_202209121305" }

peekjef72 commented 1 year ago

sorry for the delay. Hard week... well, I haven't got any Influxdb instance so never try before, ( and never at all !) it seems that targets are different for influxDB thant for prometheus: the JSON part for targets doesn't content the 'expr' attributes. I will try with a panel querying a postgres maybe it will be the same behavior.

amotl commented 1 year ago

It seems that targets are different for influxDB than for Prometheus. I will try with a panel querying a PostgreSQL, maybe it will have the same behavior.

I think you can expect every data source type to have a different behavior here (see references below), however there are some overlaps. I've exercised this aspect thoroughly when aiming to cover each data source type on submitting a health-check query to.

References

peekjef72 commented 1 year ago

Hi Andreas

Thank you. I've checked your code and make à lot of tests... things are quite more complicated than I though... I plan to make some emprovements to grafana_client model.py query_factory and datasource/smartquery(). Could you have a look to my fork please ?

Le lun. 19 sept. 2022, 15:17, Andreas Motl @.***> a écrit :

It seems that targets are different for influxDB than for Prometheus. I will try with a panel querying a PostgreSQL, maybe it will have the same behavior.

I think you can expect every data source type to have a different behavior here, however there are some overlaps. I've exercised this aspect thoroughly when aiming to cover each data source type on submitting a health-check query to. References

— Reply to this email directly, view it on GitHub https://github.com/peekjef72/grafana-snapshots-tool/issues/2#issuecomment-1251008404, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQGB3MNUZLFQ4C5ADA5PXTDV7BRXBANCNFSM6AAAAAAQHB7RSA . You are receiving this because you were mentioned.Message ID: @.***>

amotl commented 1 year ago

Dear Jean-Francois,

I plan to make some emprovements to grafana_client's model.py::query_factory and datasource/smartquery(). Could you have a look to my fork please?

You want me to have a look at https://github.com/panodata/grafana-client/compare/main...peekjef72:grafana-client:main, right?

I think you are aiming to generalize the data query interface beyond doing health-check queries only? Thanks a stack for working on those important details to fill the corresponding gaps to make that possible. Because my iteration time was constrained by other obligations, I was sad to leave those gaps in the code base, so I really appreciate it to see you picking up the torch on it.

I think the best way to work on that would be that you submit a pull request, so we can discuss the implementation there, and to stop hijacking this conversation. I am very much looking forward to your improvements. Please bear with me if I may take some time to respond.

When submitting the patch, please make sure to delete the smartquery_old function from the code altogether. In this way, the changes needed to be made to the original function will become clear immediately.

Thank you in advance and with kind regards, Andreas.

amotl commented 1 year ago

Dear Jean-Francois,

thanks a stack for submitting https://github.com/panodata/grafana-client/pull/38. I will need some time to review and respond, so I am asking for your understanding.

With kind regards, Andreas.

peekjef72 commented 1 year ago

Sure, no problem. It is a kind of first try. To start a discussion about this subject.

Le mar. 20 sept. 2022, 12:59, Andreas Motl @.***> a écrit :

Dear Jean-Francois,

thanks a stack for submitting panodata/grafana-client#38 https://github.com/panodata/grafana-client/pull/38. I will need some time to review and respond, so I am asking for your understanding.

With kind regards, Andreas.

— Reply to this email directly, view it on GitHub https://github.com/peekjef72/grafana-snapshots-tool/issues/2#issuecomment-1252186799, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQGB3MLXYJNHOC6FGIX3FM3V7GKKTANCNFSM6AAAAAAQHB7RSA . You are receiving this because you were mentioned.Message ID: @.***>

peekjef72 commented 1 year ago

Hi aerohygenx,

Sorry for the delay.

Could you please try the devs made in 0.3.0 branch on your dashboard/datasource ?

For that you must download locally booth 0.3.0 branch and grafana-client Pull Request. Then to test, set PYTHONPATH to the two directories. something like:

 export PYTHONPATH="PYTHONPATH:~/python/grafana-client:~/python/grafana-snapshots-tool"

and use python3 grafana-snapshots-tool/local_cli.py with standard arguments.

python3 local_cli.py --verbose -b ~/grafana_snapshots/ -d [dashboard_name]
amotl commented 1 year ago

Hi again,

on your last note, I would like to add an alternative way to install both packages within a virtualenv, using pip. In this manner, you will not have to manipulate PYTHONPATH, and the installation works as if running it through a regular pip install from PyPI, so you can use the grafana-snapshots program entrypoint as usual.

Setup

python3 -m venv .venv
source .venv/bin/activate
pip install git+https://github.com/peekjef72/grafana-snapshots-tool@0.3.0
pip install git+https://github.com/peekjef72/grafana-client@main

Usage

$ grafana-snapshots --version
grafana-snapshots 0.3.0

With kind regards, Andreas.

P.S.: Sorry again for not reviewing your PR https://github.com/panodata/grafana-client/pull/38 properly yet. I hope to be able to catch some time when we are back from our travels.

amotl commented 9 months ago

Hi again,

your improvements from https://github.com/panodata/grafana-client/pull/38 have been merged and released using https://github.com/grafana-toolbox/grafana-client/pull/112. Thanks again, and apologies for the long delay.

Maybe it can help on the matters discussed here? Please let me know if you need further support, by adding more patches to grafana-client in any way. I will try to be more responsive bringing in and releasing corresponding updates and improvements.

With kind regards, Andreas.

peekjef72 commented 9 months ago

Hi Andrea,

Thanks for your actions. I will try to find some time to review grafana-snapshot-tool code; Currently I have neither special needs nor negative feedbacks (nor good ones! ) concerning this tools. It works as dersired for me. I probably have to check the changes made on datasources' accesses with version 10, but I don't need them for now.

JFPIK-

amotl commented 1 month ago

Dear Jean-Francois,

do you think it makes sense to integrate your improvements from the 0.3.0 branch into mainline? If you think yes, let me know if you a) appreciate a review and b) if you want me to run the subsequent release [^1].

With kind regards, Andreas.

[^1]: Before releasing, I would like to apply a few maintenance patches here and there.

peekjef72 commented 1 month ago

Hi Andreas,

Hope you are fine.

Yes, it makes sense but with restrictions :

So when I have tested in real life on a dashboard (the one that was at the origin of the tool!), the snapshot has failed because of transformations. :😞 For me this version 0.3.0 is an alpha. So, yes It would be nice if you could spend time on source code review, and maybe perform some tests or increase the scope of the test functions. But there is still more work to arrive at a final version.

JFPIK-

Le sam. 1 juin 2024 à 01:55, Andreas Motl @.***> a écrit :

Dear Jean-Francois,

do you think it makes sense to integrate your improvements from the 0.3.0 branch into mainline? If you think yes, let me know if you a) appreciate a review and b) if you want me to run the release, after applying a few maintenance tasks here and there.

With kind regards, Andreas.

— Reply to this email directly, view it on GitHub https://github.com/grafana-toolbox/grafana-snapshots-tool/issues/2#issuecomment-2143130465, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQGB3MJEGWJFB4PXWGOPZTTZFEEXZAVCNFSM6AAAAAAQHB7RSCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTGEZTANBWGU . You are receiving this because you were mentioned.Message ID: @.***>

amotl commented 1 month ago

Thanks for your swift reply. So, I am slotting it into my "medium priority" box, to be implemented when time permits, or if you will approach testing and functional finalizing, and ping me about that it would be time to have another release.

Actually, grafana-import also needs a release ;]. I approached it recently, but was distracted by other obligations again.