grafana-toolbox / grafana-wtf

Grep through all Grafana entities in the spirit of git-wtf.
GNU Affero General Public License v3.0
144 stars 13 forks source link

TypeError: unhashable type: 'list' in collect_datasource_items #62

Closed mauhiz closed 1 year ago

mauhiz commented 1 year ago

Env

both "explore datasources" and "explore dashboards" command fail:

[grafana_wtf.core      ] INFO   : Found 185 data source(s)
Traceback (most recent call last):
  File "~/grafana_wtf/commands.py", line 237, in run
    results = engine.explore_datasources()
  File "~/grafana_wtf/core.py", line 406, in explore_datasources
    ix = Indexer(engine=self)
  File "~/grafana_wtf/core.py", line 492, in __init__
    self.index()
  File "~/grafana_wtf/core.py", line 496, in index
    self.index_dashboards()
  File "~/grafana_wtf/core.py", line 543, in index_dashboards
    ds_templating = self.collect_datasource_items(dbdetails.templating)
  File "~/grafana_wtf/core.py", line 514, in collect_datasource_items
    datasource = self.datasource_by_name.get(ds_name, {})
TypeError: unhashable type: 'list'

As I see in #44 that 9.3.1 is not officially supported yet, I am assuming it has something to do with Grafana compatibility. Please let me know if there is something I can do to help more.

amotl commented 1 year ago

Dear Vincent,

thank you for writing in and for reporting this issue.

I am assuming it has something to do with Grafana compatibility.

That's probably true. Apparently, Grafana may have changed some bits within their dashboard structure, which will have to get compensated similar to GH-27 and GH-32.

Maybe you can share a sample of the corresponding datasource or dashboard JSON representations, where you may expect the root cause of this?

On this matter, see also ^1, where we maintain a set of dashboard variants through different versions where breaking changes occurred in the past.

As I see in https://github.com/panodata/grafana-wtf/pull/44 that 9.3.1 is not officially supported yet.

It might not be fully supported yet, because the test suite does not pass completely, but we still should aim for reasonable compatibility, regardless of this specific regression we observed.

With kind regards, Andreas.

mauhiz commented 1 year ago

Hi Andreas, thank for your quick reply. I will try to minimize a reproducer

mauhiz commented 1 year ago

I'm having trouble isolating the exact issue (and not familiar enough with Python) but I was able to get it to print the state while seeing the invalid input :

node=Munch({'current': Munch({'selected': True, 'text': ['All'], 'value': ['$__all']}), 'hide': 0, 'includeAll': True, 'multi': True, 'name': 'datasource', 'options': [], 'query': 'prometheus', 'queryValue': '', 'refresh': 1, 'regex': '/.*-storage$/', 'skipUrlSync': False, 'type': 'datasource'})

After get("current")

current_node=Munch({'selected': True, 'text': ['All'], 'value': ['$__all']})

After get("value")

ds_name=['$__all']

It seems to be related to defining a template variable that matches the following aspects

amotl commented 1 year ago

Dear Vincent,

thanks a stack for providing corresponding debug information. From those details, I can imagine that the problem is related to the recent addition of the feature »Improve discovery of data sources defined by dashboard variables« (GH-60) with version 0.14.0.

Can you try downgrading to version 0.13.4 using pip install grafana-wtf==0.13.4 and report back if that improves the situation for you, until we will come up with a corresponding fix?

With kind regards, Andreas.

amotl commented 1 year ago

Dear Vincent,

we just released grafana-wtf-0.14.1 with the fix on behalf of GH-63. We will be happy to hear back if that will resolve the problem for you. Thanks again for the excellent report.

With kind regards, Andreas.

mauhiz commented 1 year ago

Hi Andreas,

Thank you for your excellent and timely support. I confirmed that the problem is solved for me.