search

home-assistant / home-assistant-notebooks

:notebook: Sample Jupyter Notebooks to explore Home Assistant data
https://data.home-assistant.io/
Other
63 stars 32 forks source link

Getting started note book errors #23

Open mrbubble62 opened 1 year ago

mrbubble62 commented 1 year ago 
%%time

df = db.fetch_all_sensor_data()

The returned Pandas dataframe has 50000 rows of data. CPU times: user 346 ms, sys: 48.3 ms, total: 395 ms Wall time: 457 ms

df = df[df['domain']=='sensor']
df = functions.generate_features(df)
df = functions.format_dataframe(df)

KeyError Traceback (most recent call last) File /usr/local/lib/python3.9/dist-packages/pandas/core/indexes/base.py:3803, in Index.get_loc(self, key, method, tolerance) 3802 try: -> 3803 return self._engine.get_loc(casted_key) 3804 except KeyError as err:

File /usr/local/lib/python3.9/dist-packages/pandas/_libs/index.pyx:138, in pandas._libs.index.IndexEngine.get_loc()

File /usr/local/lib/python3.9/dist-packages/pandas/_libs/index.pyx:165, in pandas._libs.index.IndexEngine.get_loc()

File pandas/_libs/hashtable_class_helper.pxi:5745, in pandas._libs.hashtable.PyObjectHashTable.get_item()

File pandas/_libs/hashtable_class_helper.pxi:5753, in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'domain'

The above exception was the direct cause of the following exception:

KeyError Traceback (most recent call last) Cell In [5], line 1 ----> 1 df = df[df['domain']=='sensor'] 2 df = functions.generate_features(df) 3 df = functions.format_dataframe(df)

File /usr/local/lib/python3.9/dist-packages/pandas/core/frame.py:3804, in DataFrame.getitem(self, key) 3802 if self.columns.nlevels > 1: 3803 return self._getitem_multilevel(key) -> 3804 indexer = self.columns.get_loc(key) 3805 if is_integer(indexer): 3806 indexer = [indexer]

File /usr/local/lib/python3.9/dist-packages/pandas/core/indexes/base.py:3805, in Index.get_loc(self, key, method, tolerance) 3803 return self._engine.get_loc(casted_key) 3804 except KeyError as err: -> 3805 raise KeyError(key) from err 3806 except TypeError: 3807 # If we have a listlike key, _check_indexing_error will raise 3808 # InvalidIndexError. Otherwise we fall through and re-raise 3809 # the TypeError. 3810 self._check_indexing_error(key)

KeyError: 'domain'

duncandoo commented 1 year ago

I've experienced the same error. The problem is the underlying database structure has been changed so the field 'domain' no longer contains data. In response, fetch_all_sensor_data() in detective/core.py has been rewritten so it no longer selects 'domain'. so the field doesn't exist in the data frame, and therefore trying to filter on it fails.

What I'm trying to work out is the best way to adapt the getting started note book to incorporate these changes. Any thoughts?