agittins / bermuda

Bermuda Bluetooth/BLE Triangulation / Trilateration for HomeAssistant
MIT License
608 stars 15 forks source link

HA Database bloat after introduction of Bermuda_Global sensors #389

Open jeremysherriff opened 4 hours ago

jeremysherriff commented 4 hours ago

Configuration

Not applicable (I think?)

Describe the bug

Since v0.7 and the addition of the Global Bermuda device, my HA database has increased in size +15%. This seems to all be in the State table. My database is MariaDB so using phpMyAdmin I checked what the highest state counts are, grouped by entity_id:

select states_meta.entity_id,count(*) from states left join states_meta on states.metadata_id = states_meta.metadata_id group by entity_id order by count(*) desc;

entity_id                   count(*)    
sensor.bermuda_global_visible_device_count  151901  
sensor.bermuda_global_total_device_count     89860  
sensor.deck_sensor_humidity          31100  
sensor.atc_c280_humidity             30346  
sensor.deck_sensor_temperature           28670  
sensor.lywsd03mmc_7d73_humidity          26242  
sensor.atc_c280_temperature          24805  
sensor.lywsd03mmc_7d73_temperature       24085  
sensor.zm_memory_used                22791  
...

The sensor.bermuda_global_visible_device_count has a state change count that is an order of magnitude higher than most sensors.

I suspect a lot of the bloat is coming from the lack of state_class attributes for these sensors, which causes every state change to be recorded as a discrete value rather than being aggregated: image

I think the following attribute should be added to these sensors:

state_class: measurement

I have now manually added these attributes using the customize.yaml file and will update on success of this.

Alternatively, these sensors should perhaps be disabled by default?

Diagnostics

Not applicable (I think?)

jeremysherriff commented 2 hours ago

Update; My edits to customize.yaml did what I was hoping from the point of view that the states are now seen as numbers and can be aggregated and fed into the statistics engine. Whether this causes the state changes to be stored more efficiently I am unsure (but I have some other very high-change stats that do not bloat the database like these were, so I am quietly confident...) image