canonical / prometheus-hardware-exporter

Prometheus Hardware Exporter is an exporter for Hardware Observer
GNU General Public License v3.0
10 stars 9 forks source link

Flush cache while running ipmi commands #40

Closed sudeephb closed 7 months ago

sudeephb commented 1 year ago

Sometimes the outdated cache may cause error and cause the service to stop. We need to flush the cache when the service starts. The cache may be helpful to make the collector faster, so we can discuss further on where to and where not to flush the cache.

IPMI provides a -f command to flush cache while running commands.

pengwyn commented 8 months ago

We are encountering this on a customer at the moment. For whatever reason, the cache is out of date within 10 minutes of flushing the cache, so applying the flush at the start of the service would not be sufficient.

I found that an alternative to -f exists: --sdr-cache-recreate. This will recreate the cache only if necessary when the ipmi-sel or ipmimontoring commands are run.

I added --sdr-cache-recreate to this line and found that it worked for our customer.