search

Icinga / icingaweb2-module-puppetdb

PuppetDB module for Icinga Web 2 (ImportSource for Icinga Director)
12 stars 6 forks source link

imported_property table size #16

Closed marcelfischer closed 5 years ago

marcelfischer commented 6 years ago

We saw recently that the imported_property tablet inside the director mariadb database has a size of 15GB. We found out that most of that size comes from all of our puppet facts for each server. We have a puppetdb with around 2000 nodes in it. The core facts plus some (20 or so) custom facts.

Expected Behavior

I dont know if thats really a bug, but maybe this table or the facts import needs an improvement.

Current Behavior

This lead to a sitution where the backup of this database took 1,5 hours to complete. This lead to a icinga2 crash because database entries from icinga2 were stored in memory to fast and the out-of-memory killer came in.

Possible Solution

As a workaround we excluded this particular table with "--ignore-table=director.imported_property". Because the data inside isnt really necessary for a backup I guess.

Steps to Reproduce (for bugs)

  1. Use PuppetDB as import source
  2. Watch the imported_property table

Your Environment

Thomas-Gelf commented 5 years ago

@marcelfischer: please try to avoid using Import Sources with ever changing properties. Facts like uptime, free memory and similar are poison. Such volatile properties artificially inflate your database.

Best practice should be using resources of a custom defined type, like monitoring::host or similar. That way you can provide only the data you need, it also gives you more direct control over your created monitoring objects in your Puppet Manifests. We should document this somewhere, by the way ;-)

In case you want to shrink your DB please check/follow Director Issue #1920, I'll close this one.