JsBergbau
commented
4 years ago Some furhter information to get an impression how much space could be saved: I've converted a series of about 276000 datapoints from nanoseconds resolution to seconds resolution. the /data folder of influxdb was 509 MB bevor I've started. About 12h later, when all compactions should have been run size shrank to 467 MB. That's saving 42 MB or about 8 % for this series. I've counted all the datapoints. Currently there are about 73 million datapoints stored in this influxdb. So this table is only 0,3772 % of all datapoints and still 8 % disk space is saved. Since most of the other measurements in this influxdb are from Openhab and Openhab uses ms as time resolution the expected saved space should be in between from nanoseconds to seconds, so about half.
I know disk space is cheap this time so you could argument thats not worth it. However influxdb has a serious problem on 32 bit systems. When the database reaches about 3.6 GB. "When the TSM files approach this limit, Influx fails with "Cannot allocate memory" errors." from https://github.com/influxdata/influxdb/issues/10486 See also https://github.com/influxdata/influxdb/pull/12362
Since a lot OpenHab installations are on a RaspberryPI with 32 Bit we'll get to this limit sooner or later. So we should use all possibilities to extend this limit. When changing from ns resolution to seconds resolution saves so much space we should implement this feature saving the timestamps only at precision level of seconds.
In the influxdb persistence file a data point is stored at MilliSeconds precision. See InfluxDBPersistenceService.java Line 244
This should be configurable to seconds level. Seconds compared to milliseconds enable better compression with the build in compression functions and thus saves space. Resolution to ms is hardly needed.