Data aggregation for shorter sample times

TACC / tacc_stats

TACC Stats is an automated resource-usage monitoring and analysis package.

GNU Lesser General Public License v2.1

41 stars 15 forks source link

Open stephenlienharrell opened 1 year ago

stephenlienharrell commented 1 year ago

We should explore data aggregation to get our sample times much lower.

It is my thought to keep the data-points at 5 minutes, but store average/min/max/std over 10 samples (30 second sampling).

This would only increase the data by 4x, instead of 10x.

then we could graph with the average and min/max bars to give a more complete picture on what happened on the node.

We probably want to do this on-node.

stephenlienharrell commented 10 months ago

really a data reduction problem only, sampling size