Currently Humio has no concept of data or storage tiering - this can quickly become an economic headache for large clusters. It should be possible to introduce tiers or classes of storage and have Humio move data based on a retention policy. An example could be :
data that is younger than X is stored on SSD / NVMe for fast ingest and queries
data that is older than Y should be stored on NLSAS
data that is older than Z should be moves to SATA
This scheme should be flexible enough to support storage tiers either as locally attached storage or dedicated nodes and provide some mechanisms to control the impact of data movement (like limiting CPU usage, IO, etc).
Currently Humio has no concept of data or storage tiering - this can quickly become an economic headache for large clusters. It should be possible to introduce tiers or classes of storage and have Humio move data based on a retention policy. An example could be :
This scheme should be flexible enough to support storage tiers either as locally attached storage or dedicated nodes and provide some mechanisms to control the impact of data movement (like limiting CPU usage, IO, etc).