Chunk objects too memory inefficient

[Dieterbe]

we wrap go-tsz Series objects with Chunk objects which provide some metadata, see https://github.com/raintank/raintank-metric/blob/master/metric_tank/chunk.go#L11

these account for 7% of memory usage only, so not super urgent, but still worth looking into.

here's what the fields are for:

*tsz.Series // pointer to Series object.
T0        uint32 // unix ts divisble without remainder by chunkSpan (which is usually 600 or a multiple of that)
LastTs    uint32 // TS of last point seen.
NumPoints uint32 // points contained in Series. 
Saved     bool
Saving    bool 
LastWrite uint32 // unix ts of when last point was received

some thoughts:

• some of these values (LastWrite) are guaranteed to be >= server-start time. so we could store server start time once globally, and keep LastWrite as the delta between last write and server start, in a more compact int
• Saving and Saved bools are not orthogonal: we only really have 3 states: not saved/saving -> saving -> saved
• LastTs is for checking time order (we don't allow writing a point with timestamp <= last timestamp). we probably don't need to track this in Chunk, since Series keeps this in its .t property. maybe we could have Series enforce the time-ordering and possibly return error in Push(), or have Series expose this as a public variable or through a getter function.
• we could also store LastTs as a delta against T0 (possibly in go-tsz.Series, not in Chunk). this delta should always be a small number as our chunks have a limited span of up to 2/6 hours (later maybe up to 24)
• NumPoints: I think we can assume/enforce an upper limit of 1000 or so
• note that tsz.Series is a pointer and that we also refer to Chunks through pointers. I think we can probably either use Chunk values directly (without pointers) and keep the tsz.Series pointer, or keep using Chunks through pointers, but embed the Series as a value (without pointer). but i'm not sure.
  i also don't know if this useful at all.

I wonder what @dgryski thinks of these ideas and maybe he can come up with something better! we also don't want to do too much packing&math, because a bunch of these values are checked very frequently (i.e. at every incoming point for the series), but currently our cpu usage is much lower than our memory usage.

[Dieterbe]

also, since we're only ever writing to the latest chunk, some of the variables could be tracked in the AggMetric object instead of for each single chunk. I'm thinking of LastWrite and and LastTs

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.