Use Cassandra aggregates for aggregating

pyr / cyanite

cyanite stores your metrics

http://cyanite.io

Other

446 stars 79 forks source link

Use Cassandra aggregates for aggregating #241

Open ifesdjeen opened 8 years ago

ifesdjeen commented 8 years ago

I've made a trivial port of one of our functions (I have to say it doesn't have the date normalisation), which could be used as grounds for the further discussions to use Cassandra aggregates: https://gist.github.com/ifesdjeen/b8ca3a119d9c4b03853d85859de4ac7e

Obvious advantage is that we reduce the amount of data transferred between the nodes.

ehlerst commented 7 years ago

I've been watching data transfer between regions with our test setup. It doesnt look that bad 3Mbit/sec but any reduction would be nice.

ifesdjeen commented 7 years ago

I have a prototype of the gorilla paper locally, which compressed data more than tenfold (which might be roughly a third in terms of throughput), I might be able to push it if you'd be willing to test...

tehlers320 commented 7 years ago

Yes we will test.