Ensure metrics are also stored in Cassandra

waggle-sensor / beehive-server

Waggle cloud software for aggregation, storage and analysis of sensor data from Waggle nodes.

13 stars 17 forks source link

Ensure metrics are also stored in Cassandra #35

Closed seanshahkarami closed 6 years ago

seanshahkarami commented 6 years ago

It would be nice to have this data warehoused in Cassandra instead of just Elasticsearch in case Elasticsearch fails or we ever anticipate moving to another tool.

seanshahkarami commented 6 years ago

There's a process now which is storing all of this until we decide on how to approach the data pipeline.

For reference, the table layout is simple: ((node_id, date), timestamp, json metric data) This makes it trivial to start working with the data or moving it into the analytics pipeline.