search

oetiker / rrdtool-2.x

RRDtool 2.x - The Time Series Database
86 stars 8 forks source link

RRD data as a network service #14

Open jfesler opened 11 years ago

jfesler commented 11 years ago

This is potentially related to issue #13.

It would be neat if RRD data could be "updated" and "fetched" as a network service.

The network service could offer a level of header caching and deferred writes (see issue #13). The RRD files (or the administrator of the service?) would need to specify how much write caching is acceptable. I personally would probably risk batching up writes to 30 updates per physical write; gambling a 30 minute loss in exchange for a significant improvement on how much a single box can do.

The other side of this would be that RRD graph would allow DEF's to point to network service based data storage. This would allow a single RRD graph to use multiple back end nodes; possibly clustered together or possibly geographically diverse back ends. This would allow us to finally unify a view on a distributed RRD footprint.

This is probably getting too much into the details: but I would imagine that at least one "fetch" output format should be optimal to RRD graph purposes; not necessarily XML that is expensive over the wire and expensive to parse.

oetiker commented 11 years ago

in rrdtool 1.x there is the rrdcached which works along these lines ... the present rrdtool architecture is limiting its abilities ...

luqasz commented 10 years ago

I think this is something like making an sqlite and postgresql combined. It may work, but in future it will cause a lot of havoc. I think that some things are not ment to be 2 in 1. Simply becouse if something does 2 or 3 things it doesn't do anything good.

If you decide on making rrd 2.x a network service with a daemon it will scale really nice on large environments. Drawback is that folks with embedded platform will suffer becouse of low possible resources. Maybe write some shared code and release 2 versions, with a daemon, and without. Also single API (regardless of what backend will be used) would simplify a LOT.

Decisions, decisions ;-)