search

graphite-project / ceres

Distributable time-series database (not actively maintained)
Apache License 2.0
355 stars 77 forks source link

How usable is ceres? #15

Open abhishekwy opened 10 years ago

abhishekwy commented 10 years ago

There is no mention anywhere about the release date and/or the current state of the project. It is near impossible to tell unless one is willing to risk production metrics or digs into git issues/code.

Could we add that to the readme and keep updating it, after major commits? Or on an issue thread maybe?

:)

pcn commented 10 years ago

It's used by some people in production. So far what's your impression of it based on your searches? I'm wondering if there is an existing perception we need to manage.

erbdex commented 10 years ago

I receive around 500K metrics per minute and am planning to re-architect my graphite setup.

  1. i am looking at a HAproxy which spreads the load between say 2-10 instances, depending on the current metricsReceived count. i was wondering when Ceres would be usable in such a setup.
  2. Is it that ceres as a component is already mature but is waiting on 0.10 version of megacarbon et al to get released with the bunch?
  3. The documentation on the site mentions of the following features that would be inherent of ceres' design- i. ~33% reduction in size. ii. Allows a single metric to be fragmented across servers. iii. Ceres files fill up as metrics fill in, rather than at once.
  4. "Roll-up aggregation and data-expiration aren't provided by ceres, but are rather implemented as maintenance plugins in carbon". Is this a WIP? Should this be a deterrent? Could you explain what this rolling-up exactly is?
  5. How stable is ceres? What are the current challenges, recurring bugs, etc.

I understand that most of my concerns are subjective, but i have tried to be as objective as possible and would appreciate whatever inputs the community would be kind enough to provide. Finally, as launchpad forum was the go-to place in case of Graphite-carbon-whisper, which is the most authoritative parallel for ceres?

[i am the same guy as above, this being my non-work git id]

pcn commented 10 years ago

I'll share my impressions of your comments. I'm not a user of ceres. Since I agree it should be better documented, I'm just putting this out there to provide my understanding, and to see if I'm far from the general perception.

(1) Do you mean http load, or inbound metric load going through HAproxy? (2) This is my understanding. Master and megacarbon still have a lot of work going on. (4) Whisper will e.g. summarize older data over time, and will by its design not provide access to data older than you configured it to contain. Ceres doesn't do that when new metrics are written. Instead, it requires that you cron a processes that does the summarization of old data (if you want it) and expiration of older data when that time comes.

abhishekwy commented 10 years ago

Thank you. i'd gladly clarify-

  1. I meant the metric load inbound to carbon caches/relays.
  2. I understand that megacarbon still has a long way to go. How mature, subjectively is ceres?
  3. Is this (crons rolling up files passively) how it will be or is the design going to become active, or 'self-correcting'?
pcn commented 10 years ago
  1. the inbound metrics going via haproxy or another layer 3 load balancer works quite well. At 500k/minute on bare hardware the relay should work well, but can fall over if there are latencies. I have a spooling relay at pcn/carbon, but I/my employer is the only user of that code that I'm aware of.
  2. I know of high-volume production use of ceres so I'd say it's mature at this point and that it deserves more use than it is getting. 0.10.0 should help that.
  3. Ah, got it. That's a good question, and I don't know the answer.