[Q] How to identify correct carbon-cache size.

[hritz220]

How can I identify the correct carbon-cache size, I'm receiving 2,000,000 metrics per min? Any help will be appreciated.

[deniszh]

Hi @hritz220,

Unfortunately, in infinite cache is not working for you things going to really uncharted territory. First, cache sized not in metrics but in datapoints. If your data coming also every minute - then it would we around 2M datapoints per minute. So, 10 minutes of cache would take 20 000 000 points. Is it enough? Unfortunately, it's quite hard to say - depends on many other parametrers - available RAM, disk speed, amount of new metrics created, MAX_UPDATES_PER_SECOND value, MAX_CREATES_PER_MINUTE value, how many read requests do you have, what's nature of these requests, how long in the past these requests asking data etc. So, proper value can be only picked iteratively - i.e. by trial and error. I would recommend to check carbon metrics first - https://grafana.com/grafana/dashboards/311, then decide what you can/need to do. Maybe you can describe why default config is not working for you?

[hritz220]

@deniszh I see in /opt/graphite/conf/go-carbon.conf, how to determine the value of "max-size".

[cache]
# Limit of in-memory stored points (not metrics)
max-size = 1000000
# Capacity of queue between receivers and cache
# Strategy to persist metrics. Values: "max","sorted","noop"
#   "max" - write metrics with most unwritten datapoints first
#   "sorted" - sort by timestamp of first unwritten datapoint.
#   "noop" - pick metrics to write in unspecified order,
#            requires least CPU and improves cache responsiveness
write-strategy = "max"

l'm receiving 200000 metrics per minute, are the metrics which I'm receiving are affected by max-size? if I increase or decrease the value of max-size, what kind of issues I may face ?

[deniszh]

Hi @hritz220 ,

I have a different questions for you, though. What issues do you have now, and why do you think these issues are connected to max-size variable? No tricks or kidding, I'm assuming that you didn't ask if yhings went well, right?

[hritz220]

@deniszh hey I'm learning graphite tool, I'm curious to know about "max-size", on what basis it values 1000000, on what factors this variable depends, I'll be glad if you can explain to me.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.