[QUESTION] Metrics collector potential performance issue

[rafalzabrowarny]

Prometheus docs provide tip how to not use labels: "As a general guideline, try to keep the cardinality of your metrics below 10".

With "fusioncache_cache_instance_id" tag this could be problematic as more replicas and new deployments will be executed. Limit of 10 instances id can be quickly crossed and in result this could have impact on the metrics collector.

Wouldn't be better to remove instance id tag from metrics ? What's the purpose of providing cache id within metrics ?

[rafalzabrowarny]

Here's the link to prometheus docs: https://prometheus.io/docs/practices/instrumentation/#do-not-overuse-labels

[jodydonetti]

Hi @rafalzabrowarny and thanks for using FusionCache!

Your point is good, and I wondered about the same thing, but while talking with somebody quite expert in the specifics of OTEL & co he told me, if I haven't got it wrong, that FusionCache's InstanceId is very similar to Kubernetes' pod id, and normally that is included as a tag in the metrics.

I'll try to get back to him and see if I got it wrong or if there's a different angle to look at this.

In any case what I may do is to have an option to enable that in case somebody needs it, but maybe turn that off as a default, so the default experience is not to have a high cardinality tag in the metrics.

What do you think?

[jodydonetti]

Hi @rafalzabrowarny , after some thoughts I decided to in fact remove the cache instance id as a tag from the metrics, to be more on the safe side of things with Prometheus, high cardinality tags, etc.

In case for whatever reason it would be needed in the future, I'll add it back via a specific option in an opt-in way.

Thanks again for your time!

[jodydonetti]

Hi all, I just released v1.0.0-preview1 🥳

Please try it out so we can spot any potential issue before the official v1.0, thanks!

[rafalzabrowarny]

Thanks for quick response.

I'll test it on one of staging environments available for project I'm working on; if I find some bug I'll send you test to reproduce it.

[jodydonetti]

Hi @rafalzabrowarny , have you been able to try it out?

[rafalzabrowarny]

sorry, not yet I'll try update staging with new version on todays evening

[jodydonetti]

Thanks, really!

[jodydonetti]

Hi @rafalzabrowarny , I just released preview2 🥳

Please give it a try, thanks!

[jodydonetti]

Hi all, I've finally released v1.0 🥳

Feels a little unreal tbh, but yep: v1.0 is out.