Support custom metrics

[agross]

I'm currently evaluating means to write my own metric type. Namely I want to have a "string" histogram showing the distribution of different command types (strings) my application handles.

As far as I see the registry currently has the metric types hard-coded (counter, gauge, etc). Is it possible to extend these, probably leveraging the existing histogram infrastructure and coding a new StringHistogram?

[etishor]

I have also felt the need sometimes for a metric that would keep a set of string values and display some form of aggregation for them.

I'm not sure that a Histogram is the metric best suited for this. I'm not even sure it is possible to have a histogram around something other that numeric values.

The decision of hard-coding the metric types, instead of providing a common interface for them, was based on the fact that the different metric types don't have anything in common. It would have to be an empty interface and an empty interface is usually a sign that things need to be re-modeled better.

It would be great if you could provide more details on what you are trying to measure and what the results should be like.

I will think a bit more on how a string metric could be represented and post updates.

[agross]

My histogram would have buckets as strings, each value associated with a string would be of type long.

What I'm trying to achieve is to get a notion of how many commands of each type my system processes. Given that commands might be numerous I need an open-closed solution. Creating a counter for each and every command type is not an option.

I'd consider a simple bar chart viable for us: x-axis with strings (buckets), y-axis with the number of commands for each bucket. We have about 10 command types (strings, x-axis), the graph might just display the topmost 5 or so. That would suffice for our needs.

Let me know if you have further questions!

[dtabuenc]

It seems to me like each of these strings should be it's own counter metric.

[agross]

What if there are hundreds of command types? Should I manually create counters for all of them and find the matching metric in my generic command handler?

[etishor]

Still work in progress, but on branch value-histogram ( https://github.com/etishor/Metrics.NET/tree/value-histogram ) i've implemented something that handles this functionality.

The is no UI yet, just text visualization.

Basically the implementation is a collection of Meter metrics for a set of string values.

[agross]

:+1:

[etishor]

After more thinking, it turned out that it is easier and makes more sense to extend the Counter and Meter metrics to also support recording values per set item. ( See https://github.com/etishor/Metrics.NET/blob/dev/Src/Metrics/Counter.cs and https://github.com/etishor/Metrics.NET/blob/dev/Src/Metrics/Meter.cs )

Again the set item values are only displayed in the text reports for now.

[agross]

I'm totally OK with your decision. Looking forward to the UI bits :-)

[dekenless]

I too am looking forward to the UI bits. Cheers.

[etishor]

At the moment, before starting a new iteration of the visualization UI, my plan was something like this:

• support graphite [basic functionality done]
• support influxdb [basic functionality done]
• support elasticsearch [still in progress]
• have graphana running on top of graphite or influxdb [kind of working with influxdb]
• have kibana running on top of elastic search [i still can't manage to understand this setup]

After having graphana and kibana running and configured - i'm really hopping they have enough functionality to fit most scenarios. But anyway the idea is to get a broad picture of what others are doing and how they are using it and what kind of information is presented etc.

I'm hoping this would allow me to make better decisions with the second iteration of the visualization ui,

[kadaan]

Is there a work in progress branch for the Elasticsearch integration? Also, what don't you understand about the kibana -> elasticsearch setup?

[etishor]

The elasticsearch integration is not pushed yet, as it is really the first bits. it is on my home machine at the moment but I can push-it today or tomorrow if there is interest.

My issue with elasticsearch and kibana was that I've managed relatively easy to create and push the metrics to elasticsearch but I was not able to get kibana to display anything meaningful or remotely useful in the first couple of hours. I admit skipping a lot of documentation so I don't blame it on kibana or ES - but it is not something simple to do.

As a future goal, I think it is possible to also automatically generate the JSON dashboards for Graphana / Kibana.

[kadaan]

I would love to try the branch as I have an ES cluster up and running. That said, settling on a metric format that could be written to disk, pulled by LogStash and imported into ES might be the best long term route.

On Dec 11, 2014, at 10:00 AM, Iulian Margarintescu notifications@github.com wrote:

The elasticsearch integration is not pushed yet, as it is really the first bits. it is on my home machine at the moment but I can push-it today or tomorrow if there is interest.

My issue with elasticsearch and kibana was that I've managed relatively easy to create and push the metrics to elasticsearch but I was not able to get kibana to display anything meaningful or remotely useful in the first couple of hours. I admit skipping a lot of documentation so I don't blame it on kibana or ES - but it is not something simple to do.

As a future goal, I think it is possible to also automatically generate the JSON dashboards for Graphana / Kibana.

— Reply to this email directly or view it on GitHub.

[etishor]

I've moved the ElasticSearch discussion to issue #53