Metric value should be a number

[DifferentialOrange]

Recently I have worked on TDG Grafana Dashboard. There are some cases (I'm researching it right now) when some metrics have not a numerical, but boolean value:

tdg_kafka_topic_partitions_desired{partition="0",client_id="rdkafka",alias="tdg",type="consumer",connector_name="kafka",name="rdkafka#consumer-1",topic="orders"} true

It breaks up Prometheus metrics collection:

The solution is obvious: just put appropriate values in collectors. However it's easy to miss when, for example, you're exporting metrics from external module:

local function update_metrics(name, stats, labels)
    local gauges = all_gauges[name]

    for metric_name, metric_stats in pairs(stats) do
        for gauge_name, gauge in pairs(gauges) do
            if metric_stats[gauge_name] ~= nil then
                gauge:set(metric_stats[gauge_name], labels)
            end
        end
    end
end

So we should do something with it. Maybe each collector should have something like checks('number') on observe. Maybe we should add some condition to strip invalid values on Prometheus export. Having such a simple way to break up metrics collection sounds not good.