Add function to parse /metrics

Vinalti commented 1 year ago

Summary

The endpoint /metrics allow authentication with API key and provide most metrics. It can be useful to retrieve status of monitoring and kuma server in one go.

It is a polling endpoint and therefore it doesn't use the SocketIO protocol but a simple HTTP GET request is enough.

The only drawback is that this endpoint provide data in a specific format for Prometheus, so a good parsing will be required to extract the values.

I propose to add that to the next main release.

Checklist

[ ] Fetch data from /metrics
[ ] Authentication with API Key functionnal
[ ] Parse Prometheus syntax
[ ] Return clean dictionary / object with retrieved data

Syntax

metric_name [
  "{" label_name "=" `"` label_value `"` { "," label_name "=" `"` label_value `"` } [ "," ] "}"
] value [ timestamp ]
In the sample syntax:

metric_name and label_name carry the usual Prometheus expression language restrictions. label_value can be any sequence of UTF-8 characters, but the backslash (\), double-quote ("), and line feed (\n) characters have to be escaped as \\, \", and \n, respectively. value is a float represented as required by Go's ParseFloat() function. In addition to standard numerical values, NaN, +Inf, and -Inf are valid values representing not a number, positive infinity, and negative infinity, respectively. The timestamp is an int64 (milliseconds since epoch, i.e. 1970-01-01 00:00:00 UTC, excluding leap seconds), represented as required by Go's ParseInt() function.

_source_

It would seem that it's made to be parsed in Golang, also, a go package already exists for that. I am not sure if we could use that as a "library" or middleware to extract the data.

Alternatively, the prometheus syntax is based on EBNF language. EBNF parsers exists in different languages so we could use that as well.

Ressources

Vinalti commented 1 year ago

the way I see it we have many keys classified in categories , and they all are shown like that :

<key>{<content>} <value>

example:

monitor_status{monitor_name="router",monitor_type="http",monitor_url="https://192.168.0.1/",monitor_hostname="null",monitor_port="null"} 1
# ^key              ^content                                                                                                             ^value

Also the content is optional.

Monitors

Starts with monitor_

Process Status

starts with process_
Contains information about CPU usage and memory used

NodeJS informations

starts with nodejs_
contains info about eventloop, heap size, software version, ...

app info

starts with app_
so far only app_version

http info

starts with http_
information about http request and response
Empty for me

ExpressJS info

starts with expressjs_
number of open connections

Parsing

each group of line is preceded by description with the following keys:

# HELP <key> <description>
# TYPE <key> <type>

I would think we can split the key with the first _ in order to group them by categories.

Output

I think the output could be an object like that:

{
    "monitors": [
        {
            "key": "<key>",
            "description": "<description>",
            "type": "<type>",
            "content": [
                {
                    "value": "<value>"
                    #... (Content of the dict)
                }
            ]
        }
    ]
}

lucasheld commented 1 year ago

I dont think it makes sense to parse the metrics directly in this python module. The python module prometheus-client that you mentioned above contains a parser that does just that. So if you need this data, you can install the library and parse the metric string. This way, there is no need to add another dependency for a feature that is probably not used very often.

But what probably makes sense is to add a method that returns the metrics string. Something like that:

def metrics(self, api_key: str) -> dict:
    r = requests.get(f"{self.url}/metrics", auth=("user", api_key))
    return r.text

lucasheld / uptime-kuma-api