search

rycus86 / prometheus_flask_exporter

Prometheus exporter for Flask applications
https://pypi.python.org/pypi/prometheus-flask-exporter
MIT License
646 stars 161 forks source link

Question: is it possible to group metrics of endpoints that use path parameters? #158

Closed fgsalomon closed 1 year ago

fgsalomon commented 1 year ago

Hi,

First of all, thank you for your work on this package.

I'm wondering if it would be possible to group metrics of endpoints that use path parameters.

For example, given the following application:

from flask import Flask
from prometheus_flask_exporter import PrometheusMetrics

app = Flask(__name__)
metrics = PrometheusMetrics(app)

metrics.info('app_info', 'Application info', version='1.0.3')

@app.route('/')
def main():
    return "Hi there", 200

@app.route('/test')
def test():
    return "Test", 200

@app.route('/test/<id>')
def test_by_id(id):
    return f"{id}", 200

The current behavior of the package is to differentiate between calls of /test/<id> depending on the value of id.

Metrics ``` # HELP python_gc_objects_collected_total Objects collected during gc # TYPE python_gc_objects_collected_total counter python_gc_objects_collected_total{generation="0"} 856.0 python_gc_objects_collected_total{generation="1"} 520.0 python_gc_objects_collected_total{generation="2"} 0.0 # HELP python_gc_objects_uncollectable_total Uncollectable object found during GC # TYPE python_gc_objects_uncollectable_total counter python_gc_objects_uncollectable_total{generation="0"} 0.0 python_gc_objects_uncollectable_total{generation="1"} 0.0 python_gc_objects_uncollectable_total{generation="2"} 0.0 # HELP python_gc_collections_total Number of times this generation was collected # TYPE python_gc_collections_total counter python_gc_collections_total{generation="0"} 129.0 python_gc_collections_total{generation="1"} 11.0 python_gc_collections_total{generation="2"} 1.0 # HELP python_info Python platform information # TYPE python_info gauge python_info{implementation="CPython",major="3",minor="10",patchlevel="10",version="3.10.10"} 1.0 # HELP flask_exporter_info Information about the Prometheus Flask exporter # TYPE flask_exporter_info gauge flask_exporter_info{version="0.22.3"} 1.0 # HELP flask_http_request_duration_seconds Flask HTTP request duration in seconds # TYPE flask_http_request_duration_seconds histogram flask_http_request_duration_seconds_bucket{le="0.005",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.01",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.025",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.05",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.075",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.1",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.25",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.5",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.75",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="1.0",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="2.5",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="5.0",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="7.5",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="10.0",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="+Inf",method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_count{method="GET",path="/",status="200"} 1.0 flask_http_request_duration_seconds_sum{method="GET",path="/",status="200"} 0.0005045840007369407 flask_http_request_duration_seconds_bucket{le="0.005",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="0.01",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="0.025",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="0.05",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="0.075",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="0.1",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="0.25",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="0.5",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="0.75",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="1.0",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="2.5",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="5.0",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="7.5",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="10.0",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_bucket{le="+Inf",method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_count{method="GET",path="/test/one",status="200"} 3.0 flask_http_request_duration_seconds_sum{method="GET",path="/test/one",status="200"} 0.000904748998436844 flask_http_request_duration_seconds_bucket{le="0.005",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.01",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.025",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.05",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.075",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.1",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.25",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.5",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.75",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="1.0",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="2.5",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="5.0",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="7.5",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="10.0",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="+Inf",method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_count{method="GET",path="/test/two",status="200"} 1.0 flask_http_request_duration_seconds_sum{method="GET",path="/test/two",status="200"} 0.00028991600265726447 flask_http_request_duration_seconds_bucket{le="0.005",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.01",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.025",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.05",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.075",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.1",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.25",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.5",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="0.75",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="1.0",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="2.5",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="5.0",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="7.5",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="10.0",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_bucket{le="+Inf",method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_count{method="GET",path="/test/three",status="200"} 1.0 flask_http_request_duration_seconds_sum{method="GET",path="/test/three",status="200"} 0.000275792001048103 # HELP flask_http_request_duration_seconds_created Flask HTTP request duration in seconds # TYPE flask_http_request_duration_seconds_created gauge flask_http_request_duration_seconds_created{method="GET",path="/",status="200"} 1.683028523631882e+09 flask_http_request_duration_seconds_created{method="GET",path="/test/one",status="200"} 1.683028533197392e+09 flask_http_request_duration_seconds_created{method="GET",path="/test/two",status="200"} 1.6830285360972118e+09 flask_http_request_duration_seconds_created{method="GET",path="/test/three",status="200"} 1.683028545348586e+09 # HELP flask_http_request_total Total number of HTTP requests # TYPE flask_http_request_total counter flask_http_request_total{method="GET",status="200"} 6.0 # HELP flask_http_request_created Total number of HTTP requests # TYPE flask_http_request_created gauge flask_http_request_created{method="GET",status="200"} 1.683028523632361e+09 # HELP flask_http_request_exceptions_total Total number of HTTP requests which resulted in an exception # TYPE flask_http_request_exceptions_total counter # HELP app_info Application info # TYPE app_info gauge app_info{version="1.0.3"} 1.0 ```

In my use case it would be more useful to have the metrics grouped in /test/<id>. Could this be achievable?

Thanks.

rycus86 commented 1 year ago

Hi,

I think you're looking for the group_by config parameter, see https://github.com/rycus86/prometheus_flask_exporter#configuration If you set it to endpoint then it should capture the function name instead of the URI, the url_rule might also work for you but I can't remember right now what that results in (maybe the path pattern).

Does this help?

fgsalomon commented 1 year ago

Yes, it seems that url_rule was what I was looking for.

Metrics ``` # HELP python_gc_objects_collected_total Objects collected during gc # TYPE python_gc_objects_collected_total counter python_gc_objects_collected_total{generation="0"} 895.0 python_gc_objects_collected_total{generation="1"} 520.0 python_gc_objects_collected_total{generation="2"} 0.0 # HELP python_gc_objects_uncollectable_total Uncollectable object found during GC # TYPE python_gc_objects_uncollectable_total counter python_gc_objects_uncollectable_total{generation="0"} 0.0 python_gc_objects_uncollectable_total{generation="1"} 0.0 python_gc_objects_uncollectable_total{generation="2"} 0.0 # HELP python_gc_collections_total Number of times this generation was collected # TYPE python_gc_collections_total counter python_gc_collections_total{generation="0"} 129.0 python_gc_collections_total{generation="1"} 11.0 python_gc_collections_total{generation="2"} 1.0 # HELP python_info Python platform information # TYPE python_info gauge python_info{implementation="CPython",major="3",minor="10",patchlevel="10",version="3.10.10"} 1.0 # HELP flask_exporter_info Information about the Prometheus Flask exporter # TYPE flask_exporter_info gauge flask_exporter_info{version="0.22.3"} 1.0 # HELP flask_http_request_duration_seconds Flask HTTP request duration in seconds # TYPE flask_http_request_duration_seconds histogram flask_http_request_duration_seconds_bucket{le="0.005",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="0.01",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="0.025",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="0.05",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="0.075",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="0.1",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="0.25",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="0.5",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="0.75",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="1.0",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="2.5",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="5.0",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="7.5",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="10.0",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_bucket{le="+Inf",method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_count{method="GET",status="200",url_rule="/test"} 1.0 flask_http_request_duration_seconds_sum{method="GET",status="200",url_rule="/test"} 0.0002818749999278225 flask_http_request_duration_seconds_bucket{le="0.005",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="0.01",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="0.025",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="0.05",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="0.075",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="0.1",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="0.25",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="0.5",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="0.75",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="1.0",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="2.5",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="5.0",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="7.5",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="10.0",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_bucket{le="+Inf",method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_count{method="GET",status="200",url_rule="/test/"} 4.0 flask_http_request_duration_seconds_sum{method="GET",status="200",url_rule="/test/"} 0.0018419590014673304 flask_http_request_duration_seconds_bucket{le="0.005",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="0.01",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="0.025",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="0.05",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="0.075",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="0.1",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="0.25",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="0.5",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="0.75",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="1.0",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="2.5",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="5.0",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="7.5",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="10.0",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_bucket{le="+Inf",method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_count{method="GET",status="200",url_rule="/"} 1.0 flask_http_request_duration_seconds_sum{method="GET",status="200",url_rule="/"} 0.00035812499845633283 # HELP flask_http_request_duration_seconds_created Flask HTTP request duration in seconds # TYPE flask_http_request_duration_seconds_created gauge flask_http_request_duration_seconds_created{method="GET",status="200",url_rule="/test"} 1.6830323634057999e+09 flask_http_request_duration_seconds_created{method="GET",status="200",url_rule="/test/"} 1.683032366863081e+09 flask_http_request_duration_seconds_created{method="GET",status="200",url_rule="/"} 1.683032377763504e+09 # HELP flask_http_request_total Total number of HTTP requests # TYPE flask_http_request_total counter flask_http_request_total{method="GET",status="200"} 6.0 # HELP flask_http_request_created Total number of HTTP requests # TYPE flask_http_request_created gauge flask_http_request_created{method="GET",status="200"} 1.683032363405945e+09 # HELP flask_http_request_exceptions_total Total number of HTTP requests which resulted in an exception # TYPE flask_http_request_exceptions_total counter # HELP app_info Application info # TYPE app_info gauge app_info{version="1.0.3"} 1.0 ```

I didn't want to group by enpdoint, but should have tested the url_rule before asking.

Thank you!