prometheus / client_python

Prometheus instrumentation library for Python applications
Apache License 2.0
3.97k stars 797 forks source link

ASGI mounted to FastAPI mounts metrics to `/metrics/` instead of `/metrics` #1016

Open hmellor opened 7 months ago

hmellor commented 7 months ago

The documentation gives an example application which mounts the ASGI app to /metrics in a FastAPI app

from fastapi import FastAPI
from prometheus_client import make_asgi_app

# Create app
app = FastAPI(debug=False)

# Add prometheus asgi middleware to route /metrics requests
metrics_app = make_asgi_app()
app.mount("/metrics", metrics_app)

and says that you should be able to see the metrics at http://localhost:8000/metrics, however if you try to access this endpoint

$ uvicorn app:app
INFO:     Started server process [1620866]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     127.0.0.1:34428 - "GET / HTTP/1.1" 404 Not Found
INFO:     127.0.0.1:34428 - "GET /favicon.ico HTTP/1.1" 404 Not Found
INFO:     127.0.0.1:34444 - "GET /metrics HTTP/1.1" 307 Temporary Redirect
INFO:     127.0.0.1:34444 - "GET /metrics/ HTTP/1.1" 200 OK

you get redirected to /metrics/.

It would be better if this redirect could be avoided.

csmarchbanks commented 7 months ago

@kakkoyun this might be something to look at to learn more about the project as well.

Sakoes commented 7 months ago

Thanks for pointing this out @hmellor. I was am doing something similar but got an "Method Not Allowed" when doing a GET request for http://localhost:8000/metrics. I am using hypercorn but I do not get the redirect. However, this does seem to work

curl http://localhost:8000/metrics/ 
Sakoes commented 7 months ago

I think I found the root cause. It has to do with how FastApi and Starlette (which FastApi is based on) are mounting sub-applications. A metrics sub-application is created as follows:

metrics_app = make_asgi_app()
app.mount("/metrics", metrics_app)

And from FastApi and Starlette docs you can read that sub-applications are mounted under a prefix, in this case "/metrics". So when you want to actually access this sub-application I guess a new root endpoint is automatically added thus creating "/metrics/". Not sure if or how you could fix this, but it looks like this is intended behavior when running the prometheus client as a FastApi sub-application.

Here you can find the docs:

hmellor commented 7 months ago

In aioprometheus (a third party Prometheus client) they use add_route to expose the metrics and they don't have this problem

from aioprometheus.asgi.starlette import metrics
app.add_route("/metrics", metrics)
Sakoes commented 7 months ago

Yes, that is exactly the cause. In this client the make_asgi_app() creates a sub-application that is mounted with app.mount() thus creating a "/metrics" prefix vs an actual route. While the aioprometheus.asgi.starlette implementation is adding a route directly with _app.addroute(). So "/metrics" is the actual route and not a prefix.

hmellor commented 6 months ago

This workaround appears to work (changes route.path_regex from ^/metrics/(?P<path>.*)$ to ^/metrics(?P<path>.*)$):

import re
from starlette.routing import Mount

...

# Add prometheus asgi middleware to route /metrics requests
route = Mount("/metrics", make_asgi_app())
route.path_regex = re.compile('^/metrics(?P<path>.*)$')
app.routes.append(route)