Metrics endpoint is not available in v2

[project0]

Preflight checklist

• [X] I could not find a solution in the existing issues, docs, nor discussions.
• [X] I agree to follow this project's Code of Conduct.
• [X] I have read and am following this repository's Contribution Guidelines.
• [ ] This issue affects my Ory Network project.
• [ ] I have joined the Ory Community Slack.
• [ ] I am signed up to the Ory Security Patch Newsletter.

Describe the bug

There is no /metrics/prometheus route in v2. I tried different versions, only in 1.x i could make it work. it seems to me there is just no route bound to the admin anymore.

Reproducing the bug

docker run -ti -p 4445:4445 -p 4444:4444 -e DSN=memory  --rm oryd/hydra:v2.0.3
curl localhost:4445/metrics/prometheus

Relevant log output

INFO[2023-03-16T09:49:47Z] completed handling request                    http_request=map[headers:map[accept:*/* user-agent:curl/7.88.1] host:localhost:4444 method:GET path:/metrics/prometheus query:<nil> remote:172.17.0.1:43564 scheme:http] http_response=map[headers:map[content-type:text/plain; charset=utf-8 x-content-type-options:nosniff] size:19 status:404 text_status:Not Found took:285.86µs]

Relevant configuration

No response

Version

2.0.3

On which operating system are you observing this issue?

Linux

In which environment are you deploying?

Kubernetes

Additional Context

No response

[aeneasr]

We migrated to otel which takes care of this part and no longer requires a prometheus scraper endpoint.

[project0]

With other words there is no openmetrics endpoint anymore? There is tons of reference in this repository and the docs does not reflect this breaking change. Also the changelog did not give any hint.

I mean, at least the docs should reflect this and provide guidance: https://www.ory.sh/docs/ecosystem/deployment#prometheus https://www.ory.sh/docs/hydra/self-hosted/production#exposing-administrative-and-public-api-endpoints not to forget the quickstart docker-compose in the repo itself.

So, i am forced to use otel collector?

[project0]

@aeneasr i played around with making to work with the OTLP protocol, but it does not seem to function. Can you provide some guidance how to setup OTEL with hydra?

I have setup the OTEL environment variables, but it does not seem to be enough in this case:

            - name: OTEL_METRICS_EXPORTER
              value: "otlp"
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: otel-collector.observability.svc.cluster.local:4317

[project0]

Turns out metrics collection with OTEL is not working (and also not documented), but the Prometheus endpoint is now available under /admin/metrics/prometheus. This is either a bug/breaking change or documentation is outdated .

[aeneasr]

Oh, certainly the docs are outdated! Fix is here: https://github.com/ory/docs/pull/1374