"/metrics" is empty when using prometheus as exporter

[younes101020]

What happened?

hello, basically I only use prometheus but since 2 days I got interested in opentelemetry and I decided to use this tool for application monitoring and let prometheus manage the infrastructure part.

But I can't generate metrics to the “/metrics” endpoints to expose them to prometheus. I can see the ConsoleSpanExporter() logs, but nothing on “/metrics”.

I'm sure I've missed something and I'd be grateful to the person who can tell me what, I've been trying to debug for 1 whole day now but nothing's happened.

note: everything runs locally I have neither docker nor kubernetes

Steps to Reproduce

1. Use simple express app, im using this repo https://github.com/pichlermarc/repro-4474/tree/main
2. Setup collector and add config with specifying prometheus as exporter on 0.0.0.0:8889
3. Have a fully functional prometheus instance and add opentelemetry as job with localhost:8889 target

Expected Result

Having data to scrap for prometheus

Actual Result

Dont have data to scrap for prometheus

Additional Details

Collector status + configuration:

Prometheus configuration:

note: Prometheus can reach opentelemetry target on http://localhost:8889/metrics

OpenTelemetry Setup Code

No response

package.json

No response

Relevant log output

No response

[pichlermarc]

Hi @younes101020, thanks for reaching out.

I've tried to reproduce what you're showing here, but the app reproducer you linked offers the prometheus endpoint at http://localhost:9464/metrics.

I'm not sure how the setup you're showing above is expected to work though :thinking: Are you trying to have one of these setups?

1. App <---scrapes-- Otel Collector <---scrapes--- Prometheus (for this your collector needs a prometheus receiver)
2. App ---pushes---> OTel Collector <---scrapes--- Prometheus (for this your app needs an OTLP Exporter)
3. App <---scrapes-- Prometheus (for this your scrape target needs to point to the app (http://localhost:9464/metrics in the example that's linked)

But looking at the collector config you linked leads me to assume that your example does the following:

• App ---no link--- OTel Collector <---scrapes--- Prometheus

Which leads to no metrics being available for scraping by the OTel Collector as the Collector does never receive metrics from the app, there's no communication - neither pushing nor scraping.

[younes101020]

thank you for the response @pichlermarc im trying to implement this pipeline :

(ignore the prometheus remote write)

This is your second option App ---pushes---> OTel Collector <---scrapes--- Prometheus (for this your app needs an [OTLP Exporter](https://github.com/open-telemetry/opentelemetry-js/tree/main/experimental/packages/opentelemetry-exporter-metrics-otlp-grpc)), so the things that i have to update are the addition of OTLPMetricExporter() on sdk and add exports.otlp into otel collector config is that right ?

[younes101020]

I ended up using option 1 by declaring the metric receiver prometheus in the collector config, thank you very much @pichlermarc

[younes101020]

My goal is to be able to monitor several applications on the same machine. My current working approach is to use the Node.js module PrometheusExporter() without going through the collector's exporter, thus initiating a PrometheusExporter() instance for each application. So basically, I'm exposing a specific port for each application, for example: app1: localhost:8880/metrics, app2: localhost:8881/metrics... This would result in each application being associated with a separate Prometheus instance. I'm not sure if this is viable in terms of performance or if using the collector could improve things. What do you think? @pichlermarc