[feature] `/metrics` for prometheus

[John-H-Smith]

Version and OS 0.41 on linux/docker

Describe the use-case and give concrete real-world examples One would like to integrate the alertings of changedetection.io to grafana. As such, a metrics endpoint would be great to be integrated to prometheus or something like that.

[dgtlmoon]

see https://changedetection.io/docs/api_v1/index.html#api-System_Information-Get_Info

[John-H-Smith]

Okay, let me go go on: The current metrics are not in a format that prometheus can read. Also, there are no details about specific watches and details about the count of errors for specific watches and so on. Here you'll find the documentation of the metrics for prometheus: https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md

As grafana is very popular, especially querying from prometheus or influxdb, I think that would be a very nice feature!

[intropedro]

It would be great functionality. I was looking if it could be exported to that format.

[onedr0p]

@dgtlmoon could this issue be reopen? Having a /metrics endpoint with prometheus consumable data would be amazing for observability. With someone familiar with the projects structure, it shouldn't be too hard to add using the https://github.com/prometheus/client_python library.

For those looking for something in the meantime checkout https://github.com/schaermu/changedetection.io-exporter

[dgtlmoon]

it can be re-opened with a PR if you have one

[dgtlmoon]

actually oh my god https://github.com/schaermu/changedetection.io-exporter that is insanely complete and complex, theres no way i could have the time to maintain this additional code to support prometheus...

[onedr0p]

actually oh my god schaermu/changedetection.io-exporter that is insanely complete and complex, theres no way i could have the time to maintain this additional code to support prometheus...

You are looking at this wrong IMO, that project is only so "complex" because it needs to implement a whole REST API for gathering the data from the status endpoints of changedetection and then exporting the data in prometheus format. If you aren't familiar with Go, that can be a whole other thing as well.

If this were built into the app, it would not be as complex or complicated as you can pull this data directly from the database to export it via the https://github.com/prometheus/client_python library on a /metrics endpoint.

I'd say it's a good thing to learn something new like Prometheus, especially for observability as Prometheus is a standard and widely used.

But I get it, it's probably not a priority however it would be nice to reopen the issue.

[dgtlmoon]

I'd say it's a good thing to learn something new like Prometheus, especially for observability as Prometheus is a standard and widely used.

yes but my brain is full

so what is required here? like some stats about how many watches etc etc?

[onedr0p]

Yeah, pretty much these would be nice:

The types are explained here and labels are explained here

[dgtlmoon]

so every watch has a /metrics too right? cdio.local/3fddfa17-0e08-4995-9c24-23906a5fcceb/metrics ? the request is referring specifically to that each watch has a metric?

[onedr0p]

so every watch has a /metrics too right? cdio.local/3fddfa17-0e08-4995-9c24-23906a5fcceb/metrics ? the request is referring specifically to that each watch has a metric?

Nope, there's only one /metrics endpoint. The metrics have labels attached to them to do things like refer to which request it belongs to.

[dgtlmoon]

so every watch has a /metrics too right? cdio.local/3fddfa17-0e08-4995-9c24-23906a5fcceb/metrics ? the request is referring specifically to that each watch has a metric?

Nope, there's only one /metrics endpoint. The metrics have labels attached to them to do things like refer to which request it belongs to.

let me rephrase this, if you want this feature you need to help me as much as possible, it is unfair to expect me to learn all about prometheous - i simply do not have the time, or you are more than welcome to submit a PR

I still dont understand what it is you want, what metrics? so theres meant to be one /metrics link which contains all the information about all of the watches on a single JSON output?

[onedr0p]

I hope this is a good example, the /metrics endpoint would contain information about all the items watched. The below example would be if changedetection was only monitoring two items... I hope you might start to see a pattern of how the /metrics endpoint would be formed.

# Item 1
changedetectionio_watch_check_count{title="Test Watch", source="https://example.com"} 20
changedetectionio_watch_fetch_time{title="Test Watch", source="https://example.com"} 1.298768
changedetectionio_watch_notification_alert_count{title="Test Watch", source="https://example.com"} 1
changedetectionio_watch_last_check_status{title="Test Watch", source="https://example.com"} 1
changedetectionio_watch_price{title="Test Watch", source="https://example.com"} 3.000000
# Item 2
changedetectionio_watch_check_count{title="Another Test Watch", source="https://example.com"} 100
changedetectionio_watch_fetch_time{title="Another Test Watch", source="https://example.com"} 2.298768
changedetectionio_watch_notification_alert_count{title="Another Test Watch", source="https://example.com"} 5
changedetectionio_watch_last_check_status{title="Another Test Watch", source="https://example.com"} 0
changedetectionio_watch_price{title="Another Test Watch", source="https://example.com"} 2.000000
# Item 3
... etc

On the left you have the metric name with the labels (in{}) and on the right you have the datapoint

[dgtlmoon]

Ok and what syntax is this exactly? It's not JSON, it's some standard?

On 16 April 2024 17:54:01 UTC, Devin Buhl @.***> wrote:

I hope this is a better example, the /metrics endpoint would contain all information about all the items watched. The below example would be if changedetection was only monitoring two items...

changedetectionio_watch_check_count{title="Test Watch", source="https://example.com"} 20
changedetectionio_watch_fetch_time{title="Test Watch", source="https://example.com"} 1.298768
changedetectionio_watch_notification_alert_count{title="Test Watch", source="https://example.com"} 1
changedetectionio_watch_last_check_status{title="Test Watch", source="https://example.com"} 1
changedetectionio_watch_price{title="Test Watch", source="https://example.com"} 3.000000
changedetectionio_watch_check_count{title="Another Test Watch", source="https://example.com"} 100
changedetectionio_watch_fetch_time{title="Another Test Watch", source="https://example.com"} 2.298768
changedetectionio_watch_notification_alert_count{title="Another Test Watch", source="https://example.com"} 5
changedetectionio_watch_last_check_status{title="Another Test Watch", source="https://example.com"} 0
changedetectionio_watch_price{title="Another Test Watch", source="https://example.com"} 2.000000

-- Reply to this email directly or view it on GitHub: https://github.com/dgtlmoon/changedetection.io/issues/1555#issuecomment-2059646017 You are receiving this because you modified the open/close state.

Message ID: @.***>

[onedr0p]

Yes, it's a standard used by Prometheus for ingestion of these metrics.

Here's a blog I found that might help explain things further, could be good to skim thru. https://codeburst.io/prometheus-by-example-4804ab86e741