search

cessda / cessda.cvs.two

Apache License 2.0
0 stars 2 forks source link

Exposing CVS Metrics to Prometheus Stack #275

Open cessda-bitbucket-importer opened 3 years ago

cessda-bitbucket-importer commented 3 years ago

Original report on BitBucket by Joshua Tetteh Ocansey (GitHub: Joshocan).

There is an example here as a guide https://github.com/prometheus/client_java

You can also follow the detail for instrumenting and exposing metrics from the Prometheus official github guide.

Note: Expose it as /metrics

‌

What to measure:

‌

cessda-bitbucket-importer commented 3 years ago

Original comment by Joshua Tetteh Ocansey (GitHub: Joshocan).

Metrics Expose implementation with   cdc-indexer  as an example   using micrometer 

‌

cessda-bitbucket-importer commented 2 years ago

Original comment by Joshua Tetteh Ocansey (GitHub: Joshocan).

Guide to instrument application using Prometheus client library in Java. You can either implement Counter or Histogram metrics. Check this for Best practices for Instrumentation

‌

matthew-morris-cessda commented 1 month ago

Exposed as of https://github.com/cessda/cessda.cvs.deploy/commit/e9b2e6b98861f555629865b2a25debb20b813741

Custom metrics have not been integrated yet.

Lorenz1983 commented 2 weeks ago

For which group are the metrics intended? And should users even know about these metrics?

pakoselo commented 2 weeks ago

@Lorenz1983 I am not sure which groups are interested for these metrics, but definitely not user group. This will be better to ask John @john-shepherdson. We didn't work on it, because that time we were overloaded with the new workflow and new versioning system.

matthew-morris-cessda commented 2 weeks ago

Some metrics are useful for the operators of the service as it can be used to alert on service malfunctions (e.g. response times longer than 1s)

Lorenz1983 commented 2 weeks ago

@pakoselo @matthew-morris-cessda Thanks for the response. Please implement the following points as already mentioned above: What to measure: Number of CVs Downloaded per format Number of CVs Downloaded per language Time in seconds used for CVs downloads Number of CVs uploaded per format by whom Number of CVs uploaded per language by whom

pakoselo commented 2 weeks ago

@Lorenz1983 I am not sure that it will go so fast and we need tothink how much time it will consume.

Lorenz1983 commented 2 weeks ago

@pakoselo Thanks for the information. If it doesn't work out for the 3.4.0 release, I'll remove the milestone and take it into account for the next release. But please let me know how long it will take. Thank you very much.

Stifo commented 2 weeks ago

This is hard to properly estimate how long it will take. My guess is around 1PM. If there's a requirement on providing the collected metrics not just as a raw text-based data but visually in a form of graphs, Grafana or other compatible tool should be deployed and configured as well along with the Prometheus server. In addition to this, such tool should be secured and restricted to be accessible to specific users only.

john-shepherdson commented 1 week ago

CESSDA has already got the infrastructure in place for Observability (see https://docs.tech.cessda.eu/technical-infrastructure/observability/index.html). This is about instrumenting CVS so that the data can be collected and processed by it. So no need to take into account Prometheus, Grafana etc in this issue.

Stifo commented 1 week ago

That makes things easier. It would help, if we can take a look on how some other CESSDA product is instrumented as an example. Is there such a case? @matthew-morris-cessda