Full Prometheus support (pull-based HTTP endpoint)

[sevagh]

Hello Francis,

So, I use a lot of Prometheus lately - mostly with the Go and Python clients. If I were to contribute "Full Prometheus support" to this crate, how would you envision that?

• There's https://github.com/pingcap/rust-prometheus which is based on the upstream Prometheus protobufs - would you want to introduce a dependency, or redo the Prometheus metric layer/exposition format in this code from scratch?
• "Real" Prometheus is a pull model, as you describe in the README (No backend for "pull" metrics yet. Should at least provide tiny-http listener capability) - is this something you're interested in? It's more "the right way" than Pushgateway. Not sure how this fits in the dipstick model, e.g.:

METRICS.target(Prometheus::listen_on("localhost:8080/metrics").expect("serving").metrics());
COUNTER.count(32);

I can start some preliminary commits to get the ball rolling.

[fralalonde]

Hey Sevagh,

tiny-http is what I thought I'd be using, yes. There may be traces of it in Cargo and scheduler. The listen_on API you suggest looks just right.

Not sure about the rust-prometheus dependency. I believe exporting to prometheus text format to be simple enough to not require to whole crate but if it's more complicated than that then I'd be fine with adding it as a dep.

BTW, I've contributed to rust-prometheus a patch to make it possible to avoid protobuf since it is no longer supported in v2.0. Not sure if it's been released yet.

In any case, I'll trust your call. If it's heavy on the deps, just make it a feature.

Also, if anything you see sucks, tell me. Glad to have you on board.

[fralalonde]

Maybe also keep in mind that someone using a web framework such as Actix or Rocket might prefer to use that to expose the Prometheus endpoint. I don't know how this would interact with the standalone endpoint, if at all.

[sevagh]

Am I crazy or did https://github.com/fralalonde/dipstick/pull/54 break the code? make clean all is giving me hell of output - lots of errors around Rust 2018, until I check out one commit before that one.

[sevagh]

A taste:

error[E0659]: `test` is ambiguous (name vs any other name during import resolution)
   --> src/bucket/atomic.rs:454:9
    |
454 |     use test;
    |         ^^^^ ambiguous name
    |
note: `test` could refer to the module imported here
   --> src/bucket/atomic.rs:453:9
    |
453 |     use super::*;
    |         ^^^^^^^^
    = help: use `self::test` to refer to this module unambiguously
note: `test` could also refer to the extern crate imported here
   --> src/lib.rs:14:1
    |
14  | extern crate test;
    | ^^^^^^^^^^^^^^^^^^
    = help: use `::test` to refer to this extern crate unambiguously

warning: unused import: `bucket::atomic::AtomicBucket`
   --> src/core/proxy.rs:280:9
    |
280 |     use bucket::atomic::AtomicBucket;
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: #[warn(unused_imports)] on by default

warning: unused import: `core::attributes::*`
   --> src/output/graphite.rs:206:9
    |
206 |     use core::attributes::*;
    |         ^^^^^^^^^^^^^^^^^^^

warning: unused import: `core::input::*`
   --> src/output/graphite.rs:207:9
    |
207 |     use core::input::*;
    |         ^^^^^^^^^^^^^^

warning: unused import: `core::attributes::*`
   --> src/output/statsd.rs:266:9
    |
266 |     use core::attributes::*;
    |         ^^^^^^^^^^^^^^^^^^^

warning: unused import: `core::input::*`
   --> src/output/statsd.rs:267:9
    |
267 |     use core::input::*;
    |         ^^^^^^^^^^^^^^

warning: unused import: `core::attributes::*`
   --> src/output/prometheus.rs:208:9
    |
208 |     use core::attributes::*;
    |         ^^^^^^^^^^^^^^^^^^^

warning: unused import: `core::input::*`
   --> src/output/prometheus.rs:209:9
    |
209 |     use core::input::*;
    |         ^^^^^^^^^^^^^^

error[E0599]: no method named `metrics` found for type `output::graphite::Graphite` in the current scope
   --> src/output/graphite.rs:212:63
    |
34  | pub struct Graphite {
    | ------------------- method `metrics` not found for this
...
212 |         let sd = Graphite::send_to("localhost:2003").unwrap().metrics();
    |                                                               ^^^^^^^
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope, perhaps add a `use` for it:
            `use crate::core::input::Input;`

error[E0599]: no method named `metrics` found for type `output::graphite::Graphite` in the current scope
   --> src/output/graphite.rs:223:14
    |
34  | pub struct Graphite {
    | ------------------- method `metrics` not found for this
...
223 |             .metrics();
    |              ^^^^^^^
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope, perhaps add a `use` for it:
            `use crate::core::input::Input;`

error[E0599]: no method named `metrics` found for type `output::statsd::Statsd` in the current scope
   --> src/output/statsd.rs:272:61
    |
27  | pub struct Statsd {
    | ----------------- method `metrics` not found for this
...
272 |         let sd = Statsd::send_to("localhost:2003").unwrap().metrics();
    |                                                             ^^^^^^^
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope, perhaps add a `use` for it:
            `use crate::core::input::Input;`

error[E0599]: no method named `metrics` found for type `output::statsd::Statsd` in the current scope
   --> src/output/statsd.rs:283:14
    |
27  | pub struct Statsd {
    | ----------------- method `metrics` not found for this
...
283 |             .metrics();
    |              ^^^^^^^
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope, perhaps add a `use` for it:
            `use crate::core::input::Input;`

error[E0599]: no method named `metrics` found for type `output::prometheus::Prometheus` in the current scope
   --> src/output/prometheus.rs:214:65
    |
20  | pub struct Prometheus {
    | --------------------- method `metrics` not found for this
...
214 |         let sd = Prometheus::push_to("localhost:2003").unwrap().metrics();
    |                                                                 ^^^^^^^
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope, perhaps add a `use` for it:
            `use crate::core::input::Input;`

error[E0599]: no method named `metrics` found for type `output::prometheus::Prometheus` in the current scope
   --> src/output/prometheus.rs:225:14
    |
20  | pub struct Prometheus {
    | --------------------- method `metrics` not found for this
...
225 |             .metrics();
    |              ^^^^^^^
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope, perhaps add a `use` for it:
            `use crate::core::input::Input;`

error: aborting due to 31 previous errors

Some errors occurred: E0412, E0425, E0432, E0433, E0599, E0659.
For more information about an error, try `rustc --explain E0412`.
error: Could not compile `dipstick`.

To learn more, run the command again with --verbose.
make: *** [Makefile:28: bench] Error 101

[sevagh]

I'll follow the fix-forward principle and try to fix Rust 2018.

[fralalonde]

Strange, I pulled master and saw it was broken too, but after I rolledback and applied cargo fix --edition myself it worked, even though those auto-changes were the same as the rolledback ones.

[fralalonde]

If you don't mind, I'll roll back the 2018 edition changes. I'm not in a hurry to adopt it and it needs to be done properly. I've reopened the issue #47.

[sevagh]

SGTM. I'm working on a "better version of rust-2018 branch" right now, so it's best to have master work correctly until that new PR is ready.