Metrics Integration

[Sanskar95]

This PR integrate two types of metrics: Hardware counters Software metrics enabled using following feature flags respectively.

hardware_counters

• Enables counters like cache misses, branch misses etc.
• It is to be noted that this feature is only compatible with linux OS as it uses perf_event_open() under the hood
• One has to provide CAP_SYS_ADMIN capability to use it for eg: setcap cap_sys_admin+ep target/debug/collection , this takes the built file as an argument.
• Not executing the above command will result into "Operation not permitted" error assuming the feature flag is enabled.

metrics

• Enables software metrics for node like epoch_counter, water_mark_counter etc.
• It also enables source node realted metrics like inbound_throughput, error_counter etc.
• One can also register custom counters using the operator_context if the feature flag is on.

[Max-Meldrum]

Nice work!

Below are some high-level comments, I'll also add some smaller review comments later.

Feature flags and default recorder

We can remove the arcon_metrics feature as we are not using metrics-util + that I think prometheus exporter should be its own feature flag under "prometheus_exporter". That is, if only metrics flag is enabled then we should default to a LogRecorder.

prometheus_exporter = ["metrics-exporter-prometheus", "metrics"] hardware_counters = ["perf-event", "metrics"]

// in src/metrics/
pub struct LogRecorder {
  logger: ArconLogger,
}

impl Recorder for LogRecorder {
   fn increment_counter(&self, key: &Key, value: u64) {
      info!(self.logger, "Key {}, Value {}", key, value);
   }
   ... so on
}

// Application::new()
#[cfg(all(feature = "prometheus_exporter", feature = "metrics"))]
// Prometheus recorder
#[cfg(feature = "metrics")]
// LogRecorder

Runtime_Metrics

For this part, there is no need to have the MetricValue trait and individual implementations of Counters. We can use the metrics macros directly.

// Node Setup
register_counter!("node_name_id_epoch_counter");
register_counter!("node_name_id_watermark_counter");
// Node Execution
increment_counter!("node_name_id_epoch_counter");
increment_counter!("node_name_id_watermark_counter");

We can still keep a NodeMetrics struct for other stuff that is needed. Like the Meter.

pub struct NodeMetrics {
  inbound_throughput: Meter,
}
// Node
self.metrics.inbound_throughput.mark_n(message.total_events());
gauge!("node_name_id_inbound_throughput", self.metrics.inbound_throughput.get_mean_rate());

Histogram

1. I realised we should probably use histogram for perf-events rather than gauge.
2. We might as well add histogram to the OperatorContext while we are at it.

[Max-Meldrum]

Looks good!

Closes #181, closes #215, closes #216, closes #217