Proposal: streamsx.metrics toolkit

[m-kotowski]

The com.ibm.streamsx.metrics toolkit supports monitoring Streams applications by providing an operator that can produce a stream of metrics. Other operators to write metrics to backends might follow.

IBM Streams supports standard and custom metrics that are assigned to operators in a Streams application, for example, the number of received and sent tuples. These metrics can be observed with the Streams console or Streams Studio to identify whether a Streams application runs as expected or whether it has some issues.

IBM Streams provides the spl.adapter::MetricsSink operator to set metrics values from within a Streams applications, and SPL functions to create custom metrics, and to set and get metrics values:

• spl.utility::createCustomMetric
• spl.utility::getCustomMetricNames
• spl.utility::getCustomMetricValue
• spl.utility::getInputPortMetricValue
• spl.utility::getOutputPortMetricValue
• spl.utility::hasCustomMetric
• spl.utility::setCustomMetricValue

The functions are limited to the operator that own a metric. An operator A cannot access the metrics of an operator B.

If you want to monitor Streams or Streams applications, you can use the following API:

• JMX API
• REST API

The com.ibm.streamsx.metrics toolkit completes this list of APIs with the com.ibm.streamsx.metrics::MetricsSource operator that uses the JMX API to get metrics values and provide them as tuple stream.

[mikespicer]

+1

[chanskw]

+1

[leongor]

+1

2016-11-01 20:19 GMT+02:00 Samantha Chan notifications@github.com:

+1

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/IBMStreams/administration/issues/100#issuecomment-257648964, or mute the thread https://github.com/notifications/unsubscribe-auth/AGvlAypkeD2PKgJNFLipk1dEHHNpxa5Yks5q54LOgaJpZM4KlzIO .

Best regards, Leonid Gorelik.

[ddebrunner]

I've been thinking about proposing a streamsx.diag/diagnostic toolkit that would include useful applications for understanding a running streams instance. E.g.

• Operator/application that gets the environment for a PE on a resource, e.g. to validate that an environment variable or Java property is set.
• Application that measures stream tuple pings between hosts.

Would it make sense for a metrics source to be part of that toolkit rather than its own toolkit? Just wondering if we will end up with a metrics toolkit with a single operator?

[mikespicer]

@ddebrunner There has also been discussion of composites that could output metrics to common monitoring tooling like grafana and kibana. I think its clearer to find this functionality with a name like metrics and If it does end up being a single operator we could always combine the 2 toolkits later.

[ddebrunner]

Sounds good, just wasn't clear what else could be in the metrics toolkit.

I'll propose streamsx.diagnostic separately

[petenicholls]

+1....starting creation process.

[petenicholls]

Done empty repos created as streamsx.metrics with @m-kotowski as committer. Please continue discussion about content in that repos.

Closing issue.