RuntimeTools / appmetrics

Node Application Metrics provides a foundational infrastructure for collecting resource and performance monitoring data for Node.js-based applications.
https://developer.ibm.com/open/node-application-metrics/
Apache License 2.0
970 stars 125 forks source link
appmetrics debug health-center-client metrics metrics-gathering monitoring monitoring-data monitoring-server node-application-metrics node-js node-sdk nodejs performance-metrics performance-monitoring

Node Application Metrics

Node Application Metrics monitoring and profiling agent

Codacy Badge Build Status codebeat badge codecov.io Apache 2 Homepage Module LTS Adopted' IBM Support

Node Application Metrics instruments the Node.js runtime for performance monitoring, providing the monitoring data via an API. Additionally the data can be visualized by using the Node Application Metrics Dashboard.

The data can also be visualized in Eclipse using the IBM Monitoring and Diagnostics Tools - Health Center client. Profiling data is available in Health Center, but is not yet available in the Dashboard. See https://www.ibm.com/developerworks/java/jdk/tools/healthcenter/ for more details.

Node Application Metrics provides the following built-in data collection sources:

Source Description
Environment Machine and runtime environment information
CPU Process and system CPU
Memory Process and system memory usage
GC Node/V8 garbage collection statistics
Event Loop Event loop latency information
Loop Event loop timing metrics
Function profiling Node/V8 function profiling (disabled by default)
HTTP HTTP request calls made of the application
HTTP Outbound HTTP requests made by the application
socket.io WebSocket data sent and received by the application
LevelDB LevelDB queries made by the application
MySQL MySQL queries made by the application
MongoDB MongoDB queries made by the application
PostgreSQL PostgreSQL queries made by the application
MQTT MQTT messages sent and received by the application
MQLight MQLight messages sent and received by the application
Memcached Data that is stored or manipulated in Memcached
OracleDB OracleDB queries made by the application
Oracle Oracle queries made by the application
StrongOracle StrongOracle database queries made by the application
Redis Redis commands issued by the application
Riak Riak methods called by the application
Request tracking A tree of application requests, events and optionally trace (disabled by default)
Function trace Tracing of application function calls that occur during a request (disabled by default)

Performance overhead

Our testing has shown that the performance overhead in terms of processing is minimal, adding less than 0.5 % to the CPU usage of your application. The additional memory required is around 20 MB to gather information about your system and application.

We gathered this information by monitoring the sample application Acme Air. We used MongoDB as our datastore and used JMeter to drive load though the program. We have performed this testing with Node.js version 6.10.3

Getting Started

Pre-requisites:

Appmetrics uses node-gyp to compile and build local binary libraries to enhance execution performance. If the following compilation and build logs contain errors, make sure you have the node-gyp pre-requisites installed (https://github.com/nodejs/node-gyp#installation). If you have them and the build still had errors, see if there are any related issues at https://github.com/RuntimeTools/appmetrics/issues). If there aren\'t, feel free to open a new issue to report the bug.

Installation

You can get Node Application Metrics from 3 different places:

Configuring Node Application Metrics

Node Application Metrics can be configured in two ways, by using the configuration file described below or via a call to configure(options).

Node Application Metrics comes with a configuration file inside the module installation directory (.../node_modules/appmetrics/appmetrics.properties). This can be used to configure connection options, logging and data source options.

Node Application Metrics will attempt to load appmetrics.properties from one of the following locations (in order):

  1. the application directory
  2. the current working directory
  3. the appmetrics module installation directory

The default configuration has minimal logging enabled, will attempt to send data to a local MQTT server on the default port and has method profiling disabled.

Many of the options provide configuration of the Health Center core agent library and are documented in the Health Center documentation: Health Center configuration properties.

The following options are specific to appmetrics:

Running Node Application Metrics

Preloading appmetrics

In previous versions appmetrics came with an executable, node-hc, which could be used instead of the node command to run your application and load and start appmetrics. This has been removed in version 4.0.0, instead you can use:

$ node --require appmetrics/start app.js

to preload and start appmetrics, or in Node.js from versions 8.0.0 and 6.12.0 onwards, use the NODE_OPTIONS environment variable:

$ export NODE_OPTIONS="--require appmetrics/start"

Modifying your application to use appmetrics

If you locally install this module with npm then you will additionally have access to the monitoring data via the appmetrics API (see API Documentation).

To load appmetrics and get the monitoring API object, add the following to the start-up code for your application:

var appmetrics = require('appmetrics');
var monitoring = appmetrics.monitor();

The call to appmetrics.monitor() starts the data collection agent, making the data available via the API and to the Heath Center client via MQTT.

You should start your application using the node command as usual (not node-hc).

You must call require('appmetrics'); before the require statements for any npm modules you want to monitor. Appmetrics must be initialized first so that it can instrument modules for monitoring as they are loaded. If this is a problem due to the structure of your application you can require the module on the node command line by using -r or --require or by setting NODE_OPTIONS as described above to make sure it is pre-loaded.

Once you have loaded appmetrics you can then use the monitoring object to register callbacks and request information about the application:

monitoring.on('initialized', function (env) {
    env = monitoring.getEnvironment();
    for (var entry in env) {
        console.log(entry + ':' + env[entry]);
    };
});

monitoring.on('cpu', function (cpu) {
    console.log('[' + new Date(cpu.time) + '] CPU: ' + cpu.process);
});

Health Center Eclipse IDE client

Not supported on z/OS

Connecting to the client

Connecting to the Health Center client requires the additional installation of a MQTT broker. The Node Application Metrics agent sends data to the MQTT broker specified in the appmetrics.properties file or set via a call to configure(options). Installation and configuration documentation for the Health Center client is available from the Health Center documentation in IBM Knowledge Center.

Note that both the API and the Health Center client can be used at the same time and will receive the same data. Use of the API requires a local install and application modification (see Modifying your application to use the local installation).

Further information regarding the use of the Health Center client with Node Application Metrics can be found on the appmetrics wiki: Using Node Application Metrics with the Health Center client.

API Documentation

appmetrics.configure(options)

Sets various properties on the appmetrics monitoring agent. If the agent has already been started, this function does nothing.

Property name Property value type Property description
applicationID string Specifies a unique identifier for the mqtt connection
mqtt string['off'\|'on'] Specifies whether the monitoring agent sends data to the mqtt broker. The default value is 'on'
mqttHost string Specifies the host name of the mqtt broker
mqttPort string['[0-9]*'] Specifies the port number of the mqtt broker
profiling string['off'\|'on'] Specifies whether method profiling data will be captured. The default value is 'off'

appmetrics.start()

Starts the appmetrics monitoring agent. If the agent is already running this function does nothing.

appmetrics.stop()

Stops the appmetrics monitoring agent. If the agent is not running this function does nothing.

appmetrics.enable(type, config)

Enable data generation of the specified data type. Cannot be called until the agent has been started by calling start() or monitor().

The following data types are disabled by default: profiling, requests, trace

appmetrics.disable(type)

Disable data generation of the specified data type. Cannot be called until the agent has been started by calling start() or monitor().

appmetrics.setConfig(type, config)

Set the configuration to be applied to a specific data type. The configuration available is specific to the data type.

appmetrics.emit(type, data)

Allows custom monitoring events to be added into the Node Application Metrics agent.

appmetrics.writeSnapshot([filename],[callback])

Not supported on z/OS Dumps the v8 heap via heapdump. For more information, see https://github.com/bnoordhuis/node-heapdump/blob/master/README.md

appmetrics.monitor()

Creates a Node Application Metrics agent client instance. This can subsequently be used to get environment data and subscribe to data events. This function will start the appmetrics monitoring agent if it is not already running.

appmetrics.monitor.getEnvironment()

Requests an object containing all of the available environment information for the running application. This will not contain all possible environment information until an 'initialized' event has been received.

Event: 'cpu'

Not supported on z/OS Emitted when a CPU monitoring sample is taken.

Event: 'eventloop'

Emitted every 5 seconds, summarising sample based information of the event loop latency

Event: 'gc'

Emitted when a garbage collection (GC) cycle occurs in the underlying V8 runtime.

Event: 'initialized'

Emitted when all possible environment variables have been collected. Use appmetrics.monitor.getEnvironment() to access the available environment variables.

Event: 'loop'

Emitted every 5 seconds, summarising event tick information in time interval

Event: 'memory'

Emitted when a memory monitoring sample is taken.

Event: 'profiling'

Emitted when a profiling sample is available from the underlying V8 runtime.

API: Dependency Events (probes)

Event: 'http'/'https'

Emitted when a HTTP/HTTPS request is made of the application.

Event: 'http-outbound'/'https-outbound'

Emitted when the application makes an outbound HTTP/HTTPS request.

Event: 'leveldown'

Emitted when a LevelDB query is made using the leveldown module.

Event: 'loopback-datasource-juggler'

Emitted when a function is called on the loopback-datasource-juggler module

Event: 'memcached'

Emitted when a data is stored, retrieved or modified in Memcached using the memcached module.

Event: 'mongo'

Emitted when a MongoDB query is made using the mongodb module.

Event: 'mqlight'

Emitted when a MQLight message is sent or received.

Event: 'mqtt'

Emitted when a MQTT message is sent or received.

Event: 'mysql'

Emitted when a MySQL query is made using the mysql module.

Event: 'oracle'

Emitted when a query is executed using the oracle module.

Event: 'oracledb'

Emitted when a query is executed using the oracledb module.

Event: 'postgres'

Emitted when a PostgreSQL query is made to the pg module.

Event: 'redis'

Emitted when a Redis command is sent.

Event: 'riak'

Emitted when a Riak method is called using the basho-riak-client module.

Event: 'socketio'

Emitted when WebSocket data is sent or received by the application using socketio.

Event: 'strong-oracle'

Emitted when a query is executed using the strong-oracle module.

API: Requests

Event: 'request'

Requests are a special type of event emitted by appmetrics. All the probes named above can also create request events if requests are enabled. However requests are nested within a root incoming request (usually http). Request events are disabled by default.

Supported platforms

The Node Application Metrics agent supports the following runtime environments where a Node.js runtime is available:

Troubleshooting

Find below some possible problem scenarios and corresponding diagnostic steps. Updates to troubleshooting information will be made available on the appmetrics wiki: Troubleshooting. If these resources do not help you resolve the issue, you can open an issue on the Node Application Metrics appmetrics issue tracker.

Checking Node Application Metrics has started

By default, a message similar to the following will be written to console output when Node Application Metrics starts:

[Fri Aug 21 09:36:58 2015] com.ibm.diagnostics.healthcenter.loader INFO: Node Application Metrics 1.0.1-201508210934 (Agent Core 3.0.5.201508210934)

Error "The specified module could not be found ... appmetrics.node"

This error indicates there was a problem while loading the native part of the module or one of its dependent libraries. On Windows, appmetrics.node depends on a particular version of the C runtime library and if it cannot be found this error is the likely result.

Check:

Note: On Windows, the global module installation directory might be shared between multiple Node.js runtime environments. This can cause problems with globally installed modules with native components, particularly if some of the Node.js runtime environments are 32-bit and others are 64-bit because the native components will only work with those with matching bitness.

Error "Failed to open library .../libagentcore.so: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found"

This error indicates there was a problem while loading the native part of the module or one of its dependent libraries. On non-Windows platforms, libagentcore.so depends on a particular (minimum) version of the C runtime library and if it cannot be found this error is the result.

Check:

No profiling data present for Node.js applications

Method profiling data is not collected by default, check Configuring Node Application Metrics for information on how to enable it.

If collection is enabled, an absence of method profiling data from a Node.js application could be caused by the type of tasks that are being run by your application -- it may be running long, synchronous tasks that prevent collection events from being scheduled on the event loop.

If a task uses the Node.js thread exclusively then shuts down the Node.js runtime environment, the Health Center agent may not get the opportunity to obtain any profiling data. An example of such an application is the Octane JavaScript benchmark suite, which loads the CPU continuously rather than dividing the load across multiple units of work.

Source code

The source code for Node Application Metrics is available in the appmetrics project. Information on working with the source code -- installing from source, developing, contributing -- is available on the appmetrics wiki.

License

This project is released under an Apache 2.0 open source license.

Versioning scheme

The npm package for this project uses a semver-parsable X.0.Z version number for releases, where X is incremented for breaking changes to the public API described in this document and Z is incremented for bug fixes and for non-breaking changes to the public API that provide new function.

Development versions

Non-release versions of this project (for example on github.com/RuntimeTools/appmetrics) will use semver-parsable X.0.Z-dev.B version numbers, where X.0.Z is the last release with Z incremented and B is an integer. For further information on the development process go to the appmetrics wiki: Developing.

Module Long Term Support Policy

This module adopts the Module Long Term Support (LTS) policy, with the following End Of Life (EOL) dates:

Module Version Release Date Minimum EOL EOL With Status
V4.x.x Jan 2018 Dec 2019 Maintenance
V5.x.x May 2019 Dec 2020 Current

Version

5.1.1

Release History

5.1.1 - Node13 support, bump dependency versions and a trace probe fix.
5.0.5 - zAppmetrics fixes, and bump agentcore for Alpine support.
5.0.3 - Bug fix.
5.0.2 - Bump level of omragentcore.
5.0.1 - Bug fix for incorrect timiings on http request.
5.0.0 - Add Node 12 support, remove Node 6 support.
4.0.1 - Bug fix release including adding Node 10 support on Windows (Unix already working).
4.0.0 - Remove node-hc and add support for preloading.
3.1.3 - Packaging fix.
3.1.2 - Bug fixes.
3.1.1 - Node v6 on z/OS support.
3.1.0 - HTTPS probe added. Remove support for Node v7.
3.0.2 - Probe defect for Node 8 support.
3.0.1 - Packaging bug fix to allow build from source if binary not present.
3.0.0 - Remove express probe. Additional data available in http and request events. Code improvements.
2.0.1 - Remove support for Node.js 0.10, 0.12, 5. Add heapdump api call.
1.2.0 - Add file data collection capability and option configuration via api.
1.1.2 - Update agent core to 3.0.10, support Node.js v7.
1.1.1 - Fix node-gyp rebuild failure and don't force MQTT broker to on
1.1.0 - Bug fixes, improved MongoDB data, updated dependencies, CPU watchdog feature
1.0.13 - Express probe, strong-supervisor integration
1.0.12 - Appmetrics now fully open sourced under Apache 2.0 license
1.0.11 - Bug fixes
1.0.10 - Bug fixes
1.0.9 - Loopback and Riak support, bug fixes and update to agent core 3.0.9.
1.0.8 - Oracle support, bug fixes and api tests runnable using 'npm test'.
1.0.7 - StrongOracle support, support for installing with a proxy, expose MongoDB, MQLight and MySQL events to connectors.
1.0.6 - OracleDB support and bug fixes.
1.0.5 - Expose HTTP events to connectors (including MQTT).
1.0.4 - Redis, Leveldown, Postgresql, Memcached, MQLight and MQTT support, higher precision timings, and improved performance.
1.0.3 - Node.js v4 support.
1.0.2 - HTTP, MySQL, MongoDB, request tracking and function tracing support.
1.0.1 - Mac OS X support, io.js v2 support.
1.0.0 - First release.