search

elasticio / sailor-nodejs

The official elastic.io library for bootstrapping and executing for Node.js connectors.
4 stars 4 forks source link
elasticio ipaas nodejs platform

readme

elasticio-sailor-nodejs Build Status Dependency Status

The official elastic.io library for bootstrapping and executing for Node.js connectors.

NPM

elasticio-sailor-nodejs is a required dependency for all components build for elastic.io platform in Node.js. Add the dependency in the package.json file in the following way:

    "dependencies": {
        "q": "^1.4.1",
        "elasticio-sailor-nodejs": "^2.2.1",
        "elasticio-node": "^0.0.8"
    }

Building components in Node.js

If you plan to build a component for elastic.io platform in Node.js then you can visit our dedicated documentation page which describes how to build a component in node.js.

Before you start

Before you can deploy any code into our system you must be a registered elastic.io platform user. Please see our home page at http://www.elastic.io to learn how.

Any attempt to deploy a code into our platform without a registration would fail.

After the registration and opening of the account you must upload your SSH Key into our platform.

If you fail to upload you SSH Key you will get permission denied error during the deployment.

Getting Started

After registration and uploading of your SSH Key you can proceed to deploy it into our system. At this stage we suggest you to:

Examples of Node.js components

Here is a list of components build on Node.js:

Sailor logging

Sailor uses bunyan logging framework.

Supported log levels are:

Default log level is INFO. You can change default log level with environment variable LOG_LEVEL.

Sailor logger adds the following extra context to log messages:

Component logging

Sailor exposes logger object for use in a component. Component logger inherits log level from sailor's logger and adds the following extra context to log messages:

Component logger usage example:

this.logger.fatal('message');
this.logger.error('message');
this.logger.warn('message');
this.logger.info('message');
this.logger.debug('message');
this.logger.trace('message');

Flow control

When working in the multi-tenant integration environment it's important to obey the API and consumption limits imposed by the platform. This is not only a condition for you integrations to run on the platform (and not begin suspended), but also a good integration practice to sustainably and efficiently use resources.

Imagine a system where one party (published) publishes to the queue and one or more consumers consume from the queue. If publishers are writing to the queue faster than consumers read data from the queue, queue will earlier or later be overfilled. Once one queue of your integration flow will grow to a particular limit, the complete integration flow will be suspended and author will be informed about it. Flow control is a build-in mechanism in the SDK that will help you to prevent the overflow to happen.

There are two types of flow control:

Let's take a look at the simple example:

'use strict';

exports.process = process;

async function process(msg, cfg, snapshot) {
    for (let i = 0; i < 100000; i++) {
        this.logger.info('Sending message %s', i);
        await this.emit('data', {
            body: {
                counter: i,
                hi: 'there'
            }
        });
        this.logger.info('Message %s was sent', i);
    }
}

This simple component, once started on the platform will generate 100k messages. Without flow control this example will quickly bring the integration queue to the limits and integration flow will be suspended. With flow control the publishing rate of the messages will be slowed down so both publisher and consumers will operate in balance.

How to configure it

There is a set of environment variables that are responsible for the configuration of the static flow control (dynamic flow control is implemented in the message-oriented middleware of the platform hence can't be configured on the component level)

Sailor hooks

Init hook

/**
* cfg - This is the same config as the one passed to "processMessage" method of the trigger or action
*/
exports.init = function(cfg) {
    //do stuff
    return Promise.resolve();
}

Startup hook

/**
* cfg - This is the same config as the one passed to "processMessage" method of the trigger or action
*/
exports.startup = function(cfg) {
    //do stuff
    const data = {
        message: 'Hello from STARTUP_HOOK'
    };
    return Promise.resolve(data);
}

Startup state data - either return value or the result of the promise

Shutdown hook

/**
* cfg - This is the same config as the one passed to "processMessage" method of the trigger or action
* startData - result from the startup
*/
exports.shutdown = function(cfg, startData) {
    //corresponding to the startup example above, startData is { message: 'Hello from STARTUP_HOOK' }
    //do stuff
    return Promise.resolve();
}

TBD - log for shutdown hooks?