awolden / brakes

Hystrix compliant Node.js Circuit Breaker Library
MIT License
300 stars 35 forks source link
circuit-breaker circuit-breaker-library fault-tolerance health-check hystrix hystrix-dashboard

Dependency Status Build Status Coverage Status npm version Code Climate contributions welcome badges

brakes

Brakes is a circuit breaker library for Node. A circuit breaker provides latency and fault protection for distributed systems. Brakes will monitor your outgoing requests, and will trip an internal circuit if it begins to detect that the remote service is failing. Circuit protection allows you to redirect requests to sane fallbacks, and back-off the downstream services so they can recover. This module is largely based on Netflix's Hystrix

Requires Node 4.2.0 or higher

http://martinfowler.com/bliki/CircuitBreaker.html

https://github.com/Netflix/Hystrix/wiki/How-it-Works

https://en.wikipedia.org/wiki/Circuit_breaker_design_pattern


Bluebird and Promisify

Bluebird

This module utilizes bluebird promises. For more on the features of bluebird visit their site: http://bluebirdjs.com/. Brakes uses bluebird over native promises in order to provide more feature rich promises and because bluebird offers performance comparable to that of raw callbacks.

Promisify

If you pass an async function that relies on callback, brakes will promisify it into a bluebird promise. If you pass a promise to brakes, it will use that promise as is.

Note: brakes will only detect async callback functions that use callbacks with one of the following names: cb, callback, callback_, or done.

Examples

Promise

  function promiseCall(foo){
    return new Promise((resolve, reject) =>{
      if (foo) resolve(foo);
      else reject(foo);
    });
  }

  const brake = new Brakes(promiseCall, {timeout: 150});

  brake.exec('bar')
    .then((result) =>{
      console.log(`result: ${result}`);
    })
    .catch(err =>{
      console.error(`error: ${err}`);
    });

Callback

  function asyncCall(foo, cb){
    if (foo) cb(null, foo);
    else cb(new Error(foo));
  }

  const brake = new Brakes(asyncCall, {timeout: 150});

  brake.exec('bar')
    .then((result) =>{
      console.log(`result: ${result}`);
    })
    .catch(err =>{
      console.error(`error: ${err}`);
    });

Fallback

  function promiseCall(foo){
    return new Promise((resolve, reject) =>{
      if (foo) resolve(foo);
      else reject(foo);
    });
  }

  function fallbackCall(foo){
    return new Promise((resolve, reject) =>{
      resolve('I always succeed');
    });
  }

  const brake = new Brakes(promiseCall, {timeout: 150});

  brake.fallback(fallbackCall)

  brake.exec(false)
    .then((result) =>{
      console.log(`result: ${result}`);
    })
    .catch(err =>{
      console.error(`error: ${err}`);
    });

Health check

  function promiseCall(foo){
    return new Promise((resolve, reject) =>{
      if (foo) resolve(foo);
      else reject(foo);
    });
  }

  function fallbackCall(foo){
    return new Promise((resolve, reject) =>{
      resolve('I always succeed');
    });
  }

  function healthCheckCall(foo){
    return new Promise((resolve, reject) => {
      //this will return 20% true, 80% false
      if (Math.random() > 0.8){
        resolve('Health check success');      
      } else {
        reject('Health check failed');
      }
    });
  }

  const brake = new Brakes(promiseCall, {timeout: 150});

  brake.fallback(fallbackCall);

  brake.healthCheck(healthCheckCall);

  brake.exec(false)
    .then((result) =>{
      console.log(`result: ${result}`);
    })
    .catch(err =>{
      console.error(`error: ${err}`);
    });

Sub Circuits

Brakes exposes the ability to create a main Brakes instance that can then contain subCircuits that all report to a central stats object. This allows all subCircuits to be tripped at the same time when the overall health of all the subCircuits crosses a defined threshold.

See examples/sub-circuit.js for a more complete example.

function promiseCall(foo){
  return new Promise((resolve, reject) =>{
    if (foo) resolve(foo);
    else reject(foo);
  });
}

const brake = new Brakes({
  timeout: 150,
  fallback: () => Promise.resolve('Response from fallback'),
});

const subCircuit1 = brake.subCircuit(promiseCall);
const subCircuit2 = brake.subCircuit(promiseCall);

subCircuit1.exec('bar')
  .then((result) =>{
    console.log(`result: ${result}`);
  })
  .catch(err =>{
    console.error(`error: ${err}`);
  });

// all stats are reported through the main brakes instance
brake.on('snapshot', snapshot => {
  console.log(`Stats received -> ${snapshot}`);
});

Demonstration

For a terminal based demonstration:

General Demo npm install && node examples/example1.js

Fallback Demo npm install && node examples/fallback-example.js

Health check Demo npm install && node examples/healthCheck-example.js

Hystrix Stream Demo npm install && node examples/hystrix-example.js

Sub Circuits npm install && node examples/sub-circuit.js

Methods

Method Argument(s) Returns Description
getGlobalStats N/A globalStats Returns a reference to the global stats tracker
static getGlobalStats N/A globalStats Returns a reference to the global stats tracker
exec N/A Promise Executes the circuit
fallback function (must return promise or accept callback) N/A Registers a fallback function for the circuit
healthCheck function (must return promise or accept callback) N/A Registers a health check function for the circuit
subCircuit function (required), function (optional), opts (optional) N/A Create a new sub circuit that rolls up into the main circuit it is created under.
on eventName, function N/A Register an event listener
destroy N/A N/A Removes all listeners and deregisters with global stats tracker.
isOpen N/A boolean Returns true if circuit is open

Events

Every brake is an instance of EventEmitter that provides the following events:

Configuration

Available configuration options.

Stats

Based on the opts.statInterval an event will be fired at regular intervals that contains a snapshot of the running state of the application.

// ...
  brake.on('snapshot', snapshot => {
    console.log(`Stats received -> ${snapshot}`);
  });
// ...

Example Stats Object

{ name: 'defaultBrake',
  group: 'defaultBrakeGroup',
  time: 1463297869298,
  circuitDuration: 15000,
  threshold: 0.5,
  waitThreshold: 100,
  stats:
   { failed: 0,
     timedOut: 0,
     total: 249,
     shortCircuited: 0,
     latencyMean: 100,
     successful: 249,
     percentiles:
      { '0': 100,
        '1': 102,
        '0.25': 100,
        '0.5': 100,
        '0.75': 101,
        '0.9': 101,
        '0.95': 102,
        '0.99': 102,
        '0.995': 102 }
    }
  }

Global Stats Stream

Brakes automatically tracks all created instances of brakes and provides a global stats stream for easy consumption and reporting on all brakes instances. These streams will aggregate all stat events into one single stream.

const globalStats = Brakes.getGlobalStats();

globalStats.getRawStream().on('data', (stats) =>{
  console.log('received global stats ->', stats);
});

Hystrix Dashboard

Using the global stats stream with a special transform, brakes makes it incredibly easy to generate a SSE stream that is compliant with the hystrix dashboard and turbine.

monitorer

Example:

const globalStats = Brakes.getGlobalStats();

/*
Create SSE Hystrix compliant Server
*/
http.createServer((req, res) => {
  res.setHeader('Content-Type', 'text/event-stream;charset=UTF-8');
  res.setHeader('Cache-Control', 'no-cache, no-store, max-age=0, must-revalidate');
  res.setHeader('Pragma', 'no-cache');
  globalStats.getHystrixStream().pipe(res);
}).listen(8081, () => {
  console.log('---------------------');
  console.log('Hystrix Stream now live at localhost:8081/hystrix.stream');
  console.log('---------------------');
});

To aid in testing it might be useful to have a local instance of the hystrix dashboard running:

docker run -d -p 8080:8080 -p 9002:9002 --name hystrix-dashboard mlabouardy/hystrix-dashboard:latest

Additional Reading: Hystrix Metrics Event Stream, Turbine, Hystrix Dashboard


Development

We gladly welcome pull requests and code contributions. To develop brakes locally clone the repo and use the following commands to aid in development:

npm install
npm run test
npm run test:lint
npm run coverage

Change Log

See: https://github.com/awolden/brakes/releases