Validating hive data with swiz

Problem

If we discover we have stored invalid data due to a bug in swiz, we probably want to perform an audit to find all the invalid data.

Example

Swiz had a bug that allowed for invalid hostnames to be set in the target_hostname field of a check.

Step 1: Fix the bug in swiz

e.g. https://github.com/racker/node-swiz/pull/52

Step 2: Get the relevant data from hive

Step 3: Setup a simple test environment

Ensure you have a copy of the corrected version of swiz. If you have fixed it locally and the version is not yet bumped in npm, you can just make a new directory for your test which has swiz in a node_modules folder under it with all the relevant changes.

The directory will just contain the script you write to test the data and the swiz module.

$ find . -maxdepth 2
.
./node_modules
./node_modules/swiz
./validate-target-hostnames.js

Step 3: Write a script to pass the data through swiz

Need to ensure you use the correct validator(s) you need to pass the data through. In this case isAllowedFQDNOrIP.

var readline = require('readline');
var swiz = require('swiz');
var V = swiz.Valve;
var C = swiz.Chain;

var validIps = 0,
    invalidIps = 0;

var v = new V({
  a: C().isAllowedFQDNOrIP([])
});

var lineReader = readline.createInterface({
  input: require('fs').createReadStream('hostnames.log')
});

lineReader.on('line', function (line) {
  if (line) {
    validateIp(v, line);
  }
})
.on('error', function (err) {
  console.error('Error occurred', err);
})
.on('close', function() {
  console.log('Valid:', validIps);
  console.log('Invalid:', invalidIps);
});

function validateIp(validator, value) {
  var obj = {'a': value};
  validator.check(obj, function(err, cleaned) {
    if (err) {
      console.error('Invalid IP: ' + value + ' (' + err.message + ')');
      invalidIps++;
    } else {
      validIps++;
    }
  });
}

Step 4: Run the script

$ node validate-target-hostnames.js
Invalid IP:  204.232.227.86 (Not a valid hostname, IPv4 or IPv6 address)
Invalid IP: \u0438\u043D\u0434\u0443\u0441\u0442\u0440\u0438\u0430\u043B\u044C\u043D\u044B\u0439\u043F\u0430\u0440\u043A.\u0440\u0444 (Not a valid hostname, IPv4 or IPv6 address)
Invalid IP:  \t5.134.151.72 (Not a valid hostname, IPv4 or IPv6 address)
Invalid IP:  \t34.213.81.195 (Not a valid hostname, IPv4 or IPv6 address)
Invalid IP:  \t4.236.110.143 (Not a valid hostname, IPv4 or IPv6 address)
Invalid IP: \u00A071.21.210.231 (Not a valid hostname, IPv4 or IPv6 address)
Invalid IP: bob (Not a valid hostname, IPv4 or IPv6 address)
Invalid IP: bob (Not a valid hostname, IPv4 or IPv6 address)
Invalid IP: bob (Not a valid hostname, IPv4 or IPv6 address)
Invalid IP: bob (Not a valid hostname, IPv4 or IPv6 address)
Invalid IP: bob (Not a valid hostname, IPv4 or IPv6 address)
Invalid IP:     10.56.87.11 (Not a valid hostname, IPv4 or IPv6 address)
Valid: 42
Invalid: 12

jjbuchan / docs