Sanitise Data leaving Travis

[matt-oc]

The data leaving the Travis module (and npm and github) needs to be decided upon before it leaves. The problem is we have alot of data for each module and the data for each query may be different. Below are three example dumps from the Travis module:

Searching for Fuge:

{
  "name": "fuge",
  "url": "https:\/\/travis-ci.org\/apparatus\/fuge",
  "id": 6206563,
  "slug": "apparatus\/fuge",
  "active": true,
  "description": "",
  "last_build_id": 120434834,
  "last_build_number": "128",
  "last_build_state": "passed",
  "last_build_duration": 286,
  "last_build_language": null,
  "last_build_started_at": "2016-04-03T14:47:51Z",
  "last_build_finished_at": "2016-04-03T14:50:32Z",
  "github_language": "JavaScript",
  "sudo": false,
  "language": "node_js",
  "node_js": [
    0.1,
    0.12,
    4
  ],
  ".result": "configured",
  "group": "stable",
  "dist": "precise"
}

Lodash:

{
  "name": "lodash",
  "url": "https:\/\/travis-ci.org\/lodash\/lodash",
  "id": 48500,
  "slug": "lodash\/lodash",
  "active": true,
  "description": "A modern JavaScript utility library delivering modularity, performance, & extras.",
  "last_build_id": 120653169,
  "last_build_number": "4380",
  "last_build_state": "started",
  "last_build_duration": null,
  "last_build_language": null,
  "last_build_started_at": "2016-04-04T15:48:33Z",
  "last_build_finished_at": null,
  "github_language": "JavaScript",
  "language": "node_js",
  "sudo": false,
  "node_js": [
    "5"
  ],
  "env": [
    "",
    "BIN=\"phantomjs\"",
    "ISTANBUL=true",
    "SAUCE_LABS=true"
  ],
  "matrix": {
    "include": [
      {
        "node_js": "0.10",
        "env": null,
        "dist": "precise"
      },
      {
        "node_js": "0.12",
        "env": null,
        "dist": "precise"
      },
      {
        "node_js": "4",
        "env": null,
        "dist": "precise"
      }
    ]
  },
  "git": {
    "depth": 10
  },
  "branches": {
    "only": [
      "master"
    ]
  },
  "notifications": {
    "webhooks": {
      "urls": [
        "https:\/\/webhooks.gitter.im\/e\/4aab6358b0e9aed0b628"
      ],
      "on_success": "change",
      "on_failure": "always"
    }
  },
  "before_install": [
    "nvm use $TRAVIS_NODE_VERSION",
    "npm set loglevel error",
    "npm set progress false",
    "npm i -g npm@\"^2.0.0\"",
    "PATTERN[0]=\"|\\s*if\\s*\\(isHostObject\\b[\\s\\S]+?\\}(?=\\n)|\"\nPATTERN[1]=\"|\\s*if\\s*\\(enumerate\\b[\\s\\S]+?\\};\\s*\\}|\"\nPATTERN[2]=\"|\\s*while\\s*\\([^)]+\\)\\s*\\{\\s*iteratee\\(index\\);\\s*\\}|\"\nPATTERN[3]=\"|\\s*else\\s*\\{\\s*assocSet\\(data\\b[\\s\\S]+?\\}|\"\nPATTERN[4]=\"|\\bcase\\s+(?:dataView|set|map|weakMap)CtorString:.+|g\"\nPATTERN[5]=\"|\\bindex,\\s*iterable\\)\\s*===\\s*false\\)[^}]+?(break;)|\"\nPATTERN[6]=\"|\\s*if\\s*\\(\\!lodashFunc\\)\\s*\\{\\s*return;\\s*\\}|\"\nPATTERN[7]=\"|\\s*define\\([\\s\\S]+?\\);|\"\nPATTERN[8]=\"|\\s*root\\._\\s*=\\s*_;|\"\n\nif [ $ISTANBUL == true ]; then\n  set -e\n  for PTRN in ${PATTERN[@]}; do\n    node .\/test\/remove.js \"$PTRN\" .\/lodash.js\n  done\nfi\n",
    "git clone --depth=10 --branch=master git:\/\/github.com\/lodash\/lodash-cli .\/node_modules\/lodash-cli && mkdir $_\/node_modules && cd $_ && ln -s ..\/..\/..\/ .\/lodash && cd ..\/ && npm i && cd ..\/..\/",
    "node .\/node_modules\/lodash-cli\/bin\/lodash -o .\/dist\/lodash.js"
  ],
  "script": [
    "[ $ISTANBUL == false ]   || istanbul cover -x \"**\/vendor\/**\" --report lcovonly .\/test\/test.js -- .\/lodash.js",
    "[ $ISTANBUL == false ]   || [ $TRAVIS_SECURE_ENV_VARS == false ] || (cat .\/coverage\/lcov.info | coveralls) || true",
    "[ $ISTANBUL == false ]   || [ $TRAVIS_SECURE_ENV_VARS == false ] || (cat .\/coverage\/coverage.json | codecov) || true",
    "[ $SAUCE_LABS == true ]  || [ $ISTANBUL == true ] || cd .\/test",
    "[ $SAUCE_LABS == true ]  || [ $ISTANBUL == true ] || $BIN $OPTION .\/test.js ..\/lodash.js",
    "[ $SAUCE_LABS == true ]  || [ $ISTANBUL == true ] || [ $TRAVIS_SECURE_ENV_VARS == false ] || $BIN $OPTION .\/test.js ..\/dist\/lodash.min.js",
    "[ $SAUCE_LABS == false ] || rm -rf .\/node_modules\/lodash",
    "[ $SAUCE_LABS == false ] || ($BIN .\/node_modules\/lodash-cli\/bin\/lodash modularize exports=node -o .\/node_modules\/lodash && node .\/node_modules\/lodash-cli\/bin\/lodash -d -o .\/node_modules\/lodash\/lodash.js)",
    "[ $SAUCE_LABS == false ] || $BIN .\/node_modules\/lodash-cli\/bin\/lodash core -o .\/dist\/lodash.core.js",
    "[ $SAUCE_LABS == false ] || npm run build",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"lodash tests\"     runner=\"test\/index.html?build=..\/dist\/lodash.js&noglobals=true\"     tags=\"development\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"lodash tests\"     runner=\"test\/index.html?build=..\/dist\/lodash.min.js&noglobals=true\" tags=\"production\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"lodash-fp tests\"  runner=\"test\/fp.html?noglobals=true\"                                tags=\"development\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"underscore tests\" runner=\"test\/underscore.html?build=..\/dist\/lodash.js\"               tags=\"development,underscore\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"underscore tests\" runner=\"test\/underscore.html?build=..\/dist\/lodash.min.js\"           tags=\"production,underscore\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"backbone tests\"   runner=\"test\/backbone.html?build=..\/dist\/lodash.js\"                 tags=\"development,backbone\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"backbone tests\"   runner=\"test\/backbone.html?build=..\/dist\/lodash.min.js\"             tags=\"production,backbone\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"backbone tests\"   runner=\"test\/backbone.html?build=..\/dist\/lodash.core.js\"            tags=\"development,backbone\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"backbone tests\"   runner=\"test\/backbone.html?build=..\/dist\/lodash.core.min.js\"        tags=\"production,backbone\""
  ],
  ".result": "configured",
  "global_env": [
    "BIN=\"node\" ISTANBUL=false OPTION=\"\"",
    "SAUCE_LABS=false SAUCE_USERNAME=\"lodash\"",
    {
      "secure": "tg1JFsIFnxzLaTboFPOnm+aJCuMm5+JdhLlESlqg9x3fwro++7KCnwHKLNovhchaPe4otC43ZMB\/nfWhDnDm11dKbm\/V6HlTkED+dadTsaLxVDg6J+7yK41QhokBPJOxLV78iDaNaAQVYEirAgZ0yn8kFubxmNKV+bpCGQNc9yU="
    }
  ],
  "group": "stable",
  "dist": "precise"
}

Nodezoo:

{
  "name": "nodezoo",
  "url": "https:\/\/travis-ci.org\/rjrodger\/nodezoo",
  "id": 376765,
  "slug": "rjrodger\/nodezoo",
  "active": true,
  "description": "A microservices workshop for the Seneca framework.",
  "last_build_id": 110126578,
  "last_build_number": "37",
  "last_build_state": "failed",
  "last_build_duration": 20,
  "last_build_language": null,
  "last_build_started_at": "2016-02-18T13:56:04Z",
  "last_build_finished_at": "2016-02-18T13:56:24Z",
  "github_language": "JavaScript",
  ".result": "not_found",
  "language": "ruby",
  "group": "stable",
  "dist": "precise"
}

Proposed object:

{
"name":
"url":
"id":
"group":
"active":
"buildState":
"lastBuilt":
}

[mcdonnelldean]

@matt-oc Can you run them through a pretty printer and use the code formatting, otherwise I can't read them

[matt-oc]

I like ID, Group, Last State and Time, they are valuable and seem to be consistent and likely to have correct information (the language for instance is usually incorrect on Travis)

[matt-oc]

@mcdonnelldean I'm trying to find one to give a nice output, bear with me!

[mcdonnelldean]

Also, A link to the latest build

[matt-oc]

Yes thats a good point so we have ID, Group, Last State and time, and a URL link to the Travis page.

That output looks a bit better now too :+1:

[mcdonnelldean]

Can you make a fake payload at the bottom of your original message, with the ones you think are right, I can edit accordingly then.

[mcdonnelldean]

As a JSON object

[matt-oc]

Initial proposed object now implemented

[mcdonnelldean]

@matt-oc Lets leave this and the others open for the time being, just in case we want more from them

[matt-oc]

@mcdonnelldean Yep will do, I'm updating the original comment whenever the data is changed so people know where we are with regards to data.