voxpupuli / metadata-json-lint

Tool to check the validity of Puppet metadata.json files
Apache License 2.0
29 stars 27 forks source link

Update tests to expect specific output #57

Closed james-stocks closed 7 years ago

james-stocks commented 7 years ago

Test cases that expect metadata-json-lint to return non-zero should also test that the output from metadata-json-lint matches some expected output. Otherwise the test will pass if any problem occurs.

This PR also contains a small change to exception handling (if metadata.json does not exist) and a fix to one of the existing tests (so that it will fail for the expected reason)

james-stocks commented 7 years ago

The output of the 'broken' test is:

Error: Unable to parse metadata.json: 743: unexpected token at '{ "name": "puppetlabs-postgresql" "version": "3.4.1", "author": "Inkling/Puppet Labs", "summary": "PostgreSQL defined resource types", "license": "Apache-2.0", "source": "git://github.com/puppetlabs/puppet-postgresql.git", "project_page": "https://github.com/puppetlabs/puppet-postgresql", "issues_url": "https://github.com/puppetlabs/puppet-postgresql/issues", "operatingsystem_support": [ { "operatingsystem": "RedHat", "operatingsystemrelease": [ "5", "6", "7" ] }, { "operatingsystem": "CentOS", "operatingsystemrelease": [ "5", "6", "7" ] }, { "operatingsystem": "OracleLinux", "operatingsystemrelease": [ "5", "6", "7" ] }, { "operatingsystem": "Scientific", "operatingsystemrelease": [ "5", "6", "7" ] }, { "operatingsystem": "Debian", "operatingsystemrelease": [ "6", "7" ] }, { "operatingsystem": "Ubuntu", "operatingsystemrelease": [ "10.04", "12.04", "14.04" ] } ], "requirements": [ { "name": "pe", "version_requirement": ">= 3.2.0 < 3.4.0" }, { "name": "puppet", "version_requirement": "3.x" } ], "dependencies": [ { "name": "puppetlabs/stdlib", "version_requirement": "4.x" }, { "name": "puppetlabs/firewall", "version_requirement": ">= 0.0.4" }, { "name": "puppetlabs/apt", "version_requirement": ">=1.1.0 <2.0.0" }, { "name": "puppetlabs/concat", "version_requirement": ">= 1.1.0 <2.0.0" } ] } '

EDIT: The test seems to specify nothing more than 'broken' so I'm going to assume that this JSON parsing error is the desired expected result.

james-stocks commented 7 years ago

The output of the 'long_summary' test is:

Actual output that needs tested: 'Error: Unable to parse metadata.json: 743: unexpected token at '{ "name": "foo-bar", "version": "1.0.0", "author": "foo/bar", "summary": "This is obviously a way way tooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo long" "license": "Apache-2.0", "source": "" "project_page": "" "issues_url": "" "operatingsystem_support": [ { "operatingsystem": "RedHat", "operatingsystemrelease": [ "5", "6", "7" ] } ], "requirements": [ ], "dependencies": [ ] } '

Is 'unexpected token' actually the expected error for this?

EDIT: This was failing on JSON syntax and not on message length. Added an additional commit to fix the JSON.

nibalizer commented 7 years ago

lgtm

rnelson0 commented 7 years ago

I like that, leaves some of the guessing out, thanks!