[Feature request] Exit codes for Continuous Integration

[Kristinita]

1. Summary

I want to add automatic testing my package via Travis CI or other Continuous Integration service. But even if my package contain failures, I get exit code 0.

2. Expected behavior

• if package_reviewer don't find failures and warnings, Travis CI build passed;
• if package_reviewer find failures and warnings, Travis CI build fail.

3. Actual behavior

See my build:

---

## Report for SashaTravis #############################

### Repository checks ###

Reporting 2 failures:

- Missing a README file

- No semantic version tags found (no tags found at all)

No warnings

### Package checks ###

Reporting 1 failures:

- The binding ['ctrl+a'] unconditionally overrides a default binding

    File: Default (Windows).sublime-keymap

No warnings

For more details on the report messages (for example how to resolve them), go to:

https://github.com/packagecontrol/package_reviewer/wiki

The command "python -m package_reviewer "https://github.com/Kristinita/SashaTravis"" exited with 0.

Done. Your build exited with 0.

4. Settings

My test Sublime Text package with failures — https://github.com/Kristinita/SashaTravis.

My .travis.yml file:

# language
language: python
# Python version
python:
  - "3.7-dev"
# command to install dependencies
install:
  - pip install virtualenv
  - git clone https://github.com/packagecontrol/package_reviewer.git
  - cd package_reviewer
  - virtualenv venv
  - source venv/bin/activate
  - pip install -r requirements.txt
# command to run tests
script:
  - python -m package_reviewer "https://github.com/Kristinita/SashaTravis"

5. Steps to reproduce

I create test repository and integrate it with Travis CI. I push commit to my repository → package with failures have exit code 0.

Thanks.

[FichteFoll]

I think I had this at some point in the source, but it probably got lost because I'm only using the tool in interactive mode currently.

Note that I can only make the exit code non-zero for failures and not for warnings.

[FichteFoll]

Fixed by:

• 3a00dfcb17a6df09d34dbdd34e17ff41afa0f72e: some exit code changes I forgot to commit
• 7a20bc4efea2e68f685259b5daa0054349cb4605: flag to exit with non-zero exit code on warnings