On error html5check.py should exit with status code != 0

[bittner]

html5check.py exits with status code != 0 when operational errors occur. That's fine.

Validation Results Not Considered

Unfortunately, validation results aren't considered for the script's status code, hence the script always exists saying "SUCCESS". Even when the HTML document in question has validation errors.

Expected Behavior

As with other command line tools the script should exit with 0 only when everything was alright, including validation results. For warnings the script may exit with 1, and with 2 when errors are present (with tidy as an example). It may be nice to allow for disabling reports of warnings and specific errors in this case.

Current Workarounds

One way I found to get this behavior is:

$ ! ./html5check.py example.html | grep "Error: "

However, the grep is a fragile test and isn't beautiful either. Also, for YAML files, e.g. Travis CI, a few tricks are needed to avoid syntax errors, e.g.:

script:
  - bash -c '! python html5check.py example.html | grep "Error:\ "'

Ignoring specific errors/warnings goes like this: :unamused: (not nice)

# check HTML, but ignore frameborder attribute error (needed for IE)
bash -c '! python html5check.py home/*.html | grep -v "Error:.*frameborder.*attribute.*iframe" | grep "Error:\ "'

[jbuck]

Hmm, I think we should be able to check for this easily enough; if we switch the script to use JSON, we can simply check the length of the messages key: https://wiki.whatwg.org/wiki/Validator.nu_JSON_Output#Example

[bittner]

Re my examples on the Travis-CI YAML file: There is an app for that! -- html5validator

install:
  - pip install html5validator
script:
  - html5validator --root home/ --ignore "Error:.*frameborder.*attribute.*iframe"

Validation is done offline, no need to bug https://html5.validator.nu for that.

Pretty neat!