docker update, setup_raises, match argument, "coverage"

[svenevs]

• [x] Update docker build to install things before source copy.
  • Fixes #12 (nested dependency cannot build without gcc and musl-dev on alpine image).
  • Means that sudo make test does not require the docker image to rebuild every time source is updated.
• [x] Add setup_raises marker, re-factor logic to be reused.
  • See below checklist.
• [x] Add match=r"expr" argument. Being able to supply regular expressions is incredibly useful.
  • Very similar to with pytest.raises(...), except explicit failure given if both message and match are provided.
    • I cannot think of a valid reason why you would ever need both at the same time.
    • Not allowing both makes logic for message testing easier. If allowing both, which to you test? Both match and message? How do you format the failure_message? Basically, force message or match not both can be removed, but need counsel on what you want when both supplied.
• [x] Add some testing coverage :tada:
  • Strange behavior with import statements, likely because of how pytest actually uses this plugin.
    • Attempts at mocking cannot rectify, see setup.cfg.
  • See it in action! It was really useful to have this during this re-structure because there were previously many things I introduced that were never tested and now they are :slightly_smiling_face:
  • [ ] Would you like to add an integration with something like https://codecov.io/ ? It will basically always be at 98% due to unknown coverage parsing, but hey that's pretty good :wink:
    • All you (or Authentize) need to do is enable the integration and I can add the badge and update .travis.yml to actually do the reporting. Codecov isn't the only one, coveralls is popular, I just love the codecov interface / it's really easy to upload to.
• [ ] Update README to explain new additions (match and setup_raises).
  • I'm hoping to get some feedback on if what I'm proposing here is actually valid, instruction update pending discussion on pytest/#4175

How it All Works

This is also detailed thoroughly in the docstrings, but here's the overview of how it all works together.

1. Share as much code as possible between raises and setup_raises markers.
   • Hook impl executed (either pytest_runtest_setup for setup_raises or pytest_runtest_call for raises like usual).
   • item, outcome, and string marker_name indicating whether its coming from setup or call forwarded to _pytest_raises_validation.
2. If marker setup_raises or raises are found, there are now three distinct scenarios tested.
   • Case 1: test raised exception, and it isinstance of the expected exception class (either via exception=ExceptionClass or the default catch-all Exception if unspecified.
     • If the message or match are specified, the exception message is then validated.
     • If the message or match do not check out, _pytest_fail_by_mark_or_set_excinfo is called.
   • Case 2: test raised an exception, but it was of an unexpected type.
     • _pytest_fail_by_mark_or_set_excinfo fails the test.
   • Case 3: the test did not raise, but since it was marked with either setup_raises or raises test is failed using _pytest_fail_by_mark_or_set_excinfo.
3. Distinctions must be made between setup_raises and raises. I don't understand the details (hence the pytest issue linked above), but basically there are problems with setting outcome.excinfo or calling pytest.fail(...).
   • The distinctions between the two are made because we do not want tests to ERROR, we want them to FAIL. See pytest/#4175 for details discovered by trial-and-error.
4. So to deal with this, I create a "handshake" using an intentionally obtusely named marker for the setup_raises so that tests properly FAIL rather than ERROR. The general idea is: during pytest_runtest_setup, cases 1-3 are "flagged" with a secret marker, which is then checked for during pytest_runtest_call. Search secret_marker in code.
5. Since (4) had to go down like this, I leveraged the same trick to also add some checks on things like is the exception=??? actually valid, etc.

I wouldn't go as far as to say the implementation is particularly clean, but it works reliably and I really need it. This is the revisited implementation of what I was discussing in #9.

There is a small concern. Consider the newly added test_pytest_mark_setup_raises_unexpected_exception_fixture in test_raises.py:

import pytest

class SomeException(Exception):
    pass

class AnotherException(Exception):
    pass

@pytest.fixture
def raise_me():
    raise AnotherException('the message')

@pytest.mark.setup_raises(exception=SomeException)
def test_pytest_mark_setup_raises_unexpected_exception_fixture(raise_me):
    pass

In any case where setup_raises is used, the test function body should be empty (just a single pass to compile). Logically, I cannot think of any scenario where if you expect setup_raises you also want to have an implementation in the test function. But the implications are:

1. If you have testing code in the function body and are banking on your setup raising, this is a contradiction.
2. You therefore cannot mark a function with both @pytest.mark.raises and @pytest.mark.setup_raises. Only one is allowed -- you either are expecting the setup to fail, in which case the test body should be empty, or you want the test body to fail, in which case the test setup better not raise since you need a valid test setup.

---

Anyway, this is a big change and should definitely be reviewed. I hesitated to put them all in one PR, but at the end of the day the docker / pylint fixes were needed for me to test things, and adding the match independently in the context of the changes for setup_raises would make for a guaranteed merge conflict between the two.

I look forward to updating / fixing / changing anything you need. If you don't want this change in this repo then I'll package my own because I can't fully test my code without setup_raises :scream: