Implement GreenTestLoader and load_test protocol

[althonos]

In regards with #87 and #88, I implemented the load_test protocol. To do so, I actually moved functions from green.loader to a new class green.loader.GreenTestLoader, inheriting from unittest.TestLoader.

This way, you can actually use the builtin unittest.TestLoader.loadTestFromName, which can run load_tests if needed.

Changes are backwards-incompatible (although some compatibility patches can be made), as I registered all functions of green.loader as methods of green.loader.GreenTestLoader.

I also removed green.loader.loadFromModule as it was duplicating the unittest.TestLoader.loadFromModule code.

I added a test containing a TestCase that is expected to fail, but the load_tests function in it will monkey patch the TestCase and make the test succeed.

[althonos]

NB: Don't merge right now, I have some strange issues on some tests I'm not sure how to reproduce, where ProtoTest objects exit the queue in runner.run.

[coveralls]

Coverage remained the same at 100.0% when pulling dce759e497b77da86ab4e00131de9e578ca9f9e8 on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[coveralls]

Coverage remained the same at 100.0% when pulling dce759e497b77da86ab4e00131de9e578ca9f9e8 on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[coveralls]

Coverage remained the same at 100.0% when pulling dce759e497b77da86ab4e00131de9e578ca9f9e8 on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[CleanCut]

Wow! You've done a lot of work on this! Let me know when you think it is ready, and I'll review it.

[althonos]

Well, if this build doesn't break, it will work well enough for me :)

Concerning doctests however, green ended up completely freezing when a DocTestSuite was added to the tests with the load_tests protocol. Unfortunately, I am not (yet) sure how to fix that, I think DocTestSuite have a different logic that should be studied, but it doesn't really come into the scope of this PR.

[coveralls]

Coverage remained the same at 100.0% when pulling 00e7cc3d4d86b168139b72fb684512a4618c49c1 on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[coveralls]

Coverage remained the same at 100.0% when pulling 00e7cc3d4d86b168139b72fb684512a4618c49c1 on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[coveralls]

Coverage remained the same at 100.0% when pulling 00e7cc3d4d86b168139b72fb684512a4618c49c1 on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[althonos]

Well actually, I'll add a test to make sure the bug I fixed doesn't come back, just wait some more ;)

[coveralls]

Coverage remained the same at 100.0% when pulling 8a41232f9ff18aaf13a008f08fd8ff650fdba539 on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[coveralls]

Coverage remained the same at 100.0% when pulling 8a41232f9ff18aaf13a008f08fd8ff650fdba539 on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[coveralls]

Coverage remained the same at 100.0% when pulling 1b76203962dda9529199e9c705c94a80c3431d05 on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[althonos]

NB: I also added a method that will flatten test suites returned by GreenTestLoader.loadTests... methods, to make sure the returned test suite does not nest other test suites (this is easier to debug if the test suite is actually just a list instead of a nested container).

[coveralls]

Coverage remained the same at 100.0% when pulling 4e2efa799eef213ceba836f626d92490e8f718d2 on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[coveralls]

Coverage remained the same at 100.0% when pulling 4e2efa799eef213ceba836f626d92490e8f718d2 on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[althonos]

@CleanCut : I suppose you can review this PR now ! ;)

[althonos]

Concerning the reorganisation, these are the advantages:

• Grouping everything into a loader class makes it easier to integrate with another unittest framework if needed, since now the "loader" is a proper object inheriting from unittest.TestLoader instead of collection of functions, and actually follows the unittest API.
• That said, the load_tests function at the root of a test package takes 3 arguments: loader, tests, pattern, where loader is supposed to be a loader object. I was able to make it work by passing the module itself as an object load_tests(sys.modules[__name__], ...) but because green.loader does not use the same names as unittest.TestLoader, users would potentially call functions using the unittest name (as they should) instead of the green name (for instance, loadFromModule vs loadTestsFromModule).
• Some code duplication can be avoided by reusing unittest code, as the current implementation only did duplicate the loadTestsFromModule and loadTestsFromName
• Static methods are just a clearer way of declaring functions "belonging" to a class, so the expected behaviour is always the same: create a loader instance, then call its methods.

At some point, I would have had to create a loader class anyway, since it is required when using load_tests: so if I had done for instance a stub class, registering the actual functions as its own methods, it would not have worked very well, because the functions themselves would have called the other functions instead of the loader methods.

Anyway, this is what I'll do right now:

• I'll fix the indents (although this, too, drives me crazy)
• I can write wrapper functions for backward compatibility, that could raise a DeprecationWarning and then do what they are expected to do.

[CleanCut]

One of Green's design decisions (vaguely described under "Traditional") is to only be a test runner for vanilla unittest. In other words, we intentionally avoid having anyone try to use Green as a library for replacing/extending unittest objects. This is to intentionally limit the scope of what Green tries to do and be, so that Green can be even better at what it does do. This is a different choice than other projects that choose to also provide unittest replacement/enhancement objects (pytest, nose, trial, etc.).

• Given the design decision described above, making Green internals easier to integrate with something else isn't an advantage.

• It is fine if we need to rename functions/methods if they need to get passed into unittest which expects a different name. There should never be any "users" of our internal code, so internal renaming should not affect users in any way.

• I am not certain what you mean about code duplication.

• If we need to group stuff into the class in order for it to work passing it into unittest, then that's a great argument for organizing the stuff into the class. Lets not include anything that unittest doesn't need, though. (I haven't checked what it does or doesn't need).

There's no reason to write "wrapper" functions! No one else should be using our code...so...what would they be for?

[althonos]

If we need to group stuff into the class in order for it to work passing it into unittest, then that's a great argument for organizing the stuff into the class. Lets not include anything that unittest doesn't need, though. (I haven't checked what it does or doesn't need).

I'll fix that, so the GreenTestLoaded only provides the required classes.

I am not certain what you mean about code duplication.

Basically, if I'm not using a class, then I would have to copy/paste the code from unittest.TestLoader.loadTestsFromModule and edit it to use the local green functions, whereas subclassing allows reusing the existing implementation.

[coveralls]

Coverage remained the same at 100.0% when pulling cc525e8652fd8691316ebeae1c9b5b0e776832fc on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[coveralls]

Coverage remained the same at 100.0% when pulling cc525e8652fd8691316ebeae1c9b5b0e776832fc on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[coveralls]

Coverage remained the same at 100.0% when pulling cc525e8652fd8691316ebeae1c9b5b0e776832fc on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[coveralls]

Coverage remained the same at 100.0% when pulling 19b41ecf826adf371f6b9f4720b78163b766a7eb on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[coveralls]

Coverage remained the same at 100.0% when pulling 19b41ecf826adf371f6b9f4720b78163b766a7eb on althonos:master into 8a7a0bdefc2ed7f2ad1121cff12e967ae0017924 on CleanCut:master.

[CleanCut]

Included in just-released version 2.9.0.

[CleanCut]

@althonos Thank you for all your hard work! This release was pretty much all you. :-)

[althonos]

Thanks ! When you use a tool daily, you just wish it gets better ;)