Handle tests/specs with nested scopes

[chrischambers]

This pull request produces better spec output for tests/specs with nested scopes, e.g.

# pytest-describe/bdd style:
def describe_some_api():

    def a_thing():
        assert True

    def describe_filtering():

        def it_works():
            assert True

        def describe_specific_filter():

            def it_also_works():
                assert True

# class style, with nested classes:
class TestSomeApi:

    def test_a_thing(self):
        assert True

    class TestFiltering:

        def test_it_works(self):
            assert True

        class TestSpecificFilter:

            def test_it_also_works(self):
                assert True

Now results in this output:

Some Api:
    [PASS] A thing

    Filtering:
        [PASS] It works

        Specific Filter:
            [PASS] It also works

Previously it would produce:

some/path/to/test.py::describe_some_api
    [PASS] A thing
    [PASS] It works
    [PASS] It also works

The only information lost is the spec_header_format, which by default includes the path to the module/component under test, but that could be reintroduced. Let me know what your thoughts are, Pawel!

[pchomik]

Thanks for code implementation. I will look into this and I have to try it with #14

[chrischambers]

I can break these changes out into separate pull requests if required, by the way.

[chrischambers]

The format of tests/specs would now look like this:

path/to/module_under_test.py:

Some Api:
  ✓ Passing test

  Filtering:
    ✗ Failing test

    Specific filter:
      ? Skipped test

[pchomik]

1. I like the idea with pass/fail/skip indicators. In my opinion we should change them back to [PASS] [FAIL] [SKIP] and add documentation how to set chars from your proposition.
2. Regarding indentation what do you think to support it via another option (e.g. flat=no, by default yes)?

[pchomik]

3. The output with random is usless

   test/test_1.py:
   
   SomeApi:
     Filtering:
       SpecificFilter:
     ✓ It also works
   ✓ A thing
   
   Filtering:
   ✓ It works

   Additional think for documentation at least.

[chrischambers]

Hey there @pchmoik - apologies for the late response, just got back from travelling.

Re: configurability via options, that seems like a good idea, though I think the pretty indicators should probably be the defaults. Ultimately it's up to you of course, but I think the vast majority of users would prefer the cleaner output. :)

As for the output with random - there are 2 approaches. As you say, one would be to document the fact that --nested=true won't play nice with any of the --random plugin options. Another would be a more extensive change: I could add a nicer API for introspecting the current pytest state, which would hopefully include the data from the previous test run. It's a more extensive change, but it would make the code cleaner and mean that the nested output wouldn't depend on sorting the tests. I'll provide an example in a couple of days when I've finished building my new rig.

[holmesjr]

Is there anything we can do to help with this? I'd really like to see pytest-spec and pytest-describe working together.

[AnneTheAgile]

bump. I love this fix. On the ticket for the warning we are trying to find out if the maintainer is around. UserWarning: arguments declared in hookspec issued in pytest 4.2.0 · Issue #21 · pchomik/pytest-spec: https://github.com/pchomik/pytest-spec/issues/21

[pchomik]

@chrischambers thank you for huge and great work. I will add few tests to check describe-based test format. Results provided by the plugin looks now awesome.