[RFC] Add example pdb checker scenario.

[OddBloke]

Here's an example behave scenario testing the pdb checker. Running behave gives the following output (though with more colour):

Feature: pdb checker # features/pdb_checker.feature:1

  Background:   # features/pdb_checker.feature:3

  Scenario: When enabled, the pdb checker raises an error with pdb statements        # features/pdb_checker.feature:7
    Given that I am in a git repository                                              # ../../home/daniel/dev/captainhook/features/steps/git_steps.py:7 0.006s
    And I have installed captainhook                                                 # ../../home/daniel/dev/captainhook/features/steps/install_steps.py:6 0.069s
    When I create a file called "pdb_fail.py" containing a pdb.set_trace() statement # ../../home/daniel/dev/captainhook/features/steps/file_steps.py:4 0.000s
    And I git add "pdb_fail.py"                                                      # ../../home/daniel/dev/captainhook/features/steps/git_steps.py:12 0.006s
    And I attempt to commit                                                          # ../../home/daniel/dev/captainhook/features/steps/git_steps.py:17 0.239s
    Then I see an error                                                              # ../../home/daniel/dev/captainhook/features/steps/output_steps.py:5 0.000s
    And there are uncommitted changes                                                # ../../home/daniel/dev/captainhook/features/steps/git_steps.py:22 0.006s

1 feature passed, 0 failed, 0 skipped
1 scenario passed, 0 failed, 0 skipped
7 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.326s

[OddBloke]

This possibly might have more (maintenance/creation) overhead than we want from our testing.

On the other hand, if we were to do it all in Python, I think we would rapidly end up with a confused mess of file-handling and actual tests; this gives us a clear separation between what we want to test (the feature files) and how we actually achieve that (playing around with files on disk).

I don't feel especially strongly about how we do it, but I do feel like we should have some sort of testing which uses "git commit" as the entry point; else we'll never really be confident that installation works as we expect.

[alexcouper]

This looks great!

I like the idea of having these end-to-end tests included to - and they can be run separately, so all good. Just going to give this a quick spin..

[alexcouper]

Only thought is that better isolation could be a good thing. Currently all checks are run in the behave test.

But I guess that's something we can deal with later...

[OddBloke]

They should be running in a separate temporary directory for each scenario.

Admittedly, I haven't observed this working... On 18 May 2014 22:07, "Alex Couper" notifications@github.com wrote:

Only thought is that better isolation could be a good thing. Currently all checks are run in the behave test.

But I guess that's something we can deal with later...

— Reply to this email directly or view it on GitHubhttps://github.com/alexcouper/captainhook/pull/34#issuecomment-43451636 .

[alexcouper]

Ah, no - what i mean is that i think the behave test should create a tox.ini and disable the irrelevant plugins.

The only evidence I have that this isn't the case is a suspicious __pycache__ folder lying around in the temp dir that was created - which suggests that the py3 checker was run.

[OddBloke]

Ah, I see. Will look at that this week. :-) On 18 May 2014 22:27, "Alex Couper" notifications@github.com wrote:

Ah, no - what i mean is that i think the behave test should create a tox.ini and disable the irrelevant plugins.

The only evidence I have that this isn't the case is a suspicious pycache folder lying around in the temp dir that was created - which suggests that the py3 checker was run.

— Reply to this email directly or view it on GitHubhttps://github.com/alexcouper/captainhook/pull/34#issuecomment-43452182 .

[OddBloke]

DocOz is a genius! I am down 12 pounds! http://huff.to/z5PxC9