Support py.test

[benoitbryon]

• In documentation, tell how to run healthchecks with py.test. See also the explanations for nose
• If necessary, add helpers in code

[ftobia]

I'll do some work on this the next chance I get.

It may entail getting the hospital.healthcheck decorator to also add a pytest marker, or define a pytest marker like pytest.mark.healthcheck that behaves the same as hospital.healthcheck, or some combination of the two.

[benoitbryon]

It may entail getting the hospital.healthcheck decorator to also add a pytest marker

Ok. Note: better if hospital does not have py.test as a requirement (i.e. avoid import pytest if possible in hospital's code).

or define a pytest marker like pytest.mark.healthcheck that behaves the same as hospital.healthcheck

Would it mean py.test can only discover healthchecks that have been written with this specific pytest.mark.healthcheck? i.e. users would have to use @pytest.mark.healthcheck instead of @hospital.healthcheck?

[ftobia]

Okay I've done some poking around. pytest does not have the same --attr option that nose does. It does have a -k for running tests by name, and a -m option for running tests with a specific marker. (From what I can tell, pytest's -m has basically the same intention as nose's -a, just a slightly different implementation).

Regarding your second question, it should be possible to write a pytest hook (without having pytest installed) that can apply a @pytest.mark.healthcheck marker to tests where is_healthcheck() evaluates to True. So the tests would still use the hospital.healthcheck decorator. But we need a pytest marker to make the collection easier.

I agree that hospital should not have py.test as a requirement. But how would you feel about checking whether pytest is installed, and if it is, having the hospital.healthcheck decorator applying a pytest marker? This is probably functionally equivalent to the above implementation, actually, so maybe that one is preferable.

Even without doing that, I can create a pytest plugin that's available by installing python-hospital. This is all without depending on having pytest installed. It can provide the following things:

• register a marker (probably called "healthcheck")
• (tests with the marker can also have the hospital.healthcheck decorator applied to them, for completeness)
• add a --healthcheck command-line option to only run the health checks that get collected
• add an entry point for pytest so that the plugin will be used by pytest -- this should do nothing if pytest isn't installed

Let me know if that sounds reasonable, and if you have any concerns, or changes you would like to make. Then I will start working on it.

[benoitbryon]

Your plan sounds reasonable :)

I agree that hospital should not have py.test as a requirement. But ...

... provided it does not block the feature. I mean, if avoiding the py.test dependency gets really complicated, do not bother too much about it:

• let's try without py.test dependency, but if it is not easy...
• first have a pull-request where the feature "py.test runs healthchecks" works, whatever the implementation,
• then we will refactor as soon as we have better ideas ;)

[ftobia]

Excellent. I will try to have a pull request within a week or so.

[ftobia]

I think I have a minimal feature. The hospital.pytest_plugin module provides a few hooks to mark tests decorated with @hospital.healthcheck with @pytest.mark.healthcheck. Now that I think about it, I could probably bypass the double decoration and just use the is_healthcheck check. Consider this a work in progress.

setup.py contains an entrypoint for py.test to pick up the plugin.

Example of running py.test with the --healthcheck option:

$ py.test foo -vv --healthcheck
======================================== test session starts ========================================
platform darwin -- Python 2.7.5 -- py-1.4.20 -- pytest-2.5.2 -- env/bin/python
plugins: hospital
collected 3 items

foo/test_foo.py:3: test_one PASSED
foo/test_foo.py:8: test_two PASSED
foo/test_foo.py:13: test_three SKIPPED

================================ 2 passed, 1 skipped in 0.01 seconds ===============================

The contents of foo/test_foo.py:

import hospital

@hospital.healthcheck
def test_one():
    assert True

@hospital.healthcheck
def test_two():
    assert True

def test_three():
    assert True

I think that ideally, instead of skipping non-healthchecks, running with --healthcheck won't even show them in the test output. It wasn't immediately obvious to me how to do this.

I wanted to get your feedback before sending a pull request because I don't consider this code pull request worthy quite yet.

[ftobia]

And of course I'll need to write some documentation for this feature.

Any ideas how I can write tests for this? Where would you like me to include the tests (like, what packages should the tests be in)?

[benoitbryon]

I think that ideally, instead of skipping non-healthchecks, running with --healthcheck won't even show them in the test output. It wasn't immediately obvious to me how to do this.

Skipping may be enough for a start. It can be improved later when we have a better alternative.

I wanted to get your feedback before sending a pull request because I don't consider this code pull request worthy quite yet.

Thanks for the feedback :) Perhaps not ready yet, but the results you showed look fine :)

Any ideas how I can write tests for this? Where would you like me to include the tests (like, what packages should the tests be in)?

hospital's own tests are currently in https://github.com/python-ospital/hospital/tree/master/hospital/tests. You can add some module there. You can take advantage of predictable healthchecks in order to check the results. As an example, that is what hospital.tests.cli does.

One thing I do not know how to manage is the "py.test" dependency. I mean, if you need to run py.test in your tests... then tests depend on py.test, and if tests are in hospital packages, then hospital package depends on py.test... I currently see 3 options to solve this:

• accept py.test as a dependency of hospital... until we find a better idea.
• have a tests/ folder at the root of the repository. It would contain tests that do not fit in hospital/tests. Since it is not inside hospital package, it can depend on py.test. In tox.ini we can install both nose and py.test.
• have a third-party project which focuses on py.test integration of hospital. This one can depend on py.test... The choice if none of the options above works.

[ftobia]

I tend to use option 2 in my own code (have a tests/ folder at the root of the repository). Then I use the tests_require setup.py option. I'll give that a try and see how it works.

[ftobia]

In writing the docs, I see that pytest de-selects tests instead of skipping them e.g. when you do py.test -m "not healthcheck". I need to figure out how to do this programmatically so the --healthcheck flag will make that happen.

[ftobia]

https://github.com/python-hospital/hospital/pull/57