search

spencerahill / aospy

Python package for automated analysis and management of gridded climate data
Apache License 2.0
82 stars 11 forks source link

Property-based testing with Hypothesis #260

Open spencerahill opened 6 years ago

spencerahill commented 6 years ago

Property-based testing is a way of writing unit tests that, rather than feeding in a specific piece of test data, repeats each test over a wide range of possible viable input data. This makes the resulting test suite much more thorough and thus more likely to catch bugs and edge cases.

Xarray has implemented this (https://github.com/pydata/xarray/pull/1972) using the Hypothesis library. As always, given that xarray is doing it, I think it would be good for us to also.

We could add/convert to these kind of tests gradually -- it wouldn't need to be all at once. If, once we've gotten the hang of them and can assess their value/performance hit (they do run slower than normal tests), we can decide whether to keep using/enforce/stop using them.

haydenbetts commented 5 years ago

Hey @spencerahill I'm looking for a good first issue to take a run at. This seems like it might be a good one. Also, it seems like a few of the tests in here are good candidates for this methodology https://github.com/spencerahill/aospy/blob/master/aospy/test/test_region.py

What do you think? Would there be a better place to start?

spencerahill commented 5 years ago

Hi @haydenbetts, great that you're interested in helping. Just curious, are you also trying to use the package, or just wanted to contribute? Either is ok :). I know activity in this repo has slowed quite a bit in recent months, but that's just due to us being tied up with other things...it's still under active use and development.

There are a few issues that we tagged low-hanging fruit that should be especially straightforward. But those are more nuts-and-bolts; I suspect this one would be more fun since it involves playing with a cool new technology in Hypothesis.

If you do pursue this one, be sure to follow along the lines of the xarray PR I linked to above re: making a separate CI environment for this...hypothesis should stay as an optional dependency. Also, @spencerkclark has more testing expertise than I do, so I'll let him weigh in too.

haydenbetts commented 5 years ago

Ah - I missed those. I'll start by taking a look at low-hanging fruit and note it on the issue if I start work on one. I'll also monitor discussion here.

Currently - contributor not user. I've gone through the AOSPY tutorial and the xarray examples related to weather data though so I have ~ a slight sense ~ of how these tools are used. I think enough to start contributing.

spencerahill commented 5 years ago

OK cool, that all sounds good. Looking forward to seeing whatever you end up contributing, and happy to assist at any point along the away.