natcap / invest

InVEST®: models that map and value the goods and services from nature that sustain and fulfill human life.
Apache License 2.0
151 stars 63 forks source link

Investigate property-based testing #536

Open phargogh opened 3 years ago

phargogh commented 3 years ago

In working through #523, the subject of testing various valid ranges of possible properties (e.g. GDAL datatypes, numpy datatypes, presence/absence of nodata, etc.) that are used by various InVEST models. In testing the Stormwater model in #523, Emily followed the advice in the Pragmatic Programmer (2nd ed.) : "property-based testing". We ended up deciding not to pursue property-based testing in that PR, but it's worth taking a closer look at some point. Hypothesis looks like one possibility for enabling this kind of support in the unittest test fixtures.

Possible things to consider here are:

emlys commented 2 years ago

Testing with signed byte rasters would be a good use for this. We don't normally use them in tests since they're a little more work to create programmatically. They probably should work, but it's hard to know without trying them out on every model.

emlys commented 2 years ago

Another use case: validation testing. Most models have a few test cases dedicated to asserting that different combinations of args do/don't return validation errors. Parameterized testing could help us run through those combinations more efficiently.

emlys commented 2 years ago

Another use case would be testing that all models can handle rasters with a nodata value of nan (#738)

emlys commented 2 years ago

758 : test all models with vector inputs in both GPKG and shapefile formats