Closed jams2 closed 1 week ago
Thanks for the detailed issue! Unfortunately we don't have any maintainers with much Django experience at the moment, so I'd be delighted to receive a PR for these. (1) should be entirely straightforward; for (2) we'll want to make sure that we're not breaking backwards-compatibility in some way.
Hello, I have been investigating the feasibility of publishing a set of Hypothesis strategies and helpers for testing Wagtail projects. Wagtail is a CMS Library for Python, built on Django. I've been pleased with the results I've had using Hypothesis to test some of my other projects, and think there would be some benefit to integrating property-based testing into Wagtail projects (I am eager to find ways to improve the quality of software delivered to clients).
While working on this, I've come across some issues with the strategies that are generated for Django's
ModelChoiceField
, which Wagtail uses frequently, andModelMultipleChoiceField
.The following test cases illustrate the issues.
1.
from_field
withModelChoiceField
requires using adeferred
strategyThis makes sense, as it looks like
from_field
generates asampled_from(field.choices)
strategy, which causes database access to evaluate the choices.A similar error is reported when using django's
manage.py test
test runner instead of pytest. This is a little confusing as the behaviour differs from usingfrom_form
in agiven
decorator, where it seems the field strategies are not eagerly evaluated. From my perspective, it would be ideal iffrom_field
handled delaying the database access to the time that the first example is evaluated, which seems to be in a context where the database connection is available. The consistency between different field types would make this more user friendly.2. Generated strategies yield incorrect types
The strategies generated for both
ModelChoiceField
andModelMultipleChoiceField
yield instances offorms.models.ModelChoiceIteratorValue
, which are not valid values for a form — see the final test. I would expect either a model instance or a queryset, respectively, to be generated (or a primary key or list of primary keys — I need to double-check the order of validation and upcasting in the form instantiation and validation pathways). My expectation is thatfrom_form
should generate values that are valid for the defined form fields — is that assumption correct?I am happy to submit a patch for these issues.