`hypothesis[django]`: CharField causes strange assertion issue

[dakotahorstman]

I'm encountering an issue with hypothesis that I originally thought was due to compatibility issues between pytest-django and hypothesis; however, after running a simple test case with unittest and hypothesis, I believe this issue is being caused by hypothesis. Curiously, this issue occurs on just the CharField on my model and not the BooleanField.

Below, I provide an MRE that can be installed on the most basic django app. If there are any further details I can provide, let me know. Testing is executed with poetry run coverage run manage.py test.

Environment

docker: dev-container, vscode python: 3.11.6 django: 4.2.7 hypothesis[django]: 6.90.0

Full environment

```python alabaster==0.7.13 ; python_version >= "3.11" and python_version < "3.12" amqp==5.2.0 ; python_version >= "3.11" and python_version < "3.12" anyascii==0.3.2 ; python_version >= "3.11" and python_version < "3.12" anyio==4.1.0 ; python_version >= "3.11" and python_version < "3.12" argon2-cffi-bindings==21.2.0 ; python_version >= "3.11" and python_version < "3.12" argon2-cffi==23.1.0 ; python_version >= "3.11" and python_version < "3.12" asgiref==3.7.2 ; python_version >= "3.11" and python_version < "3.12" async-timeout==4.0.3 ; python_version >= "3.11" and python_full_version <= "3.11.2" attrs==23.1.0 ; python_version >= "3.11" and python_version < "3.12" autobahn==23.6.2 ; python_version >= "3.11" and python_version < "3.12" automat==22.10.0 ; python_version >= "3.11" and python_version < "3.12" babel==2.13.1 ; python_version >= "3.11" and python_version < "3.12" beautifulsoup4==4.11.2 ; python_version >= "3.11" and python_version < "3.12" billiard==4.2.0 ; python_version >= "3.11" and python_version < "3.12" blinker==1.7.0 ; python_version >= "3.11" and python_version < "3.12" brotli==1.1.0 ; python_version >= "3.11" and python_version < "3.12" cachetools==5.3.2 ; python_version >= "3.11" and python_version < "3.12" celery==5.3.6 ; python_version >= "3.11" and python_version < "3.12" certifi==2023.11.17 ; python_version >= "3.11" and python_version < "3.12" cffi==1.16.0 ; python_version >= "3.11" and python_version < "3.12" cfgv==3.4.0 ; python_version >= "3.11" and python_version < "3.12" channels-redis==4.1.0 ; python_version >= "3.11" and python_version < "3.12" channels==4.0.0 ; python_version >= "3.11" and python_version < "3.12" channels[daphne]==4.0.0 ; python_version >= "3.11" and python_version < "3.12" chardet==5.2.0 ; python_version >= "3.11" and python_version < "3.12" charset-normalizer==3.3.2 ; python_version >= "3.11" and python_version < "3.12" click-didyoumean==0.3.0 ; python_version >= "3.11" and python_version < "3.12" click-plugins==1.1.1 ; python_version >= "3.11" and python_version < "3.12" click-repl==0.3.0 ; python_version >= "3.11" and python_version < "3.12" click==8.1.7 ; python_version >= "3.11" and python_version < "3.12" colorama==0.4.6 ; python_version >= "3.11" and python_version < "3.12" configargparse==1.7 ; python_version >= "3.11" and python_version < "3.12" constantly==23.10.4 ; python_version >= "3.11" and python_version < "3.12" coverage==7.3.2 ; python_version >= "3.11" and python_version < "3.12" coverage[toml]==7.3.2 ; python_version >= "3.11" and python_version < "3.12" crispy-bootstrap5==2023.10 ; python_version >= "3.11" and python_version < "3.12" cron-descriptor==1.4.0 ; python_version >= "3.11" and python_version < "3.12" cryptography==41.0.5 ; python_version >= "3.11" and python_version < "3.12" cssbeautifier==1.14.11 ; python_version >= "3.11" and python_version < "3.12" daphne==4.0.0 ; python_version >= "3.11" and python_version < "3.12" defusedxml==0.7.1 ; python_version >= "3.11" and python_version < "3.12" detect-secrets==1.4.0 ; python_version >= "3.11" and python_version < "3.12" distlib==0.3.7 ; python_version >= "3.11" and python_version < "3.12" django-allauth==0.58.2 ; python_version >= "3.11" and python_version < "3.12" django-appconf==1.0.6 ; python_version >= "3.11" and python_version < "3.12" django-celery-beat==2.5.0 ; python_version >= "3.11" and python_version < "3.12" django-compressor==4.4 ; python_version >= "3.11" and python_version < "3.12" django-cors-headers==4.3.1 ; python_version >= "3.11" and python_version < "3.12" django-coverage-plugin==3.1.0 ; python_version >= "3.11" and python_version < "3.12" django-crispy-forms==2.1 ; python_version >= "3.11" and python_version < "3.12" django-debug-toolbar==4.2.0 ; python_version >= "3.11" and python_version < "3.12" django-environ==0.11.2 ; python_version >= "3.11" and python_version < "3.12" django-extensions==3.2.3 ; python_version >= "3.11" and python_version < "3.12" django-filter==23.4 ; python_version >= "3.11" and python_version < "3.12" django-model-utils==4.3.1 ; python_version >= "3.11" and python_version < "3.12" django-modelcluster==6.1 ; python_version >= "3.11" and python_version < "3.12" django-permissionedforms==0.1 ; python_version >= "3.11" and python_version < "3.12" django-redis==5.4.0 ; python_version >= "3.11" and python_version < "3.12" django-stubs-ext==4.2.5 ; python_version >= "3.11" and python_version < "3.12" django-stubs[compatible-mypy]==4.2.6 ; python_version >= "3.11" and python_version < "3.12" django-taggit==4.0.0 ; python_version >= "3.11" and python_version < "3.12" django-timezone-field==6.1.0 ; python_version >= "3.11" and python_version < "3.12" django-treebeard==4.7 ; python_version >= "3.11" and python_version < "3.12" django-upgrade==1.15.0 ; python_version >= "3.11" and python_version < "3.12" django==4.2.7 ; python_version >= "3.11" and python_version < "3.12" djangorestframework==3.14.0 ; python_version >= "3.11" and python_version < "3.12" djlint==1.34.0 ; python_version >= "3.11" and python_version < "3.12" docutils==0.20.1 ; python_version >= "3.11" and python_version < "3.12" draftjs-exporter==2.1.7 ; python_version >= "3.11" and python_version < "3.12" drf-spectacular==0.26.5 ; python_version >= "3.11" and python_version < "3.12" editorconfig==0.12.3 ; python_version >= "3.11" and python_version < "3.12" et-xmlfile==1.1.0 ; python_version >= "3.11" and python_version < "3.12" execnet==2.0.2 ; python_version >= "3.11" and python_version < "3.12" factory-boy==3.3.0 ; python_version >= "3.11" and python_version < "3.12" faker==20.1.0 ; python_version >= "3.11" and python_version < "3.12" filelock==3.13.1 ; python_version >= "3.11" and python_version < "3.12" filetype==1.2.0 ; python_version >= "3.11" and python_version < "3.12" flask-basicauth==0.2.0 ; python_version >= "3.11" and python_version < "3.12" flask-cors==4.0.0 ; python_version >= "3.11" and python_version < "3.12" flask==3.0.0 ; python_version >= "3.11" and python_version < "3.12" flower==2.0.1 ; python_version >= "3.11" and python_version < "3.12" furo==2023.9.10 ; python_version >= "3.11" and python_version < "3.12" gevent==23.9.1 ; python_version >= "3.11" and python_version < "3.12" geventhttpclient==2.0.11 ; python_version >= "3.11" and python_version < "3.12" greenlet==3.0.1 ; platform_python_implementation == "CPython" and python_version >= "3.11" and python_version < "3.12" h11==0.14.0 ; python_version >= "3.11" and python_version < "3.12" hiredis==2.2.3 ; python_version >= "3.11" and python_version < "3.12" html-tag-names==0.1.2 ; python_version >= "3.11" and python_version < "3.12" html-void-elements==0.1.0 ; python_version >= "3.11" and python_version < "3.12" html5lib==1.1 ; python_version >= "3.11" and python_version < "3.12" httptools==0.6.1 ; python_version >= "3.11" and python_version < "3.12" humanize==4.9.0 ; python_version >= "3.11" and python_version < "3.12" hyperlink==21.0.0 ; python_version >= "3.11" and python_version < "3.12" hypothesis==6.90.0 ; python_version >= "3.11" and python_version < "3.12" identify==2.5.32 ; python_version >= "3.11" and python_version < "3.12" idna==3.6 ; python_version >= "3.11" and python_version < "3.12" imagesize==1.4.1 ; python_version >= "3.11" and python_version < "3.12" incremental==22.10.0 ; python_version >= "3.11" and python_version < "3.12" inflection==0.5.1 ; python_version >= "3.11" and python_version < "3.12" iniconfig==2.0.0 ; python_version >= "3.11" and python_version < "3.12" itsdangerous==2.1.2 ; python_version >= "3.11" and python_version < "3.12" jinja2==3.1.2 ; python_version >= "3.11" and python_version < "3.12" jsbeautifier==1.14.11 ; python_version >= "3.11" and python_version < "3.12" json5==0.9.14 ; python_version >= "3.11" and python_version < "3.12" jsonschema-specifications==2023.11.1 ; python_version >= "3.11" and python_version < "3.12" jsonschema==4.20.0 ; python_version >= "3.11" and python_version < "3.12" kombu==5.3.4 ; python_version >= "3.11" and python_version < "3.12" l18n==2021.3 ; python_version >= "3.11" and python_version < "3.12" livereload==2.6.3 ; python_version >= "3.11" and python_version < "3.12" locust==2.19.0 ; python_version >= "3.11" and python_version < "3.12" markupsafe==2.1.3 ; python_version >= "3.11" and python_version < "3.12" msgpack==1.0.7 ; python_version >= "3.11" and python_version < "3.12" mypy-extensions==1.0.0 ; python_version >= "3.11" and python_version < "3.12" mypy==1.6.1 ; python_version >= "3.11" and python_version < "3.12" nodeenv==1.8.0 ; python_version >= "3.11" and python_version < "3.12" oauthlib==3.2.2 ; python_version >= "3.11" and python_version < "3.12" openpyxl==3.1.2 ; python_version >= "3.11" and python_version < "3.12" packaging==23.2 ; python_version >= "3.11" and python_version < "3.12" pathspec==0.11.2 ; python_version >= "3.11" and python_version < "3.12" pillow-heif==0.13.1 ; python_version >= "3.11" and python_version < "3.12" pillow==10.1.0 ; python_version >= "3.11" and python_version < "3.12" platformdirs==4.0.0 ; python_version >= "3.11" and python_version < "3.12" pluggy==1.3.0 ; python_version >= "3.11" and python_version < "3.12" pre-commit==3.5.0 ; python_version >= "3.11" and python_version < "3.12" prettier==0.0.7 ; python_version >= "3.11" and python_version < "3.12" prometheus-client==0.19.0 ; python_version >= "3.11" and python_version < "3.12" prompt-toolkit==3.0.41 ; python_version >= "3.11" and python_version < "3.12" psutil==5.9.6 ; python_version >= "3.11" and python_version < "3.12" psycopg-c==3.1.13 ; python_version >= "3.11" and python_version < "3.12" psycopg[c]==3.1.13 ; python_version >= "3.11" and python_version < "3.12" pyasn1-modules==0.3.0 ; python_version >= "3.11" and python_version < "3.12" pyasn1==0.5.1 ; python_version >= "3.11" and python_version < "3.12" pycparser==2.21 ; python_version >= "3.11" and python_version < "3.12" pygments==2.17.2 ; python_version >= "3.11" and python_version < "3.12" pyjwt[crypto]==2.8.0 ; python_version >= "3.11" and python_version < "3.12" pyopenssl==23.3.0 ; python_version >= "3.11" and python_version < "3.12" pyproject-api==1.6.1 ; python_version >= "3.11" and python_version < "3.12" pytest-cov==4.1.0 ; python_version >= "3.11" and python_version < "3.12" pytest-django==4.7.0 ; python_version >= "3.11" and python_version < "3.12" pytest-factoryboy==2.6.0 ; python_version >= "3.11" and python_version < "3.12" pytest-mock==3.12.0 ; python_version >= "3.11" and python_version < "3.12" pytest-xdist==3.5.0 ; python_version >= "3.11" and python_version < "3.12" pytest==7.4.3 ; python_version >= "3.11" and python_version < "3.12" python-crontab==3.0.0 ; python_version >= "3.11" and python_version < "3.12" python-dateutil==2.8.2 ; python_version >= "3.11" and python_version < "3.12" python-dotenv==1.0.0 ; python_version >= "3.11" and python_version < "3.12" python-slugify==8.0.1 ; python_version >= "3.11" and python_version < "3.12" python3-openid==3.2.0 ; python_version >= "3.11" and python_version < "3.12" pytz==2023.3.post1 ; python_version >= "3.11" and python_version < "3.12" pywin32==306 ; python_version >= "3.11" and python_version < "3.12" and platform_system == "Windows" pyyaml==6.0.1 ; python_version >= "3.11" and python_version < "3.12" pyzmq==25.1.1 ; python_version >= "3.11" and python_version < "3.12" rcssmin==1.1.1 ; python_version >= "3.11" and python_version < "3.12" redis==5.0.1 ; python_version >= "3.11" and python_version < "3.12" referencing==0.31.0 ; python_version >= "3.11" and python_version < "3.12" regex==2023.10.3 ; python_version >= "3.11" and python_version < "3.12" requests-oauthlib==1.3.1 ; python_version >= "3.11" and python_version < "3.12" requests==2.31.0 ; python_version >= "3.11" and python_version < "3.12" rjsmin==1.2.1 ; python_version >= "3.11" and python_version < "3.12" roundrobin==0.0.4 ; python_version >= "3.11" and python_version < "3.12" rpds-py==0.13.1 ; python_version >= "3.11" and python_version < "3.12" ruff==0.1.6 ; python_version >= "3.11" and python_version < "3.12" sentry-sdk==1.37.1 ; python_version >= "3.11" and python_version < "3.12" service-identity==23.1.0 ; python_version >= "3.11" and python_version < "3.12" setuptools==69.0.2 ; python_version >= "3.11" and python_version < "3.12" six==1.16.0 ; python_version >= "3.11" and python_version < "3.12" sniffio==1.3.0 ; python_version >= "3.11" and python_version < "3.12" snowballstemmer==2.2.0 ; python_version >= "3.11" and python_version < "3.12" sortedcontainers==2.4.0 ; python_version >= "3.11" and python_version < "3.12" soupsieve==2.5 ; python_version >= "3.11" and python_version < "3.12" sphinx-autobuild==2021.3.14 ; python_version >= "3.11" and python_version < "3.12" sphinx-basic-ng==1.0.0b2 ; python_version >= "3.11" and python_version < "3.12" sphinx==7.2.6 ; python_version >= "3.11" and python_version < "3.12" sphinxcontrib-applehelp==1.0.7 ; python_version >= "3.11" and python_version < "3.12" sphinxcontrib-devhelp==1.0.5 ; python_version >= "3.11" and python_version < "3.12" sphinxcontrib-htmlhelp==2.0.4 ; python_version >= "3.11" and python_version < "3.12" sphinxcontrib-jsmath==1.0.1 ; python_version >= "3.11" and python_version < "3.12" sphinxcontrib-qthelp==1.0.6 ; python_version >= "3.11" and python_version < "3.12" sphinxcontrib-serializinghtml==1.1.9 ; python_version >= "3.11" and python_version < "3.12" sqlparse==0.4.4 ; python_version >= "3.11" and python_version < "3.12" telepath==0.3.1 ; python_version >= "3.11" and python_version < "3.12" text-unidecode==1.3 ; python_version >= "3.11" and python_version < "3.12" tokenize-rt==5.2.0 ; python_version >= "3.11" and python_version < "3.12" tornado==6.3.3 ; python_version >= "3.11" and python_version < "3.12" tox==4.11.3 ; python_version >= "3.11" and python_version < "3.12" tqdm==4.66.1 ; python_version >= "3.11" and python_version < "3.12" twisted-iocpsupport==1.0.4 ; python_version >= "3.11" and python_version < "3.12" and platform_system == "Windows" twisted[tls]==23.10.0 ; python_version >= "3.11" and python_version < "3.12" txaio==23.1.1 ; python_version >= "3.11" and python_version < "3.12" types-pytz==2023.3.1.1 ; python_version >= "3.11" and python_version < "3.12" types-pyyaml==6.0.12.12 ; python_version >= "3.11" and python_version < "3.12" typing-extensions==4.8.0 ; python_version >= "3.11" and python_version < "3.12" tzdata==2023.3 ; python_version >= "3.11" and python_version < "3.12" uritemplate==4.1.1 ; python_version >= "3.11" and python_version < "3.12" urllib3==2.1.0 ; python_version >= "3.11" and python_version < "3.12" uvicorn[standard]==0.24.0.post1 ; python_version >= "3.11" and python_version < "3.12" uvloop==0.19.0 ; (sys_platform != "win32" and sys_platform != "cygwin") and platform_python_implementation != "PyPy" and python_version >= "3.11" and python_version < "3.12" vine==5.1.0 ; python_version >= "3.11" and python_version < "3.12" virtualenv==20.24.7 ; python_version >= "3.11" and python_version < "3.12" wagtail==5.2.1 ; python_version >= "3.11" and python_version < "3.12" watchfiles==0.21.0 ; python_version >= "3.11" and python_version < "3.12" wcwidth==0.2.12 ; python_version >= "3.11" and python_version < "3.12" webencodings==0.5.1 ; python_version >= "3.11" and python_version < "3.12" websockets==12.0 ; python_version >= "3.11" and python_version < "3.12" werkzeug==3.0.1 ; python_version >= "3.11" and python_version < "3.12" whitenoise==6.6.0 ; python_version >= "3.11" and python_version < "3.12" willow[heif]==1.6.2 ; python_version >= "3.11" and python_version < "3.12" zope-event==5.0 ; python_version >= "3.11" and python_version < "3.12" zope-interface==6.1 ; python_version >= "3.11" and python_version < "3.12" ```

Error & Traceback

Found 3 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).

F..
======================================================================
FAIL: test_base_tag (wishbone.data.tests.test_models.TestBaseTag.test_base_tag)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/app/wishbone/data/tests/test_models.py", line 17, in test_base_tag
    def test_base_tag(self, instance: DeviceTag) -> None:
               ^^^^^^^
  File "/home/dev-user/.cache/pypoetry/virtualenvs/wishbone-9TtSrW0h-py3.11/lib/python3.11/site-packages/hypothesis/core.py", line 1469, in wrapped_test
    raise the_error_hypothesis_found
  File "/app/wishbone/data/tests/test_models.py", line 20, in test_base_tag
    self.assertIsInstance(instance.name, str)
AssertionError: 'New Tag' is not an instance of <class 'str'>
Falsifying example: test_base_tag(
    self=<wishbone.data.tests.test_models.TestBaseTag testMethod=test_base_tag>,
    instance=<DeviceTag: DeviceTag object (2)>,
)

Test Case

from hypothesis import given
from hypothesis.extra.django import TestCase, from_model

from wishbone.data.models import DeviceTag

# Ideally, Django would provide us a way of testing the BaseTag model
# directly--or at least in a way that doesn't require a lot of manual,
# flaky configuration, but that has yet to be solved:
# https://code.djangoproject.com/ticket/7835
# So instead, we'll use the DeviceTag model -- hypothesis will still
# generate data for each derived BaseTag model for the fields they
# share, but ¯\_(ツ)_/¯

class TestBaseTag(TestCase):
    @given(from_model(DeviceTag))
    def test_base_tag(self, instance: DeviceTag) -> None:
        self.assertTrue(len(instance.name) > 0)
        print(type(instance.name))  # <class 'django.utils.functional.lazy.<locals>.__proxy__'> (then) <class 'django.utils.functional.lazy.<locals>.__proxy__'> (again)
        self.assertIsInstance(instance.name, str)  # <--- Fails here
        self.assertIsInstance(instance.enabled, bool)  # <--- But this if fine if the above line is commented

Models

class BaseTag(models.Model):
    """An abstract model containing attributes shared across all tags.

    Cannot be used directly and should be subclassed into a concrete
    model.
    """

    # Basic
    name = models.CharField(
        _("Name"),
        max_length=64,
        default=_("New Tag"),
        validators=(validators.MinLengthValidator(limit_value=1),),
    )
    enabled = models.BooleanField(_("Enabled"), default=True)
    # TODO: TagGroup

    class Meta(object):
        abstract = True

class DeviceTag(BaseTag):
    pass

[tybug]

Apologies for the delayed response here. Presumably you are using from django.utils.translation import gettext_lazy as _, in which case this is not a bug: the default value of DeviceTag is _("New Tag") which is indeed a functional lazy wrapper as you saw printed. You can force evaluation with str(instance.name) to get the non-lazy version.

[dakotahorstman]

Ah, I see.

Very good. I appreciate it. Interesting it happens on the BooleanField, but not the CharField. Perhaps the name parameter is treated differently by Django for some reason.

[tybug]

well, Hypothesis is using the default= parameter here to pick a value for the field, not the name parameter. default is _("New Tag") in name but True in enabled. That is why instance.name is a lazy wrapper while instance.enabled is a boolean.