search

grantjenks / python-diskcache

Python disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python.
http://www.grantjenks.com/docs/diskcache/
Other
2.36k stars 133 forks source link

5.3.0: pytest is failing #244

Closed kloczek closed 2 years ago

kloczek commented 2 years ago

I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-diskcache-5.3.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-diskcache-5.3.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
/usr/lib/python3.8/site-packages/pytest_benchmark/logger.py:46: PytestBenchmarkWarning: Benchmarks are automatically disabled because xdist plugin is active.Benchmarks cannot be performed reliably in a parallelized environment.
  warner(PytestBenchmarkWarning(text))
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/tkloczko/rpmbuild/BUILD/python-diskcache-5.3.0, configfile: tox.ini
plugins: mock-3.6.1, cov-2.12.1, anyio-3.3.4, flaky-3.7.0, console-scripts-1.2.0, asyncio-0.16.0, freezegun-0.4.2, flake8-1.0.7, rerunfailures-9.1.1, yagot-0.5.0, forked-1.4.0, ordering-0.6, xdist-2.5.0, Faker-10.0.0, benchmark-3.4.1, pyfakefs-4.5.3, datadir-1.3.1, regressions-2.2.0, timeout-2.0.2, perf-0.10.1, trio-0.7.0, requests-mock-1.9.3, hypothesis-6.31.5, subtests-0.5.0, easy-server-0.8.0
gw0 [243] / gw1 [243] / gw2 [243] / gw3 [243] / gw4 [243] / gw5 [243]
............................................................................................................................................F...............F....... [ 67%]
...................................F...........................................                                                                                      [100%]
================================================================================= FAILURES =================================================================================
________________________________________________________________________ DiskCacheTests.test_pickle ________________________________________________________________________
[gw1] linux -- Python 3.8.12 /usr/bin/python3

self = <tests.test_djangocache.DiskCacheTests testMethod=test_pickle>

    def test_pickle(self):
        letters = 'abcde'
        cache.clear()

        for num, val in enumerate(letters):
            cache.set(val, num)

>       data = pickle.dumps(cache)
E       TypeError: cannot pickle '_thread.RLock' object

tests/test_djangocache.py:1036: TypeError
__________________________________________________________ DiskCacheTests.test_get_or_set_callable_returning_none __________________________________________________________
[gw3] linux -- Python 3.8.12 /usr/bin/python3

self = <tests.test_djangocache.DiskCacheTests testMethod=test_get_or_set_callable_returning_none>

    def test_get_or_set_callable_returning_none(self):
        self.assertIsNone(cache.get_or_set('mykey', lambda: None))
        # Previous get_or_set() doesn't store None in the cache.
>       self.assertEqual(cache.get('mykey', 'default'), 'default')
E       AssertionError: None != 'default'

tests/test_djangocache.py:884: AssertionError
_____________________________________________________ DiskCacheTests.test_cache_write_for_model_instance_with_deferred _____________________________________________________
[gw4] linux -- Python 3.8.12 /usr/bin/python3

self = <django.db.backends.utils.CursorDebugWrapper object at 0x7fdbe10ea550>, sql = 'DELETE FROM "tests_poll"', params = ()
ignored_wrapper_args = (False, {'connection': <django.db.backends.sqlite3.base.DatabaseWrapper object at 0x7fdbe1588580>, 'cursor': <django.db.backends.utils.CursorDebugWrapper object at 0x7fdbe10ea550>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
                # params default might be backend specific.
                return self.cursor.execute(sql)
            else:
>               return self.cursor.execute(sql, params)

/usr/lib/python3.8/site-packages/django/db/backends/utils.py:84:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <django.db.backends.sqlite3.base.SQLiteCursorWrapper object at 0x7fdbe134f5e0>, query = 'DELETE FROM "tests_poll"', params = ()

    def execute(self, query, params=None):
        if params is None:
            return Database.Cursor.execute(self, query)
        query = self.convert_query(query)
>       return Database.Cursor.execute(self, query, params)
E       sqlite3.OperationalError: database is locked

/usr/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py:423: OperationalError

The above exception was the direct cause of the following exception:

self = <tests.test_djangocache.DiskCacheTests testMethod=test_cache_write_for_model_instance_with_deferred>

    def test_cache_write_for_model_instance_with_deferred(self):
        # Don't want fields with callable as default to be called on cache write
        expensive_calculation.num_runs = 0
>       Poll.objects.all().delete()

tests/test_djangocache.py:262:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.8/site-packages/django/db/models/query.py:746: in delete
    deleted, _rows_count = collector.delete()
/usr/lib/python3.8/site-packages/django/db/models/deletion.py:410: in delete
    count = qs._raw_delete(using=self.using)
/usr/lib/python3.8/site-packages/django/db/models/query.py:762: in _raw_delete
    cursor = query.get_compiler(using).execute_sql(CURSOR)
/usr/lib/python3.8/site-packages/django/db/models/sql/compiler.py:1175: in execute_sql
    cursor.execute(sql, params)
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:98: in execute
    return super().execute(sql, params)
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:66: in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:75: in _execute_with_wrappers
    return executor(sql, params, many, context)
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:84: in _execute
    return self.cursor.execute(sql, params)
/usr/lib/python3.8/site-packages/django/db/utils.py:90: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:84: in _execute
    return self.cursor.execute(sql, params)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <django.db.backends.sqlite3.base.SQLiteCursorWrapper object at 0x7fdbe134f5e0>, query = 'DELETE FROM "tests_poll"', params = ()

    def execute(self, query, params=None):
        if params is None:
            return Database.Cursor.execute(self, query)
        query = self.convert_query(query)
>       return Database.Cursor.execute(self, query, params)
E       django.db.utils.OperationalError: database is locked

/usr/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py:423: OperationalError
============================================================================= warnings summary =============================================================================
tests/test_djangocache.py::DiskCacheTests::test_cache_write_unpicklable_object
  /usr/lib/python3.8/site-packages/django/middleware/cache.py:67: RemovedInDjango40Warning: Passing None for the middleware get_response argument is deprecated.
    super().__init__(get_response)

tests/test_djangocache.py::DiskCacheTests::test_cache_write_unpicklable_object
tests/test_djangocache.py::DiskCacheTests::test_cache_write_unpicklable_object
  /usr/lib/python3.8/site-packages/django/middleware/cache.py:130: RemovedInDjango40Warning: Passing None for the middleware get_response argument is deprecated.
    super().__init__(get_response)

-- Docs: https://docs.pytest.org/en/stable/warnings.html

---------- coverage: platform linux, python 3.8.12-final-0 -----------
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
diskcache/__init__.py         14      0      0      0   100%
diskcache/cli.py               0      0      0      0   100%
diskcache/core.py            916     10    367     15    98%   234->exit, 280->exit, 364->367, 371->373, 449->456, 670-673, 1190->1192, 1202->1204, 1325->1322, 1593->1590, 1707->1704, 1786, 1878->1881, 1946->1958, 1951->1958, 2438, 2445-2448
diskcache/djangocache.py      99      0     12      1    99%   436->446
diskcache/fanout.py          217      2     36      0    99%   490, 573
diskcache/persistent.py      282      0     80      3    99%   33->36, 147->155, 1070->exit
diskcache/recipes.py         168      1     26      1    99%   294
----------------------------------------------------------------------
TOTAL                       1696     13    521     20    98%

Required test coverage of 98% reached. Total coverage: 98.33%
========================================================================= short test summary info ==========================================================================
FAILED tests/test_djangocache.py::DiskCacheTests::test_pickle - TypeError: cannot pickle '_thread.RLock' object
FAILED tests/test_djangocache.py::DiskCacheTests::test_get_or_set_callable_returning_none - AssertionError: None != 'default'
FAILED tests/test_djangocache.py::DiskCacheTests::test_cache_write_for_model_instance_with_deferred - django.db.utils.OperationalError: database is locked
================================================================ 3 failed, 240 passed, 3 warnings in 27.68s ================================================================
kloczek commented 2 years ago

Just found that even with those tree units deselected some random units are failing. Two examples:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-diskcache-5.3.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-diskcache-5.3.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --deselect tests/test_djangocache.py::DiskCacheTests::test_cache_write_for_model_instance_with_deferred --deselect tests/test_djangocache.py::DiskCacheTests::test_get_or_set_callable_returning_none --deselect tests/test_djangocache.py::DiskCacheTests::test_pickle
/usr/lib/python3.8/site-packages/pytest_benchmark/logger.py:46: PytestBenchmarkWarning: Benchmarks are automatically disabled because xdist plugin is active.Benchmarks cannot be performed reliably in a parallelized environment.
  warner(PytestBenchmarkWarning(text))
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=1000809158
rootdir: /home/tkloczko/rpmbuild/BUILD/python-diskcache-5.3.0, configfile: tox.ini
plugins: mock-3.6.1, cov-2.12.1, anyio-3.3.4, flaky-3.7.0, console-scripts-1.2.0, asyncio-0.16.0, freezegun-0.4.2, flake8-1.0.7, rerunfailures-9.1.1, yagot-0.5.0, forked-1.4.0, ordering-0.6, xdist-2.5.0, Faker-10.0.0, benchmark-3.4.1, pyfakefs-4.5.3, datadir-1.3.1, regressions-2.2.0, timeout-2.0.2, randomly-3.10.3, perf-0.10.1, trio-0.7.0, requests-mock-1.9.3, hypothesis-6.31.5, subtests-0.5.0, easy-server-0.8.0
gw0 [240] / gw1 [240] / gw2 [240] / gw3 [240] / gw4 [240] / gw5 [240]
.................................................................................................................................................................... [ 68%]
..........................................................F.................                                                                                         [100%]
================================================================================= FAILURES =================================================================================
____________________________________________________________ DiskCacheTests.test_cache_read_for_model_instance _____________________________________________________________
[gw5] linux -- Python 3.8.12 /usr/bin/python3

self = <django.db.backends.utils.CursorDebugWrapper object at 0x7fc1c26cf250>, sql = 'DELETE FROM "tests_poll"', params = ()
ignored_wrapper_args = (False, {'connection': <django.db.backends.sqlite3.base.DatabaseWrapper object at 0x7fc1c2e6fe50>, 'cursor': <django.db.backends.utils.CursorDebugWrapper object at 0x7fc1c26cf250>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
                # params default might be backend specific.
                return self.cursor.execute(sql)
            else:
>               return self.cursor.execute(sql, params)

/usr/lib/python3.8/site-packages/django/db/backends/utils.py:84:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <django.db.backends.sqlite3.base.SQLiteCursorWrapper object at 0x7fc1c29653a0>, query = 'DELETE FROM "tests_poll"', params = ()

    def execute(self, query, params=None):
        if params is None:
            return Database.Cursor.execute(self, query)
        query = self.convert_query(query)
>       return Database.Cursor.execute(self, query, params)
E       sqlite3.OperationalError: database is locked

/usr/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py:423: OperationalError

The above exception was the direct cause of the following exception:

self = <tests.test_djangocache.DiskCacheTests testMethod=test_cache_read_for_model_instance>

    def test_cache_read_for_model_instance(self):
        # Don't want fields with callable as default to be called on cache read
        expensive_calculation.num_runs = 0
>       Poll.objects.all().delete()

tests/test_djangocache.py:249:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.8/site-packages/django/db/models/query.py:746: in delete
    deleted, _rows_count = collector.delete()
/usr/lib/python3.8/site-packages/django/db/models/deletion.py:410: in delete
    count = qs._raw_delete(using=self.using)
/usr/lib/python3.8/site-packages/django/db/models/query.py:762: in _raw_delete
    cursor = query.get_compiler(using).execute_sql(CURSOR)
/usr/lib/python3.8/site-packages/django/db/models/sql/compiler.py:1175: in execute_sql
    cursor.execute(sql, params)
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:98: in execute
    return super().execute(sql, params)
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:66: in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:75: in _execute_with_wrappers
    return executor(sql, params, many, context)
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:84: in _execute
    return self.cursor.execute(sql, params)
/usr/lib/python3.8/site-packages/django/db/utils.py:90: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:84: in _execute
    return self.cursor.execute(sql, params)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <django.db.backends.sqlite3.base.SQLiteCursorWrapper object at 0x7fc1c29653a0>, query = 'DELETE FROM "tests_poll"', params = ()

    def execute(self, query, params=None):
        if params is None:
            return Database.Cursor.execute(self, query)
        query = self.convert_query(query)
>       return Database.Cursor.execute(self, query, params)
E       django.db.utils.OperationalError: database is locked

/usr/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py:423: OperationalError
============================================================================= warnings summary =============================================================================
tests/test_djangocache.py::DiskCacheTests::test_cache_write_unpicklable_object
  /usr/lib/python3.8/site-packages/django/middleware/cache.py:67: RemovedInDjango40Warning: Passing None for the middleware get_response argument is deprecated.
    super().__init__(get_response)

tests/test_djangocache.py::DiskCacheTests::test_cache_write_unpicklable_object
tests/test_djangocache.py::DiskCacheTests::test_cache_write_unpicklable_object
  /usr/lib/python3.8/site-packages/django/middleware/cache.py:130: RemovedInDjango40Warning: Passing None for the middleware get_response argument is deprecated.
    super().__init__(get_response)

-- Docs: https://docs.pytest.org/en/stable/warnings.html

---------- coverage: platform linux, python 3.8.12-final-0 -----------
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
diskcache/__init__.py         14      0      0      0   100%
diskcache/cli.py               0      0      0      0   100%
diskcache/core.py            916     10    367     15    98%   234->exit, 280->exit, 364->367, 371->373, 449->456, 670-673, 1190->1192, 1202->1204, 1325->1322, 1593->1590, 1707->1704, 1786, 1878->1881, 1946->1958, 1951->1958, 2438, 2445-2448
diskcache/djangocache.py      99      0     12      1    99%   436->446
diskcache/fanout.py          217      2     36      0    99%   490, 573
diskcache/persistent.py      282      0     80      3    99%   33->36, 147->155, 1070->exit
diskcache/recipes.py         168      1     26      1    99%   294
----------------------------------------------------------------------
TOTAL                       1696     13    521     20    98%

Required test coverage of 98% reached. Total coverage: 98.33%
========================================================================= short test summary info ==========================================================================
FAILED tests/test_djangocache.py::DiskCacheTests::test_cache_read_for_model_instance - django.db.utils.OperationalError: database is locked
================================================================ 1 failed, 239 passed, 3 warnings in 28.72s ================================================================
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-diskcache-5.3.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-diskcache-5.3.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --deselect tests/test_djangocache.py::DiskCacheTests::test_cache_write_for_model_instance_with_deferred --deselect tests/test_djangocache.py::DiskCacheTests::test_get_or_set_callable_returning_none --deselect tests/test_djangocache.py::DiskCacheTests::test_pickle
/usr/lib/python3.8/site-packages/pytest_benchmark/logger.py:46: PytestBenchmarkWarning: Benchmarks are automatically disabled because xdist plugin is active.Benchmarks cannot be performed reliably in a parallelized environment.
  warner(PytestBenchmarkWarning(text))
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=1365837811
rootdir: /home/tkloczko/rpmbuild/BUILD/python-diskcache-5.3.0, configfile: tox.ini
plugins: mock-3.6.1, cov-2.12.1, anyio-3.3.4, flaky-3.7.0, console-scripts-1.2.0, asyncio-0.16.0, freezegun-0.4.2, flake8-1.0.7, rerunfailures-9.1.1, yagot-0.5.0, forked-1.4.0, ordering-0.6, xdist-2.5.0, Faker-10.0.0, benchmark-3.4.1, pyfakefs-4.5.3, datadir-1.3.1, regressions-2.2.0, timeout-2.0.2, randomly-3.10.3, perf-0.10.1, trio-0.7.0, requests-mock-1.9.3, hypothesis-6.31.5, subtests-0.5.0, easy-server-0.8.0
gw0 [240] / gw1 [240] / gw2 [240] / gw3 [240] / gw4 [240] / gw5 [240]
.................................................................................................................................................................... [ 68%]
............................F...............................................                                                                                         [100%]
================================================================================= FAILURES =================================================================================
_____________________________________________________ DiskCacheTests.test_cache_read_for_model_instance_with_deferred ______________________________________________________
[gw1] linux -- Python 3.8.12 /usr/bin/python3

self = <django.db.backends.utils.CursorDebugWrapper object at 0x7f52f348fd00>, sql = 'DELETE FROM "tests_poll"', params = ()
ignored_wrapper_args = (False, {'connection': <django.db.backends.sqlite3.base.DatabaseWrapper object at 0x7f52f39e5e20>, 'cursor': <django.db.backends.utils.CursorDebugWrapper object at 0x7f52f348fd00>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
                # params default might be backend specific.
                return self.cursor.execute(sql)
            else:
>               return self.cursor.execute(sql, params)

/usr/lib/python3.8/site-packages/django/db/backends/utils.py:84:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <django.db.backends.sqlite3.base.SQLiteCursorWrapper object at 0x7f52f34de3a0>, query = 'DELETE FROM "tests_poll"', params = ()

    def execute(self, query, params=None):
        if params is None:
            return Database.Cursor.execute(self, query)
        query = self.convert_query(query)
>       return Database.Cursor.execute(self, query, params)
E       sqlite3.OperationalError: database is locked

/usr/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py:423: OperationalError

The above exception was the direct cause of the following exception:

self = <tests.test_djangocache.DiskCacheTests testMethod=test_cache_read_for_model_instance_with_deferred>

    def test_cache_read_for_model_instance_with_deferred(self):
        # Don't want fields with callable as default to be called on cache read
        expensive_calculation.num_runs = 0
>       Poll.objects.all().delete()

tests/test_djangocache.py:275:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.8/site-packages/django/db/models/query.py:746: in delete
    deleted, _rows_count = collector.delete()
/usr/lib/python3.8/site-packages/django/db/models/deletion.py:410: in delete
    count = qs._raw_delete(using=self.using)
/usr/lib/python3.8/site-packages/django/db/models/query.py:762: in _raw_delete
    cursor = query.get_compiler(using).execute_sql(CURSOR)
/usr/lib/python3.8/site-packages/django/db/models/sql/compiler.py:1175: in execute_sql
    cursor.execute(sql, params)
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:98: in execute
    return super().execute(sql, params)
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:66: in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:75: in _execute_with_wrappers
    return executor(sql, params, many, context)
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:84: in _execute
    return self.cursor.execute(sql, params)
/usr/lib/python3.8/site-packages/django/db/utils.py:90: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
/usr/lib/python3.8/site-packages/django/db/backends/utils.py:84: in _execute
    return self.cursor.execute(sql, params)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <django.db.backends.sqlite3.base.SQLiteCursorWrapper object at 0x7f52f34de3a0>, query = 'DELETE FROM "tests_poll"', params = ()

    def execute(self, query, params=None):
        if params is None:
            return Database.Cursor.execute(self, query)
        query = self.convert_query(query)
>       return Database.Cursor.execute(self, query, params)
E       django.db.utils.OperationalError: database is locked

/usr/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py:423: OperationalError
============================================================================= warnings summary =============================================================================
tests/test_djangocache.py::DiskCacheTests::test_cache_write_unpicklable_object
  /usr/lib/python3.8/site-packages/django/middleware/cache.py:67: RemovedInDjango40Warning: Passing None for the middleware get_response argument is deprecated.
    super().__init__(get_response)

tests/test_djangocache.py::DiskCacheTests::test_cache_write_unpicklable_object
tests/test_djangocache.py::DiskCacheTests::test_cache_write_unpicklable_object
  /usr/lib/python3.8/site-packages/django/middleware/cache.py:130: RemovedInDjango40Warning: Passing None for the middleware get_response argument is deprecated.
    super().__init__(get_response)

-- Docs: https://docs.pytest.org/en/stable/warnings.html

---------- coverage: platform linux, python 3.8.12-final-0 -----------
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
diskcache/__init__.py         14      0      0      0   100%
diskcache/cli.py               0      0      0      0   100%
diskcache/core.py            916     10    367     15    98%   234->exit, 280->exit, 364->367, 371->373, 449->456, 670-673, 1190->1192, 1202->1204, 1325->1322, 1593->1590, 1707->1704, 1786, 1878->1881, 1946->1958, 1951->1958, 2438, 2445-2448
diskcache/djangocache.py      99      0     12      1    99%   436->446
diskcache/fanout.py          217      2     36      0    99%   490, 573
diskcache/persistent.py      282      0     80      3    99%   33->36, 147->155, 1070->exit
diskcache/recipes.py         168      1     26      1    99%   294
----------------------------------------------------------------------
TOTAL                       1696     13    521     20    98%

Required test coverage of 98% reached. Total coverage: 98.33%
========================================================================= short test summary info ==========================================================================
FAILED tests/test_djangocache.py::DiskCacheTests::test_cache_read_for_model_instance_with_deferred - django.db.utils.OperationalError: database is locked
================================================================ 1 failed, 239 passed, 3 warnings in 33.02s ================================================================

Looks like it is because I'm using pytest-randomly pytest extension. Looks like in the test suite are some dependencies between units. pytest-ransdonmly extension is causing ttaht pytet is execution test unit in random order. This may cause that pytest <file>.py::<test_uniot> may show false result. To diagnose that issue https://github.com/mrbean-bremen/pytest-find-dependencies/ can be used. As temporary workaround can be used pytest can be used.

kloczek commented 2 years ago

Just confirmed that as lonag as randomly ext is disabled pytest with deselectwed those three failing units is OK.

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-diskcache-5.3.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-diskcache-5.3.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -p no:randomly --deselect tests/test_djangocache.py::DiskCacheTests::test_cache_write_for_model_instance_with_deferred --deselect tests/test_djangocache.py::DiskCacheTests::test_get_or_set_callable_returning_none --deselect tests/test_djangocache.py::DiskCacheTests::test_pickle
/usr/lib/python3.8/site-packages/pytest_benchmark/logger.py:46: PytestBenchmarkWarning: Benchmarks are automatically disabled because xdist plugin is active.Benchmarks cannot be performed reliably in a parallelized environment.
  warner(PytestBenchmarkWarning(text))
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/tkloczko/rpmbuild/BUILD/python-diskcache-5.3.0, configfile: tox.ini
plugins: mock-3.6.1, cov-2.12.1, anyio-3.3.4, flaky-3.7.0, console-scripts-1.2.0, asyncio-0.16.0, freezegun-0.4.2, flake8-1.0.7, rerunfailures-9.1.1, yagot-0.5.0, forked-1.4.0, ordering-0.6, xdist-2.5.0, Faker-10.0.0, benchmark-3.4.1, pyfakefs-4.5.3, datadir-1.3.1, regressions-2.2.0, timeout-2.0.2, perf-0.10.1, trio-0.7.0, requests-mock-1.9.3, hypothesis-6.31.5, subtests-0.5.0, easy-server-0.8.0
gw0 [240] / gw1 [240] / gw2 [240] / gw3 [240] / gw4 [240] / gw5 [240]
.................................................................................................................................................................... [ 68%]
............................................................................                                                                                         [100%]
============================================================================= warnings summary =============================================================================
tests/test_djangocache.py::DiskCacheTests::test_cache_write_unpicklable_object
  /usr/lib/python3.8/site-packages/django/middleware/cache.py:67: RemovedInDjango40Warning: Passing None for the middleware get_response argument is deprecated.
    super().__init__(get_response)

tests/test_djangocache.py::DiskCacheTests::test_cache_write_unpicklable_object
tests/test_djangocache.py::DiskCacheTests::test_cache_write_unpicklable_object
  /usr/lib/python3.8/site-packages/django/middleware/cache.py:130: RemovedInDjango40Warning: Passing None for the middleware get_response argument is deprecated.
    super().__init__(get_response)

-- Docs: https://docs.pytest.org/en/stable/warnings.html

---------- coverage: platform linux, python 3.8.12-final-0 -----------
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
diskcache/__init__.py         14      0      0      0   100%
diskcache/cli.py               0      0      0      0   100%
diskcache/core.py            916     10    367     15    98%   234->exit, 280->exit, 364->367, 371->373, 449->456, 670-673, 1190->1192, 1202->1204, 1325->1322, 1593->1590, 1707->1704, 1786, 1878->1881, 1946->1958, 1951->1958, 2438, 2445-2448
diskcache/djangocache.py      99      0     12      1    99%   436->446
diskcache/fanout.py          217      2     36      0    99%   490, 573
diskcache/persistent.py      282      0     80      3    99%   33->36, 147->155, 1070->exit
diskcache/recipes.py         168      1     26      1    99%   294
----------------------------------------------------------------------
TOTAL                       1696     13    521     20    98%

Required test coverage of 98% reached. Total coverage: 98.33%
===================================================================== 240 passed, 3 warnings in 28.48s =====================================================================
grantjenks commented 2 years ago

Running pytest with pytest-randomly works fine for me:

$ pytest
================================== test session starts ==================================
platform darwin -- Python 3.9.1, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
Using --randomly-seed=2641636193
rootdir: /Users/grantjenks/repos/python-diskcache, configfile: tox.ini
plugins: cov-2.11.1, env-0.6.2, randomly-3.10.3, django-4.1.0, xdist-2.2.0, forked-1.3.0
gw0 [243] / gw1 [243] / gw2 [243] / gw3 [243]
................................................................................. [ 33%]
................................................................................. [ 66%]
................................................................................. [100%]
kloczek commented 2 years ago

Please try to run that test few time.