Test failure in test_threads_ttl_expiry due to updated freezegun

dotlambda commented 5 years ago

I'm getting the following test failure on Python 3.7:

============================= test session starts ==============================
platform linux -- Python 3.7.1, pytest-3.9.3, py-1.7.0, pluggy-0.8.0
rootdir: /build/source, inifile:
collected 30 items

tests/test_async_cached_property.py ...                                  [ 10%]
tests/test_cached_property.py ...............F........                   [ 90%]
tests/test_coroutine_cached_property.py ...                              [100%]

=================================== FAILURES ===================================
______________ TestCachedPropertyWithTTL.test_threads_ttl_expiry _______________

self = <tests.test_cached_property.TestCachedPropertyWithTTL testMethod=test_threads_ttl_expiry>

    def test_threads_ttl_expiry(self):
        Check = CheckFactory(self.cached_property_factory(ttl=100000), threadsafe=True)
        check = Check()
        num_threads = 5

        # Same as in test_threads
        check.run_threads(num_threads)
        self.assert_cached(check, num_threads)
        self.assert_cached(check, num_threads)

        # The cache expires in the future
        with freeze_time("9999-01-01"):
            check.run_threads(num_threads)
>           self.assert_cached(check, 2 * num_threads)

tests/test_cached_property.py:207:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_cached_property.py:69: in assert_cached
    self.assertEqual(check.add_cached, expected)
E   AssertionError: 6 != 10
===================== 1 failed, 29 passed in 8.39 seconds ======================

Do you have any idea why this could happen?

dotlambda commented 5 years ago

Seems like this is related to having freezegun-0.3.11: https://github.com/pydanny/cached-property/pull/125.

althonos commented 5 years ago

@dotlambda : here are my findings after investigation:

the new freezegun API has an internal check to see if the real time should be used when calling time.time within a freezed context, using the _should_use_real_time method
one of these checks involve comparing the current frame against a list of ignored modules that can be given with the ignore keyword argument when calling freeze_time
the threading module is always added to the ignore list

subsequently, when the cached property is accessed within a thread, the time.time call (which is at the time delegated to a freezegun.api.FakeTime instance) thinks it should use the real time, and returns the real time, causing the property to think the cache is not outdated, and failing the test.

I thinks the most sensible thing is to open an issue on spulec/freezegun and request the ignore list to be populated with defaults only when it is None, so that the cached-property tests could remove threading from the ignore list and have calls within threads use the frozen time as well.

althonos commented 5 years ago

Otherwise, as a fudge, it's possible to patch the FakeTime instance no to recurse through the call stack, making it unable to detect it is called from a Thread, and preventing it from returning the real time:

with freeze_time("9999-01-01"):
    time.time.call_stack_inspection_limit = 0
    check.run_threads(num_threads)
    self.assert_cached(check, 2 * num_threads)
    self.assert_cached(check, 2 * num_threads)

AdamWill commented 5 years ago

Note for the record, if I'm following the maze of somewhat confusing issues related to this correctly, https://github.com/spulec/freezegun/commit/028dee229f06d200d0f79a130deaad65b14779ef is intended to fix it...See:

AdamWill commented 5 years ago

Well, nope, that doesn't fix it apparently. Sigh.

AdamWill commented 5 years ago

The fudge suggested by @althonos doesn't seem to help for me either :/

shadchin commented 4 years ago

For me work next patch

        # The cache expires in the future
        import freezegun.api
        orig_value = freezegun.api.call_stack_inspection_limit
        freezegun.api.call_stack_inspection_limit = 0
        with freeze_time("9999-01-01"):
            check.run_threads(num_threads)
            self.assert_cached(check, 2 * num_threads)
            self.assert_cached(check, 2 * num_threads)
        freezegun.api.call_stack_inspection_limit = orig_value

mbehrle commented 4 years ago

For me work next patch

        # The cache expires in the future
        import freezegun.api
        orig_value = freezegun.api.call_stack_inspection_limit
        freezegun.api.call_stack_inspection_limit = 0
        with freeze_time("9999-01-01"):
            check.run_threads(num_threads)
            self.assert_cached(check, 2 * num_threads)
            self.assert_cached(check, 2 * num_threads)
        freezegun.api.call_stack_inspection_limit = orig_value

JFTR: Tested and didn't work for me:

FAIL: test_threads_ttl_expiry (tests.test_cached_property.TestCachedPropertyWithTTL)
----------------------------------------------------------------------
Traceback (most recent call last):                         
  File "/tmp/autopkgtest.UlW7E1/autopkgtest_tmp/tests/test_cached_property.py", line 207, in test_threads_ttl_expiry  
    self.assert_cached(check, 2 * num_threads)                                                                        
  File "/tmp/autopkgtest.UlW7E1/autopkgtest_tmp/tests/test_cached_property.py", line 69, in assert_cached             
    self.assertEqual(check.add_cached, expected)                                                                      
AssertionError: 6 != 10                                                                                               

======================================================================
FAIL: test_threads_ttl_expiry (tests.test_cached_property.TestThreadedCachedPropertyWithTTL)
----------------------------------------------------------------------                                                
Traceback (most recent call last):                         
  File "/tmp/autopkgtest.UlW7E1/autopkgtest_tmp/tests/test_cached_property.py", line 238, in test_threads_ttl_expiry  
    self.assert_cached(check, 2 * num_threads)             
  File "/tmp/autopkgtest.UlW7E1/autopkgtest_tmp/tests/test_cached_property.py", line 69, in assert_cached             
    self.assertEqual(check.add_cached, expected)           
AssertionError: 2 != 10

Both issues

spulec/freezegun#269
ktosiek/pytest-freezegun#6

are closed and I was testing with recent freezegun version 0.3.15.

I have also tested the patch by @althonos with no success.

Is there any way forward to get the test suite running completely again?

FedericoCeratto commented 4 years ago

pydanny / cached-property

Test failure in test_threads_ttl_expiry due to updated freezegun #131