gargoyle uses modeldict for retrieving Switches. modeldict has two levels of caching: an in-memory cache based on key and the standard Django cache.
If a test sets a switch by specifying a fixture that stores the switch values directly in the database, then later tests that don't use the fixture might still get the changed value of the switch. If the Django cache is, say, an external memcache, then later test runs might also retrieve the cached value.
As far as I can tell, gargoyle has no way of clearing the caches nor of asking it to not use these caches in tests.
This is just what I've figured out from exploring. I'm no Django expert. If this isn't actually a bug, I would be glad to learn how to get proper test isolation. Am happy to answer questions.
gargoyle uses modeldict for retrieving Switches. modeldict has two levels of caching: an in-memory cache based on key and the standard Django cache.
If a test sets a switch by specifying a fixture that stores the switch values directly in the database, then later tests that don't use the fixture might still get the changed value of the switch. If the Django cache is, say, an external memcache, then later test runs might also retrieve the cached value.
As far as I can tell, gargoyle has no way of clearing the caches nor of asking it to not use these caches in tests.
https://code.launchpad.net/~jml/canonical-identity-provider/reliable-tests/+merge/136932 has some more details about the problem.
This is just what I've figured out from exploring. I'm no Django expert. If this isn't actually a bug, I would be glad to learn how to get proper test isolation. Am happy to answer questions.
Thanks, jml