Suor / django-cacheops

A slick ORM cache with automatic granular event-driven invalidation.
BSD 3-Clause "New" or "Revised" License
2.12k stars 227 forks source link

Unusual behaviour when caching a queryset #404

Closed ymguerra closed 3 years ago

ymguerra commented 3 years ago

Hi, I am having an issue when caching a QuerySet, this is my code: my settings

CACHEOPS_REDIS = f'redis://{REDIS_SERVER_URL}/3'
CACHEOPS_DEFAULTS = {
    'timeout': 300 
}
CACHEOPS_DEGRADE_ON_FAILURE = True
CACHEOPS = {supplier.contact': {'ops': {'get', 'fetch'}}}

My code with "problems"

# This return a QuerySet (<QuerySet [1, 2, 3, 4, 5]>), I am not caching this, this came from a different table
linked_suppliers= outlet.outlet_supplier_connector.values_list('supplier', flat=True) 

# I am caching this
contacts = Contact.objects.filter(library_id=1)

# I am caching this, 
# I know I can do everything in one line and in one filter call but my code is 
# more complex than this and at the end I have this behaviour
suppliers = contacts.filter(id__in=linked_suppliers)

My problem here is that when the value of linked_suppliers change, lets say that change from <QuerySet [1, 2, 3, 4, 5]> to <QuerySet [1, 2, 3]> my suppliers returns the cache value and not the new value, however if I cast linked_suppliers to a list like this:
linked_suppliers= list(outlet.outlet_supplier_connector.values_list('supplier', flat=True)) then the code works. My feeling is that it doesn't detect that the new QuerySet is "new" or the cache key it generates is the same with the new parameters... I really don't understand what is happening, could you help me?

Btw CacheOps rocks, good work

Suor commented 3 years ago

Looks like a bug.

Can you write a test? The instruction is at the end of the README.

ymguerra commented 3 years ago

I created a test... but I think I am doing something wrong, because I am trying to use this override_settings(CACHEOPS = {'tests.post': {'ops': {'get', 'fetch'}},}) and I am not sure if I used it right, I don't want to use the .cache() since I am not doing it in my code, I will commit it so you can take a look

ymguerra commented 3 years ago

And also the test pass

Suor commented 3 years ago

In the test you change the queryset itself, do you change it in your code too? Or only the result if the queryset changes?

Suor commented 3 years ago

If the queryset is the same then this is expected, see CAVEATS, 7.

ymguerra commented 3 years ago

Oh, now I think I understand, so if the subquery is the same but the result of this subquery is different it doesn't trigger invalidation in the main query. I just check it on the test.

Thanks @Suor for your help, great work with cacheops, I think we can close this

Suor commented 3 years ago

As said in CAVEAT it might be fixed, it was never much motivation to do that.