requests-cache / aiohttp-client-cache

An async persistent cache for aiohttp requests
MIT License
118 stars 20 forks source link

`AttributeError` when using sqlite backend #230

Closed cheginit closed 6 months ago

cheginit commented 6 months ago

The problem

In my package, I have two tests for checking caching. When I run the tests with pytest, I get this warning:

  /Users/tchegini/repos/hyriver/async-retriever/.nox/tests-3-9/lib/python3.9/site-packages/_pytest/unraisableexception.py:80: PytestUnraisableExceptionWarning: Exception ignored in: <function SQLiteCache.__del__ at 0x104f849d0>

  Traceback (most recent call last):
    File "/Users/tchegini/repos/hyriver/async-retriever/.nox/tests-3-9/lib/python3.9/site-packages/aiohttp_client_cache/backends/sqlite.py", line 116, in __del__
      self._connection._tx.queue.clear()
  AttributeError: '_queue.SimpleQueue' object has no attribute 'queue'

    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

As the traceback shows, this warning is suppressed and only shows up with pytest and not when running the code.

Expected behavior

I am not sure if this behavior is intentional or not.

Steps to reproduce the behavior

from pathlib import Path
import asyncio

from aiohttp_client_cache import SQLiteBackend

import async_retriever as ar

SMALL = 1e-3

async def check_url(url, method="GET", **kwargs):
    cache = SQLiteBackend(cache_name=Path("cache", "aiohttp_cache.sqlite"))
    return await cache.has_url(url, method, **kwargs)

def test_disable_cache():
    url = "https://epqs.nationalmap.gov/v1/json"
    payload = {"params": {"x": -101, "y": 38, "units": "Meters"}}
    resp = ar.retrieve([url], "json", [payload], disable=True)
    elev = float(resp[0]["value"])
    assert abs(elev - 880.418) < SMALL
    assert not asyncio.run(check_url(url, params=payload["params"]))

def test_delete_url():
    url = "https://epqs.nationalmap.gov/v1/json"
    payload = {"params": {"x": -100, "y": 38, "units": "Meters"}}
    resp = ar.retrieve([url], "json", [payload])
    elev = float(resp[0]["value"])
    ar.delete_url_cache(url, params=payload["params"])
    assert abs(elev - 761.684) < SMALL
    assert not asyncio.run(check_url(url, params=payload["params"]))

Workarounds

N/A

Environment

JWCook commented 6 months ago

It appears that aiosqlite 0.20 changed its queue implementation in https://github.com/omnilib/aiosqlite/pull/271.

On this end, I'm using some of aiosqlite's internals to force its connection thread to end in order to avoid hanging if the application exits without using the CachedSession contextmanager or calling .close() (see #173, #187, #189).

So that's not their fault. I'll get that fixed soon.

JWCook commented 6 months ago

@cheginit Test out the changes in main when you get the chance, and let me know if that fixes the issue for you.

cheginit commented 6 months ago

I just checked with main, and it runs without throwing the warning. Thank you.