Open holymode opened 2 months ago
From the code snippet that you shared it is not clear how and where you manage the cache
variable.
I advise you to start with How to debug small programs and create a minimal reproducible example (MRE) that another developer can copy and execute. Without it, it is usually too difficult to say what is wrong. The package creator has a broad background in the subject, but the creation and sharing of MRE from your side would be highly appreciated.
Just to add more context: This function is used in a loop, that runs every 2min and probably makes around 400-600 requests each time.
cache = SQLiteBackend(
cache_name='cache/aiohttp-requests.db',
expire_after=default_expire_after,
urls_expire_after=urls_expire_after,
allowed_methods=['GET', 'POST'],
include_headers=False
)
That's the cache setup.
I will try to rewrite one but not 100% sure if i can recreate the error.
I agree this is a somewhat tricky one to debug. I don't see any problems with the code snippet you provided, and can't reproduce this myself.
Somehow the SQLite connection object is being closed before (or at the same time as) a request is sent. If the connection is closed when a request is made, it is supposed to be reopened, so this sounds like a race condition. Is this loop potentially running from multiple threads?
One thing you could try is passing autoclose=False
to the cache object, and then manually call cache.close()
when you're done. Let me know if that helps.
The problem
Sometimes when I use my function to fetch data, it errors out and gives me
ProgrammingError: Cannot operate on a closed database.
My bot serves many people, I think this is just some weird race condition? I just added more caching to my bot to save on request and notice the error popping up more and more, before it was very rarely. Traceback:Expected behavior
Working without trowing any error.
Steps to reproduce the behavior
My function:
Environment