Context-Local Thunder Protection Task Storage

[robert-schmidtke]

Fixes #221.

I see there's a perf/ directory, any idea how I could compare old vs. new performance?

[robert-schmidtke]

I tried this with the Redis backend, and unfortunately it seems that Redis is not able to run in multiple loops.

import asyncio
from threading import Thread

from cashews import cache

# cache.setup("mem://")
cache.setup("redis://localhost:6379/0?client_side=False")

@cache(ttl="2s", key="foo")
async def foo(arg: str):
    print(arg)

async def amain():
    def start_event_loop(loop):
        asyncio.set_event_loop(loop)
        loop.run_forever()

    loop = asyncio.new_event_loop()
    thread = Thread(target=start_event_loop, args=(loop,), daemon=True)
    thread.start()

    future = asyncio.run_coroutine_threadsafe(foo("threadsafe"), loop)
    await foo("mainthread")
    future.result()

if __name__ == "__main__":
    asyncio.run(amain())

# prints mainthread
# prints threadsafe
Traceback (most recent call last):
  File "/home/rschmidtke/workspace/scratch/asynciotest.py", line 30, in <module>
    asyncio.run(amain())
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/home/rschmidtke/workspace/scratch/asynciotest.py", line 25, in amain
    await foo("mainthread")
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/wrapper/decorators.py", line 69, in _call
    return await thunder_protection(decorator)(*args, **kwargs)
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/decorators/locked.py", line 107, in _wrapper
    return await task
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/decorators/cache/simple.py", line 65, in _wrap
    await backend.set(_cache_key, result, expire=_ttl, tags=_tags)
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/wrapper/tags.py", line 117, in set
    _set = await super().set(key=key, value=value, expire=expire, exist=exist)
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/wrapper/commands.py", line 24, in set
    return await self._with_middlewares(Command.SET, key)(
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/wrapper/disable_control.py", line 20, in _is_disable_middleware
    return await call(*args, **kwargs)
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/wrapper/callback.py", line 20, in __call__
    result = await call(*args, **kwargs)
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/validation.py", line 69, in _invalidate_middleware
    return await call(*args, **kwargs)
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/wrapper/auto_init.py", line 15, in _auto_init
    return await call(*args, **kwargs)
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/serialize.py", line 90, in set
    return await super().set(key, value, expire=expire, exist=exist)  # type: ignore[misc]
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/backends/redis/backend.py", line 124, in set
    _set = bool(await self._client.set(key, value, px=px, nx=nx, xx=xx))
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/backends/redis/client.py", line 31, in execute_command
    return await super().execute_command(command, *args, **kwargs)
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/redis/asyncio/client.py", line 612, in execute_command
    return await conn.retry.call_with_retry(
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/redis/asyncio/retry.py", line 59, in call_with_retry
    return await do()
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/redis/asyncio/client.py", line 586, in _send_command_parse_response
    return await self.parse_response(conn, command_name, **options)
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/redis/asyncio/client.py", line 633, in parse_response
    response = await connection.read_response()
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/redis/asyncio/connection.py", line 533, in read_response
    response = await self._parser.read_response(
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/redis/_parsers/resp2.py", line 82, in read_response
    response = await self._read_response(disable_decoding=disable_decoding)
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/redis/_parsers/resp2.py", line 90, in _read_response
    raw = await self._readline()
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/redis/_parsers/base.py", line 219, in _readline
    data = await self._stream.readline()
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/asyncio/streams.py", line 524, in readline
    line = await self.readuntil(sep)
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/asyncio/streams.py", line 616, in readuntil
    await self._wait_for_data('readuntil')
  File "/home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/asyncio/streams.py", line 501, in _wait_for_data
    await self._waiter
RuntimeError: Task <Task pending name='Task-2' coro=<foo() running at /home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/decorators/cache/simple.py:65> cb=[thunder_protection.<locals>._decor.<locals>.done_callback('foo')() at /home/rschmidtke/miniforge3/envs/testenv/lib/python3.10/site-packages/cashews/decorators/locked.py:94, Task.task_wakeup()]> got Future <Future pending> attached to a different loop

[Krukov]

Hello, about performance checks - for now projects don't have any tools/tests/checks for it and /perf dir is just some artefacts that I used for but I do not recommend to use it )) . Better way is just create own script to measure

Regarding PR , as I wrote in the issue - FMPOV it will be better to solve it inside an application internally. Or may be I am wrong lets discuss it )

[robert-schmidtke]

Yeah I think can be solved inside the application. The fix in this PR works for memory backend but not Redis, so not generally applicable.

Thanks for having a look!