async support?

[mikeckennedy]

Hi, cool project. Since the underlying Moka project describes itself like:

Thread-safe, highly concurrent in-memory cache implementations:

• Synchronous caches that can be shared across OS threads.
• An asynchronous (futures aware) cache.

I was wondering if you're planning on adding an async/await version of the API.

[kireevys]

Hi, this will work for your async functions.

import asyncio
import time

from moka_py import cached

@cached(maxsize=1024, ttl=10.0, tti=1.0)
async def f(x, y):
    print("hard computations")
    await asyncio.sleep(5)
    return x + y

if __name__ == "__main__":
    start = time.time()
    print(asyncio.run(f(1, 2)))
    print(asyncio.run(f(1, 2)))
    print(time.time() - start)

# hard computations
# 3
# 3
# 5.002709865570068

[mikeckennedy]

Awesome, good to know.

Does it make sense for cache.get() and cache.set() to have async partners?

[deliro]

Hi!

Does it make sense for cache.get() and cache.set() to have async partners?

I doubt it. cache.get() takes ~230ns (nanoseconds) on average (4.42 million calls per second) and cache.set() takes ~740ns on average (1.35 million calls per second) for a cache of size 10000 that's fully occupied (on my mac m1). Therefore, adding async machinery would be unnecessary overhead.

In future, I plan to add an async partner for Moka.get_with which seems reasonable (since it will need to receive a Python async function and run it somewhere in Rust runtime). But for now, exporting async functions is experimental in pyo3 (this is why with_concurrent=True is not supported for async functions in moka_py.cached() decorator)

[mikeckennedy]

Thanks for the extra per info. Makes sense. I plan on covering this project on the next https://pythonbytes.fm/ so having a little more inside info is helpful.