It's high time for a new release of Async RedisCache. This new version will bring a number of new features to this package and add a deprecation warning to the usage of NamespaceLock.
New features
Atomic Compound Operations
All compound operations consisting of multiple Redis commands should now be atomic from a Redis perspective due to the usage of a Lua Redis script that adds the missing functionality.
RedisQueue Task Iteration
You can now iterate over a RedisQueue with async for, optionally using its iter_tasks with wait to wait for new tasks to become available.
New type: RedisTaskQueue
A new datatype was added to the package: RedisTaskQueue. This queue is similar to a RedisQeue except that is uses task tracking: Consumed tasks are temporarily moved to a "pending" queue to make sure data is not lost if a client exits unexpectedly. Once a client is done with processing a task, it can finalize the task to remove it from the "pending" queue.
To support multiple, simultaneous clients, clients can set a "client ID" during initalization to get their own, individual pending queue. In general, clients should reschedule all the tasks in their "pending" queue when they start up to deal with tasks that were never marked as done during their previous run. However, do make sure that a client is able to handle partially processed tasks: If a client exited after processing (some of) the task, but before marking it done, rescheduling the "pending" tasks will make it appear again.
Set an expiry by duration or timestamp
You can now set an expiry on an entire namespace by either an expiration duration or a timestamp (seconds since unix epoch or a datetime.datetime). Please note that setting an expire on a subkey in a RedisCache or an item in a queue is not supported, as Redis only supports setting an expire on the outer key/namespace.
Deprecated Feature: NamespaceLock
The namespace lock was introduced to deal with non-atomic, compound operations. However, a lock can only guarantee atomicity within one client; it does not actually lock values in the Redis database. That's why I've opted to make the operations themselves atomic from a Redis point of view. This means the NamespaceLock is no longer needed, which also means that there's less of a chance for accidental deadlocks.
I've added a deprecation warning and the lock will be removed in version 1.0.0.
Note: If you're dealing with a single client application and like to make your own methods mutually exclusive/pseudo-atomic, consider using the RedisObject.atomic_transaction decorator. It's available on all redis types and all async functions decorated with this decorator will have to acquire the lock (i.e., will run sequentially).
It's high time for a new release of Async RedisCache. This new version will bring a number of new features to this package and add a deprecation warning to the usage of
NamespaceLock.New features
Atomic Compound Operations
All compound operations consisting of multiple Redis commands should now be atomic from a Redis perspective due to the usage of a Lua Redis script that adds the missing functionality.
RedisQueue Task Iteration
You can now iterate over a
RedisQueuewithasync for, optionally using itsiter_taskswithwaitto wait for new tasks to become available.New type: RedisTaskQueue
A new datatype was added to the package:
RedisTaskQueue. This queue is similar to aRedisQeueexcept that is uses task tracking: Consumed tasks are temporarily moved to a "pending" queue to make sure data is not lost if a client exits unexpectedly. Once a client is done with processing a task, it can finalize the task to remove it from the "pending" queue.To support multiple, simultaneous clients, clients can set a "client ID" during initalization to get their own, individual pending queue. In general, clients should reschedule all the tasks in their "pending" queue when they start up to deal with tasks that were never marked as done during their previous run. However, do make sure that a client is able to handle partially processed tasks: If a client exited after processing (some of) the task, but before marking it done, rescheduling the "pending" tasks will make it appear again.
Set an expiry by duration or timestamp
You can now set an expiry on an entire namespace by either an expiration duration or a timestamp (seconds since unix epoch or a datetime.datetime). Please note that setting an expire on a subkey in a RedisCache or an item in a queue is not supported, as Redis only supports setting an expire on the outer key/namespace.
Deprecated Feature: NamespaceLock
The namespace lock was introduced to deal with non-atomic, compound operations. However, a lock can only guarantee atomicity within one client; it does not actually lock values in the Redis database. That's why I've opted to make the operations themselves atomic from a Redis point of view. This means the NamespaceLock is no longer needed, which also means that there's less of a chance for accidental deadlocks.
I've added a deprecation warning and the lock will be removed in version 1.0.0.
Note: If you're dealing with a single client application and like to make your own methods mutually exclusive/pseudo-atomic, consider using the
RedisObject.atomic_transactiondecorator. It's available on all redis types and all async functions decorated with this decorator will have to acquire the lock (i.e., will run sequentially).