aws-powertools / powertools-lambda-python

A developer toolkit to implement Serverless best practices and increase developer velocity.
https://docs.powertools.aws.dev/lambda/python/latest/
MIT No Attribution
2.73k stars 378 forks source link

Feature request: Idempotent decorator for asynchronous functions #4483

Open esthermcmahon opened 1 month ago

esthermcmahon commented 1 month ago

Use case

Would love to be able to use the AsyncBatchProcessor and async_process_partial_response function along with the idempotency utility.

Solution/User Experience

def async_idempotent_function():
    def wrapper(func):
        @functools.wraps(func)
        async def wrapped(*args):
            return await func(*args)
        return wrapped
    return wrapper

Alternative solutions

No response

Acknowledgment

boring-cyborg[bot] commented 1 month ago

Thanks for opening your first issue here! We'll come back to you as soon as we can. In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

heitorlessa commented 1 month ago

hey @esthermcmahon, thank you so much for taking the time to open this feature request. Are you using DynamoDB or Redis as a persistence storage for Idempotency?

The reason we haven't added support for async in idempotency yet is that the AWS SDK is synchronous. This means it'll be blocking and you wouldn't fully benefit from async I/O - e.g., calls to store/retrieve data from Amazon DynamoDB / Redis would block the event loop for a few milliseconds.


for maintainers and anyone looking to help implementing it :)

Off the top of my head, a new async decorator along with these two changes could make it possible (needs to be tested):

PS: Adding help wanted to also signal we need to track customer demand. Personally, this is gonna become a common need in Data Engineering and large data pipelines.