Logging support in Boto Waiters

[syedahsn]

Describe the feature

It would be nice to have additional logging during the execution of the wait method when using boto Waiters. Currently, when a boto waiter is used, it does not provide any logging until a failure or success state is reached. It would be useful for users to have an additional flag (such as verbose etc.) that would make the waiter log the response from the service it is polling, which would allow users to see the status of a service during the waiting period.

Use Case

For the example of a EMR step, which can take several minutes to finish. the associated waiter does not display any of the intermediate states that the describe_step function returns. It would be nice to see the progression of the job go from PENDING to RUNNING to COMPLETED.

Proposed Solution

def wait(self, **kwargs):
    # existing code ...
    verbose = kwargs.get("verbose", False)

    while True:
        response = self._operation_method(**kwargs)
        num_attempts += 1
        for acceptor in acceptors:
            if acceptor.matcher_func(response):
                last_matched_acceptor = acceptor
                current_state = acceptor.state
                break
        else:
            if verbose:
                logger.debug("Polling response is %s", response) # response can potentially be cleaned up to give a better log message
            # existing code ...    

This might be an overly simplistic way, but something along these lines.

Other Information

No response

Acknowledgements

• [X] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

SDK version used

1.29.76

Environment details (OS name and version, etc.)

Ubuntu

[tim-finnigan]

Hi @syedahsn thanks for the feature request. I think your use case to log statuses during waiter execution makes sense. It is probably worth investigating if other AWS SDKs do this and if so, what the logging looks like. We can use this issue to track further discussion and possible approaches to implementation. Others can 👍 the issue if interested in this as well.

[o-nikolas]

I'd also like to see this feature! Not only to see status changes, but it'd be useful to monitor that things are still operating correctly. Code execution can appear to hang without any progress logging when using waiters as they are now.