SDK publishes exception segment regardless whether it's handled or not

[latacora-tomekr]

This was originally brought up in https://github.com/awslabs/aws-lambda-powertools-python/issues/334 but @heitorlessa mentioned this is could be an upstream issue here and asked me to file an Issue. The details can be found in the original issue as well but we have the following function in our lambda code:

def get_secrets(secret_name) -> dict:
    try:
        get_secret_value_response = client.get_secret_value(SecretId=secret_name)
    except ClientError as e:
        if e.response["Error"]["Code"] == "ResourceNotFoundException":
            if "AWSCURRENT" in e.response["Message"]:
                logger.warning(f"{secret_name} exists but is empty.")
            else:
                logger.warning(
                    f"The requested secret {secret_name} was not found. Create the secret first."
                )
            return {}
        else:
            raise RuntimeError("Couldn't retrieve secrets")
    else:
        return json.loads(get_secret_value_response["SecretString"])

And in our case we continue processing if the secret exists but a value hasn't been set. In this case we're catching and handling the exception, continuing execution. However, the SDK will still flag this as an exception in X-Ray which will show the service as erroring when it actually isn't.

The relevant line is at https://github.com/aws/aws-xray-sdk-python/blob/508f929fee495710656c0c1478f8873a2e9e5666/aws_xray_sdk/core/models/subsegment.py#L70-L78

Is there a way to handle this use case?

[heitorlessa]

Following up as I replied in the original issue created downstream in Lambda Powertools: https://github.com/awslabs/aws-lambda-powertools-python/issues/334#issuecomment-801814709

Bottom line here is that X-Ray is doing the right thing in notifying an Exception happened, and as the code continues it sets the Trace as non-faulty which is expected. If the challenge is the UI in knowing that an exception happened in X subsegment with boto only despite being handled correctly, then a feature flag could be an option to suppress it[1] as I think X-Ray is behaving as it should.

[1] https://github.com/aws/aws-xray-sdk-python/blob/508f929fee495710656c0c1478f8873a2e9e5666/aws_xray_sdk/ext/boto_utils.py#L73

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs in next 7 days. Thank you for your contributions.

[guywilsonjr]

This is still an issue. I just ran into it. I wish it was only unhandled exceptions. When using custom subsegments it becomes an issue

[wangzlei]

Hi, the X-Ray just records what happened in AWS SDK, add the exception if there is. As long as user covers the exception it won't propagate to upstream segment. We understand some cases user don't like the error/fault status if the exception has been handled, but unfortunately it is happened later than the exception recorded in subsegment. So, a workaround might be user manually create a subsegment wraps the aws sdk subsegment. In this manual subsegment customer can add some info to mark the exception has been handled.

[sevetseh28]

We're having the same issue with a Typer CLI App that does sys.exit(0) when finishing up. The SDK shows this as an exception: