delete_item ReturnValuesOnConditionCheckFailure does not return Attributes upon failure.

[kvr000]

Describe the bug

The delete_item is supposed to return Attributes field containing original content of item when ReturnValuesOnConditionCheckFailure is set to ALL_OLD and condition fails, or in general, when ReturnValues is set to ALL_OLD .

This is not the case though. The successful operation returns the original content when ReturnValues==ALL_OLD, however upon condition failure, there is no such fields.

This seems to be already reported in https://github.com/aws/aws-sdk-js-v3/issues/5045 for javascript version. I suspect it's more a service error than the client errors and therefore affects all clients.

Expected Behavior

response.get('Attributes') should be in the body when condition fails and ReturnValuesOnConditionCheckFailure is set to ALL_OLD .

Current Behavior

response.get('Attributes') is not present in the response.

Reproduction Steps

#!/usr/bin/env python3

# Test deleting entry for no-entry, condition-failed and successful-delete:

import logging
import boto3
import botocore.errorfactory

boto3.set_stream_logger('')

dynamodb = boto3.client("dynamodb")
dynamodbResource = boto3.resource("dynamodb")

def main():
    table = dynamodbResource.Table("zbynek_aws_exp_delete_conditional")

    #try:
    #    dynamodb.delete_table(TableName=table.table_name)
    #except Exception as ex:
    #    logging.error(ex)
    #    pass
    #table.wait_until_not_exists()

    try:
        dynamodb.create_table(
            TableName=table.table_name,
            KeySchema=[
                {
                    "AttributeName": "id",
                    "KeyType": "HASH",
                },
                {
                    "AttributeName": "keycounter",
                    "KeyType": "RANGE",
                },
            ],
            AttributeDefinitions=[
                {
                    "AttributeName": "id",
                    "AttributeType": "S",
                },
                {
                    "AttributeName": "keycounter",
                    "AttributeType": "N",
                },
            ],
            BillingMode="PAY_PER_REQUEST",
        )
    except dynamodb.exceptions.ResourceInUseException as ex:
        pass
    table.wait_until_exists()

    for keycounter in range(0, 2):
        table.put_item(
            Item={
                "id": "hello",
                "keycounter": keycounter,
            },
        )

    response = table.delete_item(
        Key={
            "id": "hello",
            "keycounter": 0,
        },
        ReturnValues='ALL_OLD',
    )
    assert response.get('Attributes') is not None

    response = table.delete_item(
        Key={
            "id": "hello",
            "keycounter": 2,
        },
        ReturnValues='ALL_OLD',
    )
    assert response.get('Attributes') is None

    try:
        response = table.delete_item(
            Key={
                "id": "hello",
                "keycounter": 1,
            },
            ConditionExpression="attribute_exists(not_existent)",
            ReturnValues='ALL_OLD',
            ReturnValuesOnConditionCheckFailure='ALL_OLD',
        )
        assert False
    except dynamodb.exceptions.ConditionalCheckFailedException as ex:
        item = ex.response.get('Item')
        assert item is not None
        assert item['id']['S'] == "hello"
        assert item['keycounter']['N'] == "1"

    try:
        response = table.delete_item(
            Key={
                "id": "hello",
                "keycounter": 2,
            },
            ReturnValues='ALL_OLD',
            ReturnValuesOnConditionCheckFailure='ALL_OLD',
            ConditionExpression="attribute_exists(not_existent)",
        )
    except dynamodb.exceptions.ConditionalCheckFailedException as ex:
        item = ex.response.get('Item')
        assert item is None

main()

Possible Solution

Return the Attributes field in the response.

Additional Information/Context

No response

SDK version used

1.28.82

Environment details (OS name and version, etc.)

MacOs

[tim-finnigan]

Hi @kvr000 thanks for reaching out. In the issue you linked, someone commented saying that the issue was due to an older Lambda runtime: https://github.com/aws/aws-sdk-js-v3/issues/5045#issuecomment-1685825723.

Here are the default runtimes along with the SDK version they come with. These get updated periodically by the Lambda team. If you the boto3/botocore version you're using in Lambda is missing functionality, then you can try adding a Lambda layer or runtime dependency with a more recent version — this documentation and article provide the steps for doing that. Here are the boto3 and botocore CHANGELOGs if you're looking for particular changes in a specific version.

The issue here may be that you're using the delete_item resource command rather than the client command. There is a feature freeze on boto3 resources, and you may see some functionality supported in client commands that isn't available for resources.

I hope that helps — please let us know if you have any follow up questions. And if you want to provide debug logs (with any sensitive info redacted) by adding boto3.set_stream_logger('') then we could review any issues in more detail. If there is a service API issue then we would need to escalate that to the DynamoDB team. (If you have a support plan we recommend reaching out directly through AWS support for API-related issues.)

[kvr000]

Thanks @tim-finnigan for the response and suggesting how to enable the log. It helped and I found the issue.

The code was checking Attributes field but the original attributes appear in Item field. The documentation mentions only Attributes but it's for different type of response when ReturnValues='ALL_OLD' is requested.

I think it's worth updating the documentation as it never mentions the Item attribute for any type of response: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb/table/delete_item.html .

I also update the request to making working with current response.

[tim-finnigan]

Thanks @kvr000 for following up and for your patience here. That page you referenced does list the following for ReturnValues:

Do you have specific feedback on how the wording could be improved here or elsewhere on that page? Please let us know, and we can continue tracking this as a request to improve the documentation.

[kvr000]

Thank @tim-finnigan . This part of documentation is only for successful update. In this case, I want to see original value if the update fails and that is missing in the documentation. It requires ReturnValuesOnConditionCheckFailure='ALL_OLD' in the request but the fact that the value is returned in Item field is not documented anywhere.

See this piece of code:

        response = table.delete_item(
            Key={ "id": "hello", "keycounter": 2, },
            ReturnValues='ALL_OLD',
            ReturnValuesOnConditionCheckFailure='ALL_OLD',  # NOTE: request old value
            ConditionExpression="attribute_exists(not_existent)",
        )
    except dynamodb.exceptions.ConditionalCheckFailedException as ex:
        item = ex.response.get('Item') # NOTE: get the old value, NOT in documentation
        assert item is None

[tim-finnigan]

Thanks for your patience here, since the DynamoDB team owns the upstream DeleteItem API this would need to be addressed by their documentation team. And any updates would get reflected in the Boto3 (and other SDK) docs. We generally recommend using the Provide feedback links to send feedback directly the appropriate team. We can also reach out to service teams on your behalf. If you'd like us to do that please create an issue here in our cross-SDK repo for tracking: https://github.com/aws/aws-sdk/issues.

[github-actions[bot]]

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.