boto / boto3

AWS SDK for Python
https://aws.amazon.com/sdk-for-python/
Apache License 2.0
8.87k stars 1.85k forks source link

ParamValidationError when using DynamoDB paginator #2300

Open usegev opened 4 years ago

usegev commented 4 years ago

I create a DynamoDB paginator like this:

    paginator = client.get_paginator('query')
    page_iterator = paginator.paginate(
        TableName=TABLE_NAME,
        IndexName=INDEX_NAME,
        KeyConditionExpression=Key('X').eq(x) & Key('Y').lte(y)
    )
    for page in page_iterator:
        print (page['Items'])
    return

When I run it, I receive the following error:

Exception has occurred: ParamValidationError
Parameter validation failed:
Invalid type for parameter KeyConditionExpression, value: <boto3.dynamodb.conditions.And object at 0x111524110>, type: <class 'boto3.dynamodb.conditions.And'>, valid types: <class 'str'>

The exact same KeyConditionExpression works when I query the table directly:

    result = table.query(
        IndexName=INDEX_NAME,
        KeyConditionExpression=Key('X').eq(x) & Key('Y').lte(y)
    )

BTW: Documentation says that KeyConditionExpression should be a string and not some condition built this way.

swetashre commented 4 years ago

@usegev - Thank you for your post. I am able to reproduce the issue. Looking into it.

swetashre commented 4 years ago

@usegev - We have a customization around resources that converts KeyConditionExpression type to string format that's why you are not getting error when query the table directly.

You can use the same format with paginator by using resource.meta.client. Something like this:

import boto3
res = boto3.resource('dynamodb')

paginator = res.meta.client.get_paginator('query')
page_iterator = paginator.paginate(
        TableName=TABLE_NAME,
        IndexName=INDEX_NAME,
        KeyConditionExpression=Key('X').eq(x) & Key('Y').lte(y)
    )

Hope it helps and please let me know if you have any questions.

usegev commented 4 years ago

Thanks @swetashre, works as you described. Will the documentation be updated? Or will the client paginator support it as well?

swetashre commented 4 years ago

We are tracking this issue internally. We are working to document this behavior.

saintross commented 4 years ago

thanks @swetashre this was making me scratch my head

kulbida commented 4 years ago

I am having the same issue with KeyConditionExpression for query and paginator

dynamodb = boto3.client('dynamodb',  . . . )
items = dynamodb.query(TableName='test',
    KeyConditionExpression=Key('user_email').eq('test@example.com'))

using paginator:

paginator = dynamodb.get_paginator('query')
items = paginator.paginate(TableName='test',
    KeyConditionExpression=Key('user_email').eq('test@example.com'))

getting:

Invalid type for parameter KeyConditionExpression,
value: <boto3.dynamodb.conditions.Equals object at 0x7f286128e990>,
type: <class 'boto3.dynamodb.conditions.Equals'>, valid types: <class 'str'>
ChetanaYogeesh-zz commented 4 years ago

What is the status on this issue ?

alexsanderp commented 4 years ago

I'm having this same problem.

ianscottknight commented 3 years ago

Any updates on this? I'm getting the same issue as @kulbida

koren-at-fundbox commented 3 years ago

Any update?

sebas-nicholls commented 2 years ago

I am having the same issue on October 2021

aadidubey7 commented 2 years ago

response = client.query( TableName=table_name, IndexName="user_id", KeyConditionExpression="user_id = :user_id", ExpressionAttributeValues= { ":user_id": { "S": "XYZ" } } )

This works for me!