No results returned when querying a date field

[aaroncraig10e]

Given a record with the following shape:

{'allocated_datetime': 1506648875.071361,
  ... snip ...
 'status': 'ALLOCATED'}

the following works with aws cli:

$ aws dynamodb scan --filter-expression='allocated_datetime BETWEEN :ago AND :now AND #status = :status' --expression-attribute-values='{":ago":{"N":"1506646742.820100"},":now":{"N":"1506646742.820110"},":status":{"S":"ALLOCATED"}}' --expression-attribute-names='{"#status":"status"}' --table-name='my-table' --select=COUNT --endpoint=http://localhost:8000
{
    "Count": 1,
    "ScannedCount": 92086,
    "ConsumedCapacity": null
}

but this does not work in a python script:

results = client.scan(
            TableName='my-table',
            Select='COUNT',
            ConsistentRead=True,
            FilterExpression='allocated_datetime BETWEEN :ago AND :now AND #status = :status',
            ExpressionAttributeNames={
                '#status': 'status'
            },
            ExpressionAttributeValues={
                ':ago': {
                    'N': '1506646742.820100'
                },
                ':now': {
                    'N': '1506646742.820110'
                },
                ':status': {
                    'S': 'ALLOCATED'
                }
            }
        )

pprint(results)

The results are:

{'Count': 0,
 'LastEvaluatedKey': {'hash_key': {'S': '8926a53fe484dd413ede6b45fb82159f76db60c49a1280cd96ccfe52'},
                      'range_key': {'S': '000080000111'}},
 'ResponseMetadata': {'HTTPHeaders': {'content-length': '161',
                                      'content-type': 'application/x-amz-json-1.0',
                                      'server': 'Jetty(8.1.12.v20130726)',
                                      'x-amz-crc32': '2370556166',
                                      'x-amzn-requestid': '09cb3dea-bd2f-4df1-a006-0828674a5f1e'},
                      'HTTPStatusCode': 200,
                      'RequestId': '09cb3dea-bd2f-4df1-a006-0828674a5f1e',
                      'RetryAttempts': 0},
 'ScannedCount': 1394}

Not sure why the cli scan would return the expected results, but the scan through flywheel does not.

[stevearc]

If you're seeing these errors when using the botocore client, then it might be better to post this issue on the botocore repo. If it's a problem in botocore, no amount of changes in flywheel will help.

I don't know what would be different between those two queries, but shot in the dark: the scanned count is different. If they're being run on the same dataset then something might be applying a scan limit somewhere. You can enable DEBUG level logging and botocore should print out all the requests as it's sending them over the wire.

[aaroncraig10e]

Cool, I'm new to this so wasn't sure which library to file this under.

I'll close this for now and raise it with boto.