youtype / mypy_boto3_builder

Type annotations builder for boto3 compatible with VSCode, PyCharm, Emacs, Sublime Text, pyright and mypy.
https://youtype.github.io/mypy_boto3_builder/
MIT License
544 stars 36 forks source link

Regression boto3-stubs[dynamodb] Table.get_item typing #210

Closed fizyk closed 1 year ago

fizyk commented 1 year ago

Describe the bug In between boto3/boto3-stubs 1.28.9 and 1.28.15 and botocore/botocore-stubs 1.31.9 to 1.31.15 there was a change to Table.get_item typing of GetItemOutputTypeDef which now states that key "Item" is Dict[str, AttributeValueTypeDef]. While in reality it's closer to what put_item is typed as:

The AttributeValueTypeDef type requires additional layer of mapping to pass typing check which simple is not there in the resulting data.

Item: Mapping[
            str,
            Union[
                bytes,
                bytearray,
                str,
                int,
                Decimal,
                bool,
                Set[int],
                Set[Decimal],
                Set[str],
                Set[bytes],
                Set[bytearray],
                Sequence[Any],
                Mapping[str, Any],
                None,
            ],
        ],

To Reproduce Steps to reproduce the behavior:

  1. Install boto3-stubs[...]
  2. Run mypy/pyright on the following code sample
import boto3

def test_dynamodb() -> None:
    """Simple test for DynamoDB.

    # Create a table
    # Put an item
    # Get the item and check the content of this item
    """

    dynamodb = boto3.resource("dynamodb")

    # create a table
    table = dynamodb.create_table(
        TableName="Test",
        KeySchema=[{"AttributeName": "id", "KeyType": "HASH"}],
        AttributeDefinitions=[{"AttributeName": "id", "AttributeType": "S"}],
        ProvisionedThroughput={
            "ReadCapacityUnits": 10,
            "WriteCapacityUnits": 10,
        },
    )

    _id = str(uuid.uuid4())

    # put an item into db
    table.put_item(
        Item={"id": _id, "test_key": "test_value"},
    )

    # get the item
    item: GetItemOutputTypeDef = table.get_item(
        Key={
            "id": _id,
        }
    )    # check the content of the item
    assert item["Item"]["test_key"] == "test_value"

Actual output

❯ pipenv run mypy test.py                                                                                                                                                                                                          ─╯
test.py:40: error: Non-overlapping equality check (left operand type: "AttributeValueTypeDef", right operand type: "Literal['test_value']")  [comparison-overlap]
        assert item["Item"]["test_key"] == "test_value"
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expected output

Success.

Additional context Your OS, boto3-stubs installation method, boto3 version, etc.

❯ pipenv requirements                                                                                                                                                                                                              ─╯
-i https://pypi.org/simple
boto3==1.28.15
boto3-stubs[dynamodb]==1.28.15
botocore==1.31.15 ; python_version >= '3.7'
botocore-stubs==1.31.15 ; python_version >= '3.7' and python_version < '4.0'
iniconfig==2.0.0 ; python_version >= '3.7'
jmespath==1.0.1 ; python_version >= '3.7'
mirakuru==2.5.1
mypy-boto3-dynamodb==1.28.15.post2
packaging==23.1 ; python_version >= '3.7'
pluggy==1.2.0 ; python_version >= '3.7'
port-for==0.7.1
psutil==5.9.5 ; sys_platform != 'cygwin'
pytest==7.4.0
python-dateutil==2.8.2 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'
s3transfer==0.6.1 ; python_version >= '3.7'
six==1.16.0 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'
types-awscrt==0.17.0 ; python_version >= '3.7' and python_version < '4.0'
types-s3transfer==0.6.1 ; python_version >= '3.7' and python_version < '4.0'
urllib3==1.26.16 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'

Plus dev packages:

❯ pipenv requirements --dev-only                                                                                                                                                                                                   ─╯
-i https://pypi.org/simple
black==23.3.0
cli-ui==0.17.2 ; python_version >= '3.7' and python_version < '4.0'
click==8.1.6 ; python_version >= '3.7'
click-default-group==1.2.2
colorama==0.4.6 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5, 3.6'
contextlib2==21.6.0 ; python_version >= '3.6'
coverage[toml]==7.2.7 ; python_version >= '3.7'
docopt==0.6.2
execnet==2.0.2 ; python_version >= '3.7'
incremental==22.10.0
iniconfig==2.0.0 ; python_version >= '3.7'
jinja2==3.1.2 ; python_version >= '3.7'
markupsafe==2.1.3 ; python_version >= '3.7'
mypy==1.4.1
mypy-extensions==1.0.0 ; python_version >= '3.5'
packaging==23.1 ; python_version >= '3.7'
pathspec==0.11.2 ; python_version >= '3.7'
platformdirs==3.10.0 ; python_version >= '3.7'
pluggy==1.2.0 ; python_version >= '3.7'
pytest==7.4.0
pytest-cov==4.1.0
pytest-xdist==3.3.1
ruff==0.0.280
schema==0.7.5
tabulate==0.8.10 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'
tbump==6.10.0
tomlkit==0.11.8 ; python_version >= '3.7'
towncrier==23.6.0
typing-extensions==4.7.1 ; python_version >= '3.7'
unidecode==1.3.6 ; python_version >= '3.5'

I'm using pipenv: https://github.com/ClearcodeHQ/pytest-dynamodb/blob/dependabot/pip/boto3-1.28.15/Pipfile https://github.com/ClearcodeHQ/pytest-dynamodb/blob/dependabot/pip/boto3-1.28.15/Pipfile.lock

Full diff of updates that arrived: https://github.com/ClearcodeHQ/pytest-dynamodb/pull/1208/files

vemel commented 1 year ago

Thank you for the report!

Will be fixed today.

vemel commented 1 year ago

Fixed in mypy-boto3-s3 1.28.16. Please update and let me know if it works as it should for you.

fizyk commented 1 year ago

Thank you! That works :)

vemel commented 1 year ago

Thanks for testing the new version so quickly! Please let me know if you encounter any other issues, or have any feature requests.