Error creating GSI with global_keys_only

nick-zh commented 2 years ago

Summary

I get an error when i create a GSI for dynamodb with global_keys_only From what i understand i don't need to include any attributes, but somehow creation gives an error

Issue Type

Bug Report

Component Name

dynamodb_table

Ansible Version

$ ansible --version

ansible 2.10.8
  config file = ansible/ansible.cfg
  configured module search path = ['.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.9.10 (main, Feb 22 2022, 13:54:07) [GCC 11.2.0]

Collection Versions

$ ansible-galaxy collection list

Collection                Version
------------------------- -------
amazon.aws                1.4.0  
ansible.netcommon         1.5.0  
ansible.posix             1.1.1  
ansible.windows           1.4.0  
arista.eos                1.3.0  
awx.awx                   14.1.0 
azure.azcollection        1.4.0  
check_point.mgmt          1.0.6  
chocolatey.chocolatey     1.0.2  
cisco.aci                 1.1.1  
cisco.asa                 1.0.4  
cisco.intersight          1.0.10 
cisco.ios                 1.3.0  
cisco.iosxr               1.2.1  
cisco.meraki              2.2.0  
cisco.mso                 1.1.0  
cisco.nso                 1.0.3  
cisco.nxos                1.4.0  
cisco.ucs                 1.6.0  
cloudscale_ch.cloud       1.3.1  
community.aws             1.3.0  
community.azure           1.0.0  
community.crypto          1.4.0  
community.digitalocean    1.0.0  
community.docker          1.2.2  
community.fortios         1.0.0  
community.general         1.3.6  
community.google          1.0.0  
community.grafana         1.1.0  
community.hashi_vault     1.1.0  
community.hrobot          1.1.0  
community.kubernetes      1.1.1  
community.kubevirt        1.0.0  
community.libvirt         1.0.0  
community.mongodb         1.2.0  
community.mysql           1.2.0  
community.network         1.3.2  
community.okd             1.0.0  
community.postgresql      1.1.1  
community.proxysql        1.0.0  
community.rabbitmq        1.0.1  
community.routeros        1.1.0  
community.skydive         1.0.0  
community.vmware          1.7.0  
community.windows         1.3.0  
community.zabbix          1.2.0  
containers.podman         1.4.1  
cyberark.conjur           1.1.0  
cyberark.pas              1.0.5  
dellemc.os10              1.0.2  
dellemc.os6               1.0.6  
dellemc.os9               1.0.3  
f5networks.f5_modules     1.7.1  
fortinet.fortimanager     1.0.5  
fortinet.fortios          1.1.8  
frr.frr                   1.0.3  
gluster.gluster           1.0.1  
google.cloud              1.0.2  
hetzner.hcloud            1.2.1  
ibm.qradar                1.0.3  
infinidat.infinibox       1.2.4  
junipernetworks.junos     1.3.0  
mellanox.onyx             1.0.0  
netapp.aws                20.9.0 
netapp.elementsw          20.11.0
netapp.ontap              20.12.0
netapp_eseries.santricity 1.1.0  
netbox.netbox             1.2.1  
ngine_io.cloudstack       1.2.0  
ngine_io.exoscale         1.0.0  
ngine_io.vultr            1.1.0  
openstack.cloud           1.2.1  
openvswitch.openvswitch   1.1.0  
ovirt.ovirt               1.3.0  
purestorage.flasharray    1.6.2  
purestorage.flashblade    1.4.0  
servicenow.servicenow     1.0.4  
splunk.es                 1.0.2  
theforeman.foreman        1.5.1  
vyos.vyos                 1.1.1  
wti.remote                1.0.1  

# .ansible/collections/ansible_collections
Collection    Version
------------- -------
amazon.aws    3.0.0  
community.aws 2.1.0

AWS SDK versions

$ pip show boto boto3 botocore

Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: mitch@garnaat.com
License: MIT
Location: /usr/local/lib/python3.9/dist-packages
Requires: 
Required-by: 
---
Name: boto3
Version: 1.18.47
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: .local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.21.47
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: .local/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed

OS / Environment

No response

Steps to Reproduce

- name: Create dynamo table with hash and range primary key
  community.aws.dynamodb_table:
    name: MyTable
    region: eu-west-1
    hash_key_name: PrimaryId
    hash_key_type: STRING
    range_key_name: SecondaryId
    range_key_type: STRING
    billing_mode: PAY_PER_REQUEST
    indexes:
      - name: SecondaryIndex
        type: global_keys_only
        hash_key_name: SecondaryId

Expected Results

Table is created and i only need to add to add the non key attributes if use the modes global_include and include as stated here

Actual Results

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: Invalid length for parameter GlobalSecondaryIndexUpdates[0].Create.Projection.NonKeyAttributes, value: 0, valid min length: 1
fatal: [localhost]: FAILED! => {"boto3_version": "1.18.47", "botocore_version": "1.21.47", "changed": false, "msg": "Failed to update table: Parameter validation failed:\nInvalid length for parameter GlobalSecondaryIndexUpdates[0].Create.Projection.NonKeyAttributes, value: 0, valid min length: 1"}

Code of Conduct

[X] I agree to follow the Ansible Code of Conduct

ansibullbot commented 2 years ago

Files identified in the description:

[plugins/modules/dynamodb_table.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/dynamodb_table.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot commented 2 years ago

cc @jillr @loia @markuman @s-hertel @tremble click here for bot help

Sonic0 commented 2 years ago

I have the same issue with Ansible 5.8.0, ansible-core 2.12.5, collection community.aws 2.4 and Python 3.8.10

tremble commented 2 years ago

@Sonic0 Is there any chance you could create your intended DynamoDB table and provide the output of aws dynamodb describe-table --table-name <table_name>

I'm not particularly familiar with DynamoDB, but might be able to fix the issue if you're able to describe the end state.

nick-zh commented 2 years ago

@tremble i have done that on an example table (removed region and arn), i hope this is what you were looking for

{
    "Table": {
        "AttributeDefinitions": [
            {
                "AttributeName": "ProfileId",
                "AttributeType": "S"
            },
            {
                "AttributeName": "UserId",
                "AttributeType": "S"
            }
        ],
        "TableName": "TestTable",
        "KeySchema": [
            {
                "AttributeName": "UserId",
                "KeyType": "HASH"
            },
            {
                "AttributeName": "ProfileId",
                "KeyType": "RANGE"
            }
        ],
        "TableStatus": "ACTIVE",
        "CreationDateTime": "2022-05-25T16:40:34.413000+02:00",
        "ProvisionedThroughput": {
            "NumberOfDecreasesToday": 0,
            "ReadCapacityUnits": 5,
            "WriteCapacityUnits": 5
        },
        "TableSizeBytes": 0,
        "ItemCount": 0,
        "TableArn": "arn:aws:dynamodb:<redacted>:<redacted>:table/TestTable",
        "TableId": "<redacted>",
        "GlobalSecondaryIndexes": [
            {
                "IndexName": "ProfileId-index",
                "KeySchema": [
                    {
                        "AttributeName": "ProfileId",
                        "KeyType": "HASH"
                    }
                ],
                "Projection": {
                    "ProjectionType": "KEYS_ONLY"
                },
                "IndexStatus": "CREATING",
                "Backfilling": true,
                "ProvisionedThroughput": {
                    "NumberOfDecreasesToday": 0,
                    "ReadCapacityUnits": 1,
                    "WriteCapacityUnits": 1
                },
                "IndexSizeBytes": 0,
                "ItemCount": 0,
                "IndexArn": "arn:aws:dynamodb:<redacted>:<redacted>:table/TestTable/index/ProfileId-index"
            }
        ]
    }
}

tremble commented 2 years ago

@nick-zh Yup, that's what I needed.

nick-zh commented 2 years ago

Here are what the other index types look like, maybe that's helpful as well:

{
    "Table": {
        "AttributeDefinitions": [
            {
                "AttributeName": "ProfileId",
                "AttributeType": "S"
            },
            {
                "AttributeName": "UserId",
                "AttributeType": "S"
            }
        ],
        "TableName": "TestTable",
        "KeySchema": [
            {
                "AttributeName": "UserId",
                "KeyType": "HASH"
            },
            {
                "AttributeName": "ProfileId",
                "KeyType": "RANGE"
            }
        ],
        "TableStatus": "ACTIVE",
        "CreationDateTime": "2022-05-25T16:40:34.413000+02:00",
        "ProvisionedThroughput": {
            "LastDecreaseDateTime": "2022-05-25T16:53:38.830000+02:00",
            "NumberOfDecreasesToday": 2,
            "ReadCapacityUnits": 1,
            "WriteCapacityUnits": 1
        },
        "TableSizeBytes": 0,
        "ItemCount": 0,
        "TableArn": "arn:aws:dynamodb:<redacted>:<redacted>:table/TestTable",
        "TableId": "<redacted>",
        "GlobalSecondaryIndexes": [
            {
                "IndexName": "ProfileId-index",
                "KeySchema": [
                    {
                        "AttributeName": "ProfileId",
                        "KeyType": "HASH"
                    }
                ],
                "Projection": {
                    "ProjectionType": "ALL"
                },
                "IndexStatus": "ACTIVE",
                "ProvisionedThroughput": {
                    "NumberOfDecreasesToday": 0,
                    "ReadCapacityUnits": 1,
                    "WriteCapacityUnits": 1
                },
                "IndexSizeBytes": 0,
                "ItemCount": 0,
                "IndexArn": "arn:aws:dynamodb:<redacted>:<redacted>:table/TestTable/index/ProfileId-index"
            },
            {
                "IndexName": "ProfileId-index2",
                "KeySchema": [
                    {
                        "AttributeName": "ProfileId",
                        "KeyType": "HASH"
                    }
                ],
                "Projection": {
                    "ProjectionType": "INCLUDE",
                    "NonKeyAttributes": [
                        "UserId"
                    ]
                },
                "IndexStatus": "ACTIVE",
                "ProvisionedThroughput": {
                    "NumberOfDecreasesToday": 0,
                    "ReadCapacityUnits": 1,
                    "WriteCapacityUnits": 1
                },
                "IndexSizeBytes": 0,
                "ItemCount": 0,
                "IndexArn": "arn:aws:dynamodb:<redacted>:<redacted>:table/TestTable/index/ProfileId-index2"
            }
        ]
    }
}

tremble commented 2 years ago

@nick-zh Any chance you could try #1162 ?

nick-zh commented 2 years ago

I was not sure what's the best way to do this, but i adjusted the fix of plugins/modules/dynamodb_table.py to my local collection and i was able to successfully create a table with global_keys_only :tada: Many thanks @tremble :pray: (as always)

Sonic0 commented 2 years ago

I applied the fix to the file plugins/modules/dynamodb_table.py in my local collection and now it is working!!! Many thanks @tremble! :partying_face: :partying_face:

ansible-collections / community.aws