Unable to query or set DynamoDB data

[KevinMarquette]

I was unable to find a way to add data to DynamoDB or get data from DynamoDB with any of the AWS PowerShell cmdlets. I was able to find Get-DDBTable and it knows how many items that it has, but I can't find a cmdlet for accessing those items.

Context

As customer, I was setting up a project that was using DynamoDB . I wanted to verify the data I posted made it into the table. Instead of turning to the web portal, I tried doing a quick query from PowerShell. Having never used powershell with DynamoDB yet, I was expecting basic functionality to already exist. The ability to query for or access the data is something that I expected to be available but I was unable to figure it out.

Current Behavior

After searching around for a quick solution, everything I found used chunks code working with dotnet objects. Because I was not automating anything at the moment that needed to talk to the table outside of my app, I logged into the console instead. Not having this feature created a poor customer experience.

Possible Solution

Basic\common operations that a user would want to do with DynamoDB should have simple commands in PowerShell. Something like this would have completely changed my opinion about the DynamoDB commands.

    Get-DDBTableItem -Name $key -Table tablename
    Set-DDBTableItem -Name $key -Value $value -Table tablename
    Remove-DDBTableItem -Name $key -Table tablename

[matteo-prosperi]

Hello, Thank you for reaching out to us. At the moment AWS Tools for PowerShell doesn't provide cmdlets to perform dataplane operation on DynamoDB.

The workaround for this would be to use directly the AmazonDynamoDBClient from the AWS SDK for .NET which is loaded whenever you import the AWS Tools for PowerShell modules.

We will consider this feature request for future implementation.

[timfitzzz]

Hi @matteo-prosperi -- I'm running into an issue with this workaround, namely that the AmazonDynamoDBClient from AWS SDK for .NET that is loaded when using either AWSPowerShell.NetCore or AWS.Tools.DynamoDBv2 doesn't seem to have the same API as the version that is loaded when using AWSPowershell. For instance, the DB client has no .Scan() method. Is this intentional? If so, is there some documentation of these changes to the API somewhere? Thanks!

[matteo-prosperi]

Hello, you are correct, AWS.Tools and AWSPowerShell.NetCore use the netstandard2.0 flavor of the AWS SDK for .NET (instead of the net35 flavor). The netstandard2.0 flavor only has the new asynchronous methods, so you will need to do this change: from

    $response = $amazonDynamoDBClient.Scan($request)

to

    $response = $amazonDynamoDBClient.ScanAsync($request).Result

or alternatively

    $response = $amazonDynamoDBClient.ScanAsync($request).GetAwaiter().GetResult()

[austoonz]

In case anyone wants additional help with this before data plane cmdlets exist, there are PowerShell code samples for accessing the DynamoDB data plane using the AWS SDK for .NET on the AWS .NET developer website.

[pcgeek86]

Any news on this? Seems horribly broken.

PS > $MyTable = [Amazon.DynamoDBv2.DocumentModel.Table]::LoadTable($DDBClient, 'trevordata')

OperationStopped: The type initializer for 'Amazon.DynamoDBv2.DynamoDBEntryConversion' threw an exception.

Types            : {Amazon.AWSConfigsDynamoDB, Amazon.Util.DynamoDBContextConfig, Amazon.Util.TableAlias, Amazon.Util.TypeMapping…}
LoaderExceptions : {System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, 
                   PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the file specified.
                   File name: 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51', System.IO.FileNotFoundException: Could not load 
                   file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the file specified.
                   File name: 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51', System.IO.FileNotFoundException: Could not load 
                   file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the file specified.
                   File name: 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51', System.IO.FileNotFoundException: Could not load 
                   file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the file specified.
                   File name: 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'…}
Message          : Unable to load one or more of the requested types.
                   Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the 
                   file specified.
                   Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the 
                   file specified.
                   Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the  
                   file specified.
                   Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the  
                   file specified.
                   Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the  
                   file specified.
                   Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the 
                   file specified.
                   Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the  
                   file specified.
                   Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the 
                   file specified.
TargetSite       : System.RuntimeType[] GetTypes(System.Reflection.RuntimeModule)
StackTrace       :    at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
                      at System.Reflection.RuntimeAssembly.get_DefinedTypes()
                      at Amazon.DynamoDBv2.DynamoDBEntryConversion.AddConverters(String suffix)
                      at Amazon.DynamoDBv2.DynamoDBEntryConversion.SetV1Converters()
                      at Amazon.DynamoDBv2.DynamoDBEntryConversion..ctor(ConversionSchema schema, Boolean isImmutable)
                      at Amazon.DynamoDBv2.DynamoDBEntryConversion..cctor()
Data             : {}
InnerException   :
HelpLink         :
Source           : System.Private.CoreLib
HResult          : -2146232830

[austoonz]

@ashishdhingra - I don't think this is a FeatureRequest anymore; the AWS.Tools.DynamoDBv2 module is currently broken for interactions with the DynamoDB data APIs.

Further details on this can be found on StackOverflow. We shouldn't have to use a workaround to install and load a separate NuGet package and before using these cmdlets.

[pcgeek86]

Any updates on this? The PowerShell SDK for AWS works pretty well, but I can't believe such an important feature is just allowed to go missing for years. @ashishdhingra

[ashishdhingra]

Any updates on this? The PowerShell SDK for AWS works pretty well, but I can't believe such an important feature is just allowed to go missing for years. @ashishdhingra

@pcgeek86 Apologies for no movement on the issue. There is some DynamoDB work planned in coming sprints. I will discuss with development team to revisit this issue and other PowerShell CmdLets.

[cole-seph-fi]

After stumbling through tutorials for quite some time, this issue exactly sums up my experience and disappointment with these modules so far. Hope this new fixed functionality ships soon!

[jared-is-ray]

A possible workaround for this in the meantime is to use PartiQL statements to manipulate table items.

The AWS Tools for PowerShell have a cmdlet called Invoke-DDBDDBExecuteStatement which can execute any PartiQL statement. Statements can also be parameterized which makes it easy to pass in native PowerShell types to statements from the CLI.

Here's an example of a parameterized INSERT statement that passes in a GUID, a string, and an integer.

Invoke-DDBDDBExecuteStatement -Statement "INSERT INTO ""MyTable"" VALUE {'Id' : ?, 'FieldA' : ?, 'FieldB': ?}" -Parameter ([Guid]::NewGuid().ToString()),"string value",1234

[mark-hubers]

A possible workaround for this in the meantime is to use PartiQL statements to manipulate table items.

The AWS Tools for PowerShell have a cmdlet called Invoke-DDBDDBExecuteStatement which can execute any PartiQL statement. Statements can also be parameterized which makes it easy to pass in native PowerShell types to statements from the CLI.

Here's an example of a parameterized INSERT statement that passes in a GUID, a string, and an integer.

Invoke-DDBDDBExecuteStatement -Statement "INSERT INTO ""MyTable"" VALUE {'Id' : ?, 'FieldA' : ?, 'FieldB': ?}" -Parameter ([Guid]::NewGuid().ToString()),"string value",1234

Hi Jared-is-ray, Do you have any examples to share with getting data from PartiQL into some hash or array of hash?

[jared-is-ray]

@mark-hubers Unfortunately read operations appear to be a little trickier because it Invoke-DDBDDBExecuteStatement appears to return records in the low-level object model. You can use pure PowerShell to handle the response but it requires some conversion.

The following example does not handle all DynamoDB data types but illustrates how you could go about reading record attributes into a PowerShell-friendly object.

function ConvertFrom-DDBRecord($item) {
  $record = @{}
  foreach($i in $item.Keys) {
    if($item[$i].SS) {
      $record[$i] = $item[$i].SS
    } elseif($item[$i].NS) {
      $record[$i] = $item[$i].NS
    } elseif($item[$i].S) {
      $record[$i] = $item[$i].S
    } elseif($item[$i].N) {
      $record[$i] = $item[$i].N
    } elseif($item[$i].NULL) {
      $record[$i] = $null
    } elseif($null -ne $item[$i].BOOL) {
      $record[$i] = $item[$i].BOOL
    } else {
      $record[$i] = [string]$attribute
    }
  }
  [pscustomobject]$record
}

Using the above function you could display records in the PowerShell table format, for example.

Invoke-DDBDDBExecuteStatement -Statement 'SELECT Key1,Key2 FROM "Table"' | % {ConvertFrom-DDBRecord $_}

Example output:

Key1    Key2
----    ----
Pk1     Value123
Pk2     Value234

[jnunn-aws]

Hi Kevin,

Thanks for posting this GitHub issue! The cmdlets for DynamoDB items have been added today in version 4.1.308:

1. Get-DDBItem
2. Set-DDBItem
3. Remove-DDBItem
4. Update-DDBItem
5. Invoke-DDBQuery
6. Invoke-DDBScan
7. Get-DDBBatchItem
8. Set-DDBBatchItem

Version 4.1.308 also includes helper cmdlets for converting between dictionaries of DynamoDB attribute values and hashtables of types that are more commonly used in PowerShell.

1. ConvertFrom-DDBItem
2. ConvertTo-DDBItem

Examples for using these new cmdlets have been included in the Get-Help documentation.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.