queryItems requires { hidden:true } to support groupByType for AWS SDK v3

[robpott]

The hidden parameter is required for queryItems to support groupByType when using AWS SDK V3. AWS SDK V2 does NOT require the hidden parameter.

debug.ts

/*
    debug.ts - Just for debug

    Edit your test case here and invoke via: "jest debug"

    Or run VS Code in the top level directory and just run.
 */
import {AWS, Client, Entity, Match, Model, Table, print, dump, delay} from './utils/init'
import {OneSchema} from '../src/index.js'

jest.setTimeout(7200 * 1000)

//  Change with your schema
const schema = {
    version: '0.0.1',
    indexes: {
        primary: {hash: 'pk', sort: 'sk'},
        gs1: {hash: 'gs1pk', sort: 'gs1sk', project: 'all'},
    },
    params: {
        createdField: 'createdAt',
        updatedField: 'updatedAt',
        isoDates: true,
        timestamps: true,
    },
    models: {
        User: {
            pk: {type: String, value: '${_type}#${email}'},
            sk: {type: String, value: '${_type}#'},
            email: {type: String, required: true},
        },
    } as const,
}

//  Change your table params as required
const table = new Table({
    name: 'DebugTable',
    client: Client,
    partial: false,
    schema,
    logger: true,
})

//  This will create a local table
test('Create Table', async () => {
    if (!(await table.exists())) {
        await table.createTable()
        expect(await table.exists()).toBe(true)
    }

    let User = table.getModel('User')

    let user = await User.create({
        email: 'user@example.com',
    })
})

test('items are grouped when hidden:true', async () => {
    let items = await table.queryItems({pk: 'User#user@example.com'}, {hidden: true})
    let collection = table.groupByType(items)
    expect(collection.User.length).toBe(1)
})

test('items are grouped when hidden is NOT defined', async () => {
    let items = await table.queryItems({pk: 'User#user@example.com'})
    let collection = table.groupByType(items)
    expect(collection.User.length).toBe(1)
})

test('Destroy Table', async () => {
    await table.deleteTable('DeleteTableForever')
    expect(await table.exists()).toBe(false)
})

When running with AWS SDK V3 below is the failure

 FAIL  test/debug.ts (8.379 s)
  ● items are grouped when hidden:true

    TypeError: Cannot read properties of undefined (reading 'length')

      59 |     let items = await table.queryItems({pk: 'User#user@example.com'}, {hidden: true})
      60 |     let collection = table.groupByType(items)
    > 61 |     expect(collection.User.length).toBe(1)
         |                            ^
      62 | })
      63 |
      64 | test('items are grouped when hidden is NOT defined', async () => {

      at Object.<anonymous> (test/debug.ts:61:28)

  ● items are grouped when hidden is NOT defined

    TypeError: Cannot read properties of undefined (reading 'length')

      65 |     let items = await table.queryItems({pk: 'User#user@example.com'})
      66 |     let collection = table.groupByType(items)
    > 67 |     expect(collection.User.length).toBe(1)
         |                            ^
      68 | })
      69 |
      70 | test('Destroy Table', async () => {

      at Object.<anonymous> (test/debug.ts:67:28)

Expected behavior When using AWS SDK V3 the hidden parameter should NOT be required

Environment (please complete the following information):

• Node Version: 18.17.0
• OneTable Version: 2.7.1
• TypeScript Version: 4.5.4
• AWS SDK Versions: 3.540.0 and 2.1516.0

[mobsense]

Thanks for the report and test case.

I'm getting different results.

• queryItems is part of the low-level API. It does not use params.parse: true so you get RAW dynamodb data. i.e. unmarshalled. Looks like {S: "value}. Use "find" which is part of the high level API or use parse:true in the params.
• If you use find, then you don't provide the PK, but rather the ingredients: i.e. "email".
• GroupByType always requires that the typeField be provided. The typeField is hidden by default, so you need to provide {hidden: true} for both V2 and V3 always.

So net/net

Use {hidden: true} AND if you use queryItems, you need {parse: true}

[robpott]

Thanks for getting back to me. In this case we need to use queryItems to search only on PK so it's an edge case.

What results do you get for those test cases with aws adk V2 and V3? I do NOT need {parse: true} for the test to pass for either aws adk V2 or V3, just need to pass {hidden: true} on for SDK v3 to pass

[mobsense]

In V2, you don't need parse: true because the underlying DocumentClient from the AWS v2 sdk does the parsing back to javascript. But safe, and recommended to use for both.

[robpott]

Thank you for information!