Malformed cursor causing issues with paginated queries

[ngomezdelcampo]

Describe the bug When I use a query the count option, my cursor is sometimes malformed. Note that the merchantid is incomplete.

eyJwayI6IiRwYWxvbW1hI21lcmNoYW50aWRfIiwic2siOiIkY3VzdG9tZXJfMSNpZF8wMWhwancwOGU0Y2c4anIyOGNkZnEzdzhqOCJ9 {"pk":"$palomma#merchantid_","sk":"$customer_1#id_01hpjw08e4cg8jr28cdfq3w8j8"}

When I make the same query with the limit option, the cursor is properly formed.

eyJwayI6IiRwYWxvbW1hI21lcmNoYW50aWRfZmVsaXBlIiwic2siOiIkY3VzdG9tZXJfMSNpZF8wMWhwaHB2bnNod2RlYTd4MzJxZWc3MzljayJ9 {"pk":"$palomma#merchantid_felipe","sk":"$customer_1#id_01hphpvnshwdea7x32qeg739ck"}

The impact is that pagination breaks with count with the following error:

The query can return at most one row and cannot be restarted

ElectroDB Version 2.13.1

ElectroDB Playground Link https://electrodb.fun/?#code/PQKgBAsg9gJgpgGzARwK5wE4Es4GcA0YuccYGeqCALgUQBYCG5YA7llXWAGbZwB2MXGBDAAUKKwBbAA5QMVMAG8wAUT5V2AT0IBlTADcsAY1IBfbhiiSwAIkRwjVSzABGNgNzijUPrgVUGFwRSAF5bABVA4I9xUDAAVWIMITUNKk1hMW9fBVQkoTC+OBZVdS0AClEwJSrqsElYRAAuGrq6-jTNFpsjPKorTBt8WrakwxNuseM4IZG6-UxcLB9ugEZZttNhtoYqJywXVCo8FsU56slMI0Z1AEkYU-O69Ok4Sf2+AHMNtrryNCw5AeYCc6G2v2qdCwMHgfBaoLg4M2SLq0MeEOqLzetj82C+Pwh-1QgLgwIRKLa5AYMAA8nwEF0QRgwU9qvAuAxKFQWuUAJRgEIAPjARgwmmk-QAdBgGAIrPF4rcACJ8p5bJ7kLiYfgTVoYrHvPHfClgdUQvgMS7o-Xi7E2XFYfEm6pEklk5mItUmmBQXqXdThW3WiEGsAAbR6RiGthM0ZsfHYNgAumAGEJsn5nWQ4ACgfCPSazb8fX6OgA5VCSFyYYO-UP2j7G1nZ3Ok-MsiFFtpwSQMLAIWttesOp3N115pkd35durSOg+OAVqs1vUhoM4xsE37jtuTz2dk2iuC7UkAQW5q7r64bRq3lOPtPpjPJY5zxInL4xYHZnOoPP5QpgEUJRKiefKSv0tw6DSOiNnyhYmqg0gwCeMDnoOzzXiOTZfiwuzXN0IB3n8b5uu2+4Yj+XL-gKwrAWAoHHOBkHQbBRrwc2xAXuUSEoccaFUABgrNtUvGoeeYAAGSSWAACEWC4GWDBluUjFwJK0hMMQPHIeJgm8iJdQAPxgGJ-HnoZ1QtPRanMVAUEwXBvIIXMM6OvAAAeJyXtULiaBAVw3FQ9wYdU0gANanNwOAIMCNgRdG3gyFASzHC0EaXBg1yysFMDJqaWa4JFSjRYgcVFYlViyKl2IRtC+UztUjVgH5ABKcBauQfC6mcX4RVFXAxXFnxLAALAlhBJdV7C1TYmXZXceUps11RFQNQ3dCNWCjRVk1VSlM3pTYmrat1MzLVm7lwB5m1jcRM5FkWygBEEZiiLyniiHkiySkeYG9fUgU5SFtgJt4PwWlatgAApwNI7BQGAsPkAAXj8JaVh0gavN0RhRuCGP+lQS7VhgawAEwAMyjQArAAbAA7AAHAAnD8PZ9gOtivPD-QAAKfL2-a-VYPxzguJMrjYlOrLLctyzYoimLykqfFAfKeN9yS-VSTEA-NQUg-GxhQBDlp2rDvOI8jcBowTvqYwG154+jDtE5LZO2KsVO04zrPs0LXPxXDCMC4HIuSGL85FB73Qy-L8uK8rqvqx9X35Drx567UBvA3FYOm+CkMWyH-RI9qdu1ITWPO-jVdu+WlakxT1P08zbPghz-bdDzoeC5zEdRxLTdS-HCey0nKtqxr6c-X92cXEDi3dAXZtQzYlsI+XqOu6WTs4zGddsg36ix17Ptt-7neBz3pdQGHA9JUPMcj570vjwrStT6nn3AMAplLC+CKKAqB1CEA4KQYg2QYAoHQGKFqVJwq4FntrNAmBNCSj8gFLKhsYDlEULnZeoMTY2GTtPAGbRvCgIvOTE0ch4Bv3gLgI+dRlaawzmgsUmD-JL1yvgwhuUV4kLIerChdQqHqBaLQp4vRkhyG6NGP+qY8LsBBHQUgg1kgKE4RkWUMDpBHDUaQWRuA5BgEdGAdR5Anj0KlkwlhTU06xH-nkR0nwwAICkOwcB6iiAOB8DAnR5ihCDSKCg3AkodHcOwQtPhBDeFG1XiI-BTxPGSHYFIuhGAGHdHsVuNh4TIlwIwVghJeD4k4LzkI8GySxHVDSRksA0iIQmPkbYRR-8GAqP8L4zRfhYHoNTAIMABienGNQHIjA5i+CWO1DY7Jdi8AONNB9IAA

Entity/Service Definitions Include your entity model (or a model that sufficiently recreates your issue) to help troubleshoot.

  {
    model: {
      entity: "customer",
      service: "service",
      version: "1",
    },
    attributes: {
      merchantId: {
        type: "string",
        required: true,
        hidden: true,
      },
      id: {
        type: "string",
        required: true,
        readOnly: true,
        default: () => ulid()
      },
      reference: {
        type: "string",
      },
      name: {
        type: "string",
        required: true,
      },
      documentType: {
        type: ["cc", "ce", "nit"] as const,
        required: true,
      },
      documentNumber: {
        type: "string",
        required: true,
      },
      email: {
        type: "string",
        required: true,
      },
      phoneNumber: {
        type: "string",
        required: true,
      },
      createdAt: {
        type: "string",
        readOnly: true,
        required: true,
        default: () => new Date().toISOString(),
      },
      updatedAt: {
        type: "string",
        watch: "*",
        required: true,
        default: () => new Date().toISOString(),
        set: (updatedAt) =>
          updatedAt && !isNaN(Date.parse(updatedAt))
            ? updatedAt
            : new Date().toISOString(),
      },
    },
    indexes: {
      byMerchantId: {
        pk: { field: "pk", composite: ["merchantId"] },
        sk: { field: "sk", composite: ["id"] },
      },
      byReference: {
        pk: { field: "gsi4pk", composite: ["merchantId"] },
        sk: { field: "gsi4sk", composite: ["reference"] },
        index: "gsi4",
      },
    },
  }

Expected behavior The cursor of the query operation with the count option should give a properly formed cursor that enables me to query the next page.

Errors

The query can return at most one row and cannot be restarted 

[tywalch]

Thank you I will look into this!

[tywalch]

@ngomezdelcampo I started a PR request for this, I still need to add unit tests but I have identified the root cause: The composite attribute merchantId is flagged as hidden which has internal ramifications preventing it from being included at cursor creation.

[tywalch]

@ngomezdelcampo Small update on this: I started working on this PR and found that the underlying implementation was very disagreeable with this scenario. It proved a bit difficult to not add more overhead to react differently in this case, and I think it will require a bit more time determine a better refactor. Can you confirm whether or not you are still in need of this fix, or if you have moved on? I am working to triage some recent issues, thank you for your patience so far