tywalch / electrodb

A DynamoDB library to ease the use of modeling complex hierarchical relationships and implementing a Single Table Design while keeping your query code readable.
MIT License
1.03k stars 66 forks source link

Modeling single-partition data #323

Closed thorhj closed 1 year ago

thorhj commented 1 year ago

My table is modelling a schedule linking a user to a specific date. I have the following access patterns:

  1. Get all schedule items for one specific date
  2. Get all schedule items since a specific date, or schedules for a given month/year

Since I am never looking to fetch a singular schedule, I am simply adding all the schedule items to the same partition using "schedule" as the partition key and composing the sort key by date and user ID (only there to ensure uniqueness). For example:

PK SK
schedule date_2023-11-07#userid_42

I can model this with the following entity:

const Schedule = new Entity(
  {
    model: {
      entity: "schedule",
      service: "my-service",
      version: "1",
    },
    attributes: {
      userId: {
        field: "user_id",
        type: "string",
      },
      date: {
        field: "date",
        type: "string",
        required: true,
      },
    },
    indexes: {
      byTemplate: {
        pk: {
          composite: [],
          field: "pk",
          template: "schedule",
        },
        sk: {
          composite: ["date", "userId"],
          field: "sk",
        },
      },
    },
  }
);

Notice I am composing the pk from nothing and using a template to set a fixed value instead. This means I can query the data like so:

// Get schedules for a single date:
await Schedule.query.byTemplate({ date: "2023-11-07" }).go();

// Get schedules for 2023:
await Schedule.query.byTemplate({}).begins({ date: "2023" }).go();

// Get schedules since August 2023:
await Schedule.query.byTemplate({}).gte({ date: "2023-08" }).go();

This approach has a few downsides though:

  1. I have to use template for the partition key, meaning that I have to manually manage namespacing (which I haven't done in my example).
  2. If I want to query without a sort key I still have to pass in an empty object in Schedule.query.byTemplate({}).begins({ date: "2023" }). This is of course a small thing, but it would be nicer to just do Schedule.query.byTemplate().begins(...) when the partition key for the entity is "fixed".

If this is indeed the best way to model a single-partition entity (please let me know if there is a better way), then I would like to suggest a more convenient definition to the key schema type for fixed-partition entities:

indexes: {
  byTemplate: {
    pk: {
      value: "schedule",
      field: "pk"
    },
    sk: {
      composite: ["date", "userId"]
      field: "sk"
    }
  }
}

Here I am using value to say I want the value in the partition key to always be "schedule". You shouldn't be able to use value and composite together of course.

With this key definition the library would then be able to set a fixed partition key for the entity while still managing the namespacing. The typing could potentially be adjusted to allow undefined for the query key object (but this is potentially difficult to implement compared to the benefit).

What do you think?

sam3d commented 1 year ago

This sounds like https://github.com/tywalch/electrodb/issues/290

tywalch commented 1 year ago

@sam3d is correct here, this is the same need she described in her RFC. I added a PR this afternoon that will address this, and I'm hoping to merge it as soon as tomorrow morning 👍