tywalch / electrodb

A DynamoDB library to ease the use of modeling complex hierarchical relationships and implementing a Single Table Design while keeping your query code readable.
MIT License
1.03k stars 67 forks source link

Feature request: Add Conditional Indexing Based on Attribute Presence #300

Closed misterjoshua closed 1 year ago

misterjoshua commented 1 year ago

Description I'd like ElectroDB to have a conditional indexing feature to prevent unnecessary GSI writes & hot partitions for certain data models.

Use Case I have a data model that supports up to five levels of hierarchical depth (lv1 through lv5). However, not all entities utilize all five levels. This leads to wasted GSI write capacity and creates hot partitions for GSIs.

Playground Link

Possible Solution I envisioned extending the index definition to accept a condition field with a function that can evaluate whether or not to populate the index fields. The idea is that if the condition function is present and it returns false, the pk & sk for that GSI are not written.

In the following example, for instance, lv5pk and lv5sk would be left unset and the byLv5 index would not be populated unless the lv5 attribute is set:

byLv5: {
  index: 'byLv5',
  // This is the new function
  condition: (attrs) => attrs.lv5 !== '',
  pk: {
    field: 'lv5pk',
    composite: ['lv5'],
  },
  sk: {
    field: 'lv5sk',
    composite: ['lv1', 'lv2', 'lv3', 'lv4'],
  }
},
tywalch commented 1 year ago

This is a reasonable request, let me look into the work necessary to get something to solve for this case

misterjoshua commented 1 year ago

Thanks dude!!!