beginner-corp / begin-data

A durable and fast key/value store for Begin built on top of DynamoDB
79 stars 8 forks source link

User-defined secondary indexes #129

Open tkadlec opened 2 weeks ago

tkadlec commented 2 weeks ago

I would love to be able to pass through a secondary index to both speed things up and filter out stuff I don't need.

Here's my specific use-case...

I have an "orders" table, an "items" table, and an accounts table.

Each item can be (but doesn't have to be) associated with an order. Each order and item are each associated with an account.

Right now, if I want to pull a page that shows all items for a user's account, and provide information about all orders as well, I think (unless I'm doing it wrong), I have to query:

I'm sure there could be a structural change I could consider, but already, I feel like being able to limit the orders and items by the account by providing that as a secondary index would speed things up quite a bit (he says as if he's got any significant experience optimizing DynamoDB).

tkadlec commented 2 weeks ago

Just added a few timers to confirm, and the delay is indeed retrieving all the items. That data.get call takes ~401ms today, and as the number of users & items increases, that's gonna go up. Would love to figure out how to get that sucker cut down significantly.