rocicorp / replicache

Realtime Sync for Any Backend Stack
https://doc.replicache.dev
1.01k stars 37 forks source link

Add support for other JavaScript types to indexes #1040

Open aboodman opened 1 year ago

aboodman commented 1 year ago

This was just something we overlooked when first implementing. We would have to add some bits to the index value to distinguish types and decide on an ordering.

arv commented 1 year ago

The obvious type to add support for is number. However, the ordering we use is UTF8 strings. How do we encode an arbitrary number in a way that we preserve the ordering? This paper looks promising: https://www.zanopha.com/docs/elen.pdf

There is also the issue of how to deal with the secondary key in scan

aboodman commented 1 year ago

I'm not following. At the basic level we can clearly encode the number however we want and decode it to compare. Is the problem that you're trying to find a way that we can avoid that and compare as a string? That's an interesting problem. It seems similar to the fracdex problem?

arv commented 1 year ago
  1. The "value" is part of the key in the underlying B+Tree we use for the index.
  2. To get scan to iterate over the keys in the correct order we need to have a mapping from number to string that preserves the numeric order. If it was only ints we could zero pad the number for example.
  3. For scan we can pass the start key which is a tuple for index scans consisting of the [secondary: string, primary: string]. If the index is over numbers we would want to pass a number into secondary