search

vlcn-io / cr-sqlite

Convergent, Replicated SQLite. Multi-writer and CRDT support for SQLite
https://vlcn.io
MIT License
2.61k stars 69 forks source link

lookaside for primary keys #344

Closed tantaman closed 9 months ago

tantaman commented 10 months ago

Primary keys for CRRs are generally going to be UUIDs or some other large-ish thing.

Moving them out into a lookaside lets us keep a variable length encoded int in the clock tables instead of the full primary key. This is useful since clock tables keep an entry per cell rather than a single entry per row.

In tables with < 65k entries this save 14 bytes. In tables not exceeding 4 billion entries, saves us 12 bytes.

(aside: we can use 64 bit ints in many cases for primary keys in cases where we can segment the population similar to https://mariadb.com/kb/en/uuid_short/)

jeromegn commented 9 months ago

Will this be a breaking change or is there an easy non-breaking upgrade path? It's fine if not, but I'll need to know since we released 0.1.0 of Corrosion and I want to upgrade to this as soon as possible :)

tantaman commented 9 months ago

Definitely breaking but we could put together a migration script that fills in the lookaside tables as an upgrade path.

On Fri, Sep 22, 2023 at 8:32 AM Jerome Gravel-Niquet < @.***> wrote:

Will this be a breaking change or is there an easy non-breaking upgrade path? It's fine if not, but I'll need to know since we released 0.1.0 of Corrosion and I want to upgrade to this as soon as possible :)

— Reply to this email directly, view it on GitHub https://github.com/vlcn-io/cr-sqlite/pull/344#issuecomment-1731426829, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHWK23M3TFXEQYHSSK7WGLX3WHNXANCNFSM6AAAAAA4N34SSA . You are receiving this because you authored the thread.Message ID: @.***>