fireproof-storage / fireproof

Realtime database, runs anywhere. Install Fireproof in your front-end app or edge function, and sync data via any backend.
https://fireproof.storage
Other
219 stars 16 forks source link

fast forward remote changes #96

Open jchris opened 4 months ago

jchris commented 4 months ago

When a remote makes a sequence of changes in quick succession, they might come faster than the local can merge them.

Most of the time when this happens we can just skip to the most recent one, assuming they are a direct chain of parent/child relationships.

https://www.youtube.com/watch?v=j0X9fRf4cTk

The video shows the queue buildup we can fast foward.

Here is the code to where we can fast foward through parent child chains in the queue https://github.com/fireproof-storage/fireproof/blob/de8c51f348ad88d3e5a479e3774ffa1ee1577af0/packages/fireproof/src/apply-head-queue.ts#L29

jchris commented 4 months ago

Immediately after composing the above, I realized this is the correct concept but the wrong CRDT. It should be applied in the encrypted-blockstore module which means doing the work of making DbMeta track parents properly (currently hacked in as runtime state). see https://github.com/fireproof-storage/fireproof/blob/de8c51f348ad88d3e5a479e3774ffa1ee1577af0/packages/encrypted-blockstore/src/loader.ts#L116 for where it should be applied

jchris commented 4 months ago

Maybe we need to add parents array to this type, where the parents are links to the same type https://github.com/fireproof-storage/fireproof/blob/de8c51f348ad88d3e5a479e3774ffa1ee1577af0/packages/connect/src/connection.ts#L139 DbMetaEventBlock