search

maxnowack / signaldb

SignalDB is a local JavaScript database with a MongoDB-like interface and TypeScript support, enabling optimistic UI with signal-based reactivity across multiple frameworks. It integrates easily with libraries like Angular, Solid.js, Preact, and Vue, simplifying data management with schema-less design, in-memory storage, and fast queries.
https://signaldb.js.org/
MIT License
274 stars 11 forks source link

Bug: SyncManager syncOperations are never cleaned up #945

Closed obedm503 closed 2 weeks ago

obedm503 commented 3 weeks ago

The SyncManager's internal syncOperations collection never shrinks. This could be an issue for people using local storage persistence because of its small storage limit but still applies to all adapters.

I have an app with 5 collections using the SyncManager and an indexed db persisted.

This is what the initial storage state looks like after it loads: Screenshot from 2024-09-17 20-31-21 Everything is good so far: 5 snapshots, 5 sync operations, no changes, and no remote changes.

Here is what it looks like after editing each type of entity:

Screenshot from 2024-09-17 20-38-39 Notice how sync-operations grew significantly.

Snapshots also grew but it never got past twice the number of collections. I assume this has something to do with how the pull method interacts with remote changes. Not a big deal.

This is what it looks like after editing all each type of entity on a different client to make this one receive remote changes: Screenshot from 2024-09-17 20-40-17

Snapshots returned to it's initial size but sync operations kept growing.

maxnowack commented 3 weeks ago

At the moment I'm not limiting the sync operations as I wasn't sure if we might need them later. Of course, we could delete all done operations and keep only the last one, but what about error cases? Should we also keep only the last error, maybe a few more or even all of them? Right now I'm thinking about a parameter that allows setting a retention of sync operations. What do you think about this solution?

maxnowack commented 2 weeks ago

For now, let's just cleanup the sync operations