feat: Collect old IDB databases

rocicorp / replicache

Realtime Sync for Any Backend Stack

https://doc.replicache.dev

1.02k stars 37 forks source link

feat: Collect old IDB databases #955

Closed arv closed 2 years ago

arv commented 2 years ago

We now collect IDB databases that have not been opened for 3 months.

We check 5 minutes after startup as well as every 12 hours. In theory we should be able to check much less frequently. We could compute when the oldest one would expire and not check until then...

Fixes rocicorp/replicache#803, rocicorp/mono#121

vercel[bot] commented 2 years ago

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/rocicorp/replicache/5HXEKj5stDuXRqH6ymDBvy2dS5cS
✅ Preview: https://replicache-git-arv-gc-old-idb-rocicorp.vercel.app

aboodman commented 2 years ago

Thanks for getting rocicorp/mono#121 in here too.

Wasn't there already some code for collecting old clients in the same idb instance? Should it be merged into this so all GC related code is shared?
I think 1 month is a better value than 3 months. For users like Vercel these will pile up fast (all documents are their own IDB).

Note: the client-collecting code I believe uses the value 1 week for the timeout, but I think it makes sense that clients have a shorter timeout than idbs, because clients are really sessions. If a session is older than say, a day, old chances of it getting revisited quickly go to zero.

arv commented 2 years ago

Wasn't there already some code for collecting old clients in the same idb instance? Should it be merged into this so all GC related code is shared?

The code that collects clients is very different. It rewrites the clients map and then the dag gc collects the chunks that are no longer reachable. This code is checking the last opened time and deletes the whole IDB database based on that. I don't see a reason to try share any code here.