search

toastclub / pinefore

🌲 Tools for a kinder web
https://pinefore.com
GNU Affero General Public License v3.0
6 stars 0 forks source link

Proper deleted entities handler #3

Closed boehs closed 3 months ago

boehs commented 5 months ago

In most applications, the client reflectors data every time the app is loaded. Pinefore is designed to handle offline replication, which poses some unique challenges.

Particularly, I do not wish to store data that the user has decided to delete. This is of course problematic because all clients should reflect deletions. How do you possibly instruct others to delete data that you don't know exists?

Conventional strategies involve tombstoning, a lightweight table that stores two entities: The IDs of deleted content, and the IDs of the users who deleted them. I suppose a timestamp as well. This is the most practical option, but it's more data than I'd like to store.

The other strategy I can think of is hash based binary searches. Basically the client asks "for this segment of replicated data, what is the hash?", and if the hash is different, hash between two sections of replicated data, and then narrow down until I don't know... 20 items? And then just replicate that part again and then the client says "hey I have this one but you don't, and I know it was replicated", but this sounds computationally expensive. This is a pretty big UX hole so we ought to sort it out.

Jak2k commented 4 months ago

What about tombstoning combined with a "garbage collector", so that the table of deleted entities get only goes back in time a certain period?

boehs commented 4 months ago

@Jak2k sorry, I missed this.

Good point, I like it. The question then becomes "what is a suitable time period"

The way sessions work is they work for 30 days, and are renewed each time they're used. That means that after 30 days, you could be sure all clients are up to date. EXCEPT

API tokens currently last 3 years, as refreshing them is a PITA I don't wish to impose upon application developers, which means it is possible (though exceedingly unlikely) that a client could reconnect after... say... 2 years... and be confused.

The latter option is a 50x longer retention period!!

boehs commented 3 months ago

Closing for issue clarity but I'm more than happy to continue discussing this