bluesky-social / ozone

web interface for labeling content in atproto / Bluesky
https://atproto.com
Other
218 stars 15 forks source link

Archival storage / db offloading (feature request) #127

Open Bossett opened 1 month ago

Bossett commented 1 month ago

Just watching the size of the ozone database climbing to about 420mb after 30 days, it occurs to me that the intro tiers of most hosting providers are going to be hit within the first year or so. It is also likely that users will want to keep data well beyond that period.

It would be good to have built-in support for archiving/compaction/offlining of data. My preferred option would be a built-in tool to dump tables to disk (or S3?), truncate all tables appropriately (i.e. maybe not including 'active' labels), and present a download interface. Users could then just choose a schedule that suits and keep the data as approp.

The benefit of building this in: the date of last archive can be displayed at the bottom of events lists, etc. where an account is older than last archive date to indicate that there may be more data about a user available.