search

netcreateorg / netcreate-2018

Please report bugs, problems, ideas in the project Issues page: https://github.com/netcreateorg/netcreate-2018/issues
Other
11 stars 2 forks source link

It'd be nice to save db backups occasionally #213

Open netcreateorg opened 2 years ago

netcreateorg commented 2 years ago

Right now, we have occasional bugs that lead to corrupted data, and it is really hard to recover. Ideally, we'd have backups that we can go back to.

One option is to save every major change to a new DB, but presumably that'll get really big.

Can we check internally somehow if the data is good and then delete if it is? A record of old databases would be nice, but it's only really crucial because of the corruption / crash and we can't always predict those yet.

Flagging for @kalanicraig to weigh in

benloh commented 2 years ago

From email: I think it should be relatively easy to implement. The question really is when you want the backup to be made. We can do it periodically, e.g. every minute, or on every save, which might be quite often (I can't remember off the top of my head, but we might be doing saves every second). I think the big worry would be how much space it can eat up over time, especially on DO instances. Granted, the files tend to be relatively small, but they can add up, especially if you're on a small drive.

benloh commented 2 years ago

@jdanish Can you please attach an example corrupted db file? (The "VF0i1.loki" file you had emailed to me was a weird binary with only 157 bytes...but maybe that's the resulting corrupted file? NOTE you'll probably need to zip it).

netcreateorg commented 2 years ago

VFOI1.loki.zip

netcreateorg commented 2 years ago

I'm thinking every update but will defer to Kalani.

benloh commented 2 years ago

One additional idea: We can save off backups to a backup folder to keep the runtime folder more pristine.

benloh commented 2 years ago

This is partially implemented: A backup is made whenever an import is completed. The functionality for creating backups is there. The keys is figuring a mechanism for triggering that -- automatic? or manual?

Save this for the next round.

netcreateorg commented 1 year ago

Yes, definitely this.

There’s an on-exit thing when someone leaves the browser and disconnects from the server, right? How often does that happen, and could that action trigger a backup? Or is that too late? (Like someone crashes after doing something stupid, triggers a backup and then the backup doesn’t get enough of what happened in the editing session?)

—k

On Mar 14, 2022, at 11:41 AM, benloh @.***> wrote:

One additional idea: We can save off backups to a backup folder to keep the runtime folder more pristine.

— Reply to this email directly, view it on GitHub https://github.com/netcreateorg/netcreate-2018/issues/213#issuecomment-1066969917, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALDNP5QYX6CQHJVBOE3YAQ3U75MZPANCNFSM5PCMCNQA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you authored the thread.