joshtrichards
commented
1 year ago Hi @jrr45 -
I'm sorry to hear about your data loss.
The key files can get overwritten by other processes, and are stored along with the data so they are less likely to be regularly backed up.
I've seen people cut corners or overlook things with their backups (it happens to all of us), but usually it's the database I see people overlook. :-)
Whatever mangled your key data could just as easily have mangled any key data mirrored in the database, but I can see how if you were retaining more past revisions of your NC database (versus of your NC datadirectory) that might have offered you a recovery option.
The documentation describes where the key files are located so that they can be backed up (independent of your data):
You should regularly backup all encryption keys to prevent permanent data loss. The encryption keys are stored in the following directories:
data/<user>/files_encryption
Users’ private keys and all other keys necessary to decrypt the users’ files
data/files_encryption
private keys and all other keys necessary to decrypt the files stored on a system wide external storage
As a longtime sysadmin, I would highly highly recommend recommend not having a data backup approach that only has the ability to rollback to the last iteration of data. There is no need to store twice as much data to retain the two most recent versions: any modern backup solution includes delta/differential or equivalent functionality. This means you can generally retain months of rollback iterations with minimal incremental space needed for storing backups.
I realize this doesn't help you now, but it may help prevent a future disaster.
The user, master, and recovery encryption keys should be stored in the database as well as in key files. The key files can get overwritten by other processes, and are stored along with the data so they are less likely to be regularly backed up.
The decrypt-all somehow mangled my master key and now I've lost all my data. My backups didn't include the key files as I didn't have enough space to include multiple copies off all my files at once. Having a single point of failure with the decryption keys seems like a bad design, especially with the other persistent encryption bugs.