search

bepaald / signalbackup-tools

Tool to work with Signal Backup files.
GNU General Public License v3.0
790 stars 38 forks source link

Recreate bit-identical backup file from dir #63

Closed RolandVor closed 2 years ago

RolandVor commented 2 years ago

Thanks a lot for this great project! I have one question:

When I dump an encrypted database (signal backup file) to disk (directory) and repack the directory to an encrypted signal backup file, the two files are not identical. The NOTE in the Readme explains that this is due to the order of AvatarFrames read from the disk.

Is it somehow possible to read the correct order of AvatarFrames to regenerate an identical version of the original signal backup file? This way, I could make sure that no data was lost/corrupted when unpacking the backup and that an identical signal backup file can be recreated from the unencrypted directory structure.

bepaald commented 2 years ago

Thanks a lot for this great project! I have one question:

Thanks!

When I dump an encrypted database (signal backup file) to disk (directory) and repack the directory to an encrypted signal backup file, the two files are not identical. The NOTE in the Readme explains that this is due to the order of AvatarFrames read from the disk.

And thanks for pointing this out, I should probably remove that as I force the order of the AvatarFrames to be correct. However, there are other reasons the file might not be bit-identical. It is important to note, the backup does not actually contain a copy of the sqlite database from the app, instead, during the backup creation, a sort of recipe for recreating the database is put in the backup. If sqlite changes the way it does things between versions (and it does) sometimes the recreated database is not bit-identical (but still of course functionally identical) to the original.

I do extract backups to dir and pack them back up regularly as part of my automated testing. For all my backup this currently works, but I need to compile with -DBUILT_FOR_TESTING in order to have this work on some backups. I'm not actually sure whether this was necessary for old backups or newer ones, but it works for me. (Note, by the way, old and new backups means backups created from old or new installs)

I do not know how you are compiling, if you use the BUILDSCRIPT.sh, I suggest adding -DBUILT_FOR_TESTING to the EXTRAOPTIONS in line 7 and trying again. Otherwise just add it to your command line when compiling. If you are using Windows, let me know and I'll compile a binary with that option for you as soon as I get the chance.

If you're wondering what that option does, it's just here: https://github.com/bepaald/signalbackup-tools/blob/efcf3b2ceea7dad5536483bad6edfbb5291c328a/signalbackup/initfromfile.cc#L92-L101

Let me know if that works.

RolandVor commented 2 years ago

Thank your very much for the fast reply! With BUILT_FOR_TESTING the repacked signal backup file is byte-identical, thanks a lot! I also tried to cross compile for Windows (that's what I am using), but couldn't get it to work (I think I need to cross compile also openssl and sqlite3). So if you find some time, a Windows binary would be nice. But no hurry!

bepaald commented 2 years ago

No problem. Here is a Windows binary (untested, but it should work ok).

signalbackup-tools_win_built_for_testing.exe.gz

Indeed if you want to cross compile (I assume with mingw, I don't know any other way myself), you will need to have openssl and sqlite libraries cross compiled as well.

Let me know if there's anything else.

RolandVor commented 2 years ago

It works! Thank you very much!