Restoring from different backups fail with "Incorrect backup passphrase" although it's verified on same phone with same Signal version

studiok87 commented 6 months ago

[X ] I have searched open and closed issues for duplicates
[X ] I am submitting a bug report for existing functionality that does not work as intended
[X ] I have read https://github.com/signalapp/Signal-Android/wiki/Submitting-useful-bug-reports
[X ] This isn't a feature request or a discussion topic

Bug description

Attempts to restore chat backup fail with "Incorrect Passphrase" error yet correct (verified) passphrase was used. Xposed is not installed. Signal-back produces an empty xml/empty "output" folder with the backup file. This happens with Signal Version 6.41.3.1365 with three different backups, the oldest was definitely restored successful in the past and is the base for the actual backup. Refering to closed issue #12743 and 7637 etc and a lot on reddit on this error. Seems to be a common problem since years and nobody from the devs cares? Why?

Steps to reproduce

Created a backup and saved it to PC
Checked the 30 digit passphrase, it is the same like for the last times restoring backups
Updated firmware of the phone, factory reset
Copied the backup file to the phone after creating folder "Signal" and subfolder "Backup" like I did the last years
Install Signal from direct apk-Link or Aurora Store (makes no difference)
Use restore function and select one of the backup files
Enter correct 30 digit passphrase
"Incorrect passphrase" error
Going back Home, open Signal again, verify phone number
Groups are visible, but empty and no further conversations
Check passphrase in Settings -> Chat -> Backup, it verifies correctly and the time stamp of the last backup is shown correctly
Exactly the same behaviour when trying this on a backup phone, old Xperia XZ1C with Android 11

Actual result: Backup not restored, groups empty and no conversations visible Expected result: Backups restored like everytime in the last years

Screenshots

Screenshot

Device info

Device: Asus Zenfone 8 and Sony Xperia XZ1 Compact Android version: 13, Firmware 33.0210.0210.314 and Android 11/Lineage 18.1 Signal version now: 6.41.3 Signal version for creating backups: 6.41.3 last backup, some older version for the older bakups

Link to debug log

Not possible to create

Benjamin-Nussbaum commented 6 months ago

It seems like the issue could be caused by a corrupt backup. I just ran into the same error and was able to solve it with https://github.com/bepaald/signalbackup-tools#fix.

studiok87 commented 6 months ago

Thank you, but kinda strange that all 8 backups (found some more) from different harddisks are corrupt, some of them where restored successful in the last year. Tried Signalbackup-Tools, it started "bruteforcing offset...", it ran for more than 3 days & nights and had then 15% of the filesize, so I stopped it. Is that normal? CPU-load only ~16% single-core. Screenshot7

Benjamin-Nussbaum commented 6 months ago

kinda strange that all 8 backups are corrupt

I agree, particularly as you mention having restored some of them previously.

it ran for more than 3 days & nights and had then 15% of the filesize, so I stopped it. Is that normal?

I ran signalbackup-tools on my ~500M backup and it spit out a new file effectively the same size (99.9%) less than 30 seconds later. I'm not sure how typical that is in terms of speed, or distribution of correctable errors in the file.

Aside from the obvious double/triple checking you gave signalbackup-tools the correct passphrase, the only other thing I can think to suggest is to make sure you're running the most recent version. Beyond that, see https://github.com/bepaald/signalbackup-tools/issues/166#issuecomment-1848988373 or look for other related issues.

kocherjj commented 5 months ago

I came here to start an issue myself but this one seems to match what I have pretty closely.

I have a Google Pixel 6 Pro that was having some calling and Watch sync issues. After troubleshooting with AT&T I was told I had to do a factory reset on my phone. Along with everything else I needed, I backed up my Signal messages to my Nextcloud account, and verified the passphrase, which I saved in my password manager. After resetting my phone I re-installed Signal and attempted to restore from the backup. Whether I copy/paste from my password manager or manually type in the passphrase, I get the 'Incorrect backup passphrase' message, and then after a while I get a 'Restore Complete' popup asking me to choose a folder for new backups to be saved. If I move forward I eventually get Signal up, and it associates me with several groups that I was a member of, but no message history shows up. What options do I have to recover my messages? I followed the official directions exactly as recorded here: https://support.signal.org/hc/en-us/articles/360007059752-Backup-and-Restore-Messages#android_restore_steps

But it is not working. Are my years of message history gone forever? Since I reset the same phone the only place these messages exist are in this backup file. I downloaded and ran the 'signalbackup-tools' program from the linked thread above and get the ' NOTE : IT IS LIKELY AN INCORRECT PASSPHRASE WAS PROVIDED ' result.

At this point I cannot trust Signal at all since being able to backup and restore is a basic functionality that should just work. I really need to get these messages back though!

AdrienChatelain commented 3 months ago

I'm also having this issue - I successfully verified my backup passphrase prior to resetting my phone and have copied a monthly backup for the past 3 months, all of which now give the 'Incorrect backup passphrase' toast after the reset. Signalbackup-tools has the same error as above:

[Warning]: Bad MAC in frame: [REDACTED]
 *** NOTE : IT IS LIKELY AN INCORRECT PASSPHRASE WAS PROVIDED ***
[Error]: Failed to get valid frame from decoded data...
         Encrypted data had failed verification (Bad MAC)

As with the others it's important to me that I'm able to restore the backups. I accept that it could be a corrupt backup in some way, however I ensured the files were the exact same size before & after copying, though I did not compare a hash.

Signal appears to be silently creating corrupt backups - these backups are a month (or more) apart, they accept the passphrase verification in the app. Some are automatic/daily and at least one was manually triggered.

If it isn't possible to fix whatever's causing this (in the near future) would it at least be possible to integrate a 'Verify backup integrity' option to the app? It would make sense to have this by the 'Verify backup passphrase' option and potentially save a lot of stress and upset as it doesn't seem like an isolated issue.

If I had known it was going to do this I would have either turned the backups off and on and tested again, or found another way to restore it (such as using a donor phone to transfer the backup to, resetting the original phone and transferring it back).

Shallrath commented 3 months ago

I just lost my data to this. I took a screenshot of the passphrase so I can be sure I have the correct one. Even signalbackup-tools tells me the passphrase is incorrect. I created the backup with the same version of signal that I'm now using to import. I reinstalled signal on the same device because I hoped that this might fix the problem that I have to manually retrieve messages because signal is never running in background.

@AdrienChatelain When creating the backup, it already displayed something like "verifying backup". So at least they try...

Suggestion: Don't encrypt local backups (especially if you don't know how). It's a local file on storage space that is already encrypted by default by Android.

AdrienChatelain commented 3 months ago

Sorry to hear about your data loss. This is true, however I'm thinking more of a tool like how signalbackup-tools can go through and find bad sectors/corruption/no end frame etc. I disagree - not encrypting local backups would be a significant privacy/security risk, local encrypted storage or no. Especially if people export their backups to other drives etc.

Improving the signal backup process and restore mechanisms; adding more/better (automatic and manual) backup integrity verification seem like better options to me. I suppose we'll have to wait to see what OWS have to say about it

Shallrath commented 3 months ago

Is there any hope that the error that caused this can be reproduced and maybe fixed by a future version of signalbackup-tools? I still have the backup and the passphrase. Would be even better if the old broken backup could be merged into a recent one to restore everything since the loss without losing everything from after it.

How about making backup encryption optional? (Maybe after a "I know what I'm doing" dialog...)

stale[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

signalapp / Signal-Android