mufunyo
commented
7 months ago Alright, I figured out how merge the desktop database with my backup. I realised from reading the documentation more thoroughly, that the --limittodates option also allows to set a date cutoff where the desktop database contains newer entries than the backup, contrary to the --autolimitdates option.
This worked! There were a few warnings, and the initial progress of loading the backup file was slowed down considerably by the writing of the frame count to the console, minimising the command prompt window made the process much faster. Maybe consider not updating the console that often for heavy operations.
Also of note was that Signal thought the backup dated to 1970-01-01, and said I had "run out of PIN attempts", even though I never once entered my PIN incorrectly. Scrolling through messages, it seems both the old messages from the backup show up properly as well as the newer ones from the desktop database.
Command output log with personal information redacted:
C:\Junk\signal>signalbackup-tools_win.exe signal-2023-07-03-00-43-08.backup <redacted> --importfromdesktop --output signal-merged.backup --limittodates "2023-07-03 00:43:09","2024-02-06 20:00:00"
*** Starting log: ***
signalbackup-tools (signalbackup-tools_win.exe) source version 20240205.120839 (Win)
BACKUPFILE VERSION: 0
BACKUPFILE SIZE: 2921312987
COUNTER: 2706765262
Reading backup file...
FRAME 74454 (100.0%)...
Read entire backup file...
done!
Database version: 196
Using range: 2023-07-03 00:43:09 - 2024-02-06 20:00:00 (1688341389000 - 1707246000000)
Trying to match conversation (1/12) (type: private)
- Importing 2 messages into thread._id 60
Trying to match conversation (2/12) (type: private)
- Importing 315 messages into thread._id 53
Trying to match conversation (3/12) (type: group)
- Importing 37 messages into thread._id 211
Trying to match conversation (4/12) (type: private)
- Importing 13 messages into thread._id 50
Skipping conversation, conversation has no messages in requested time period (5/12)
Trying to match conversation (6/12) (type: private)
- Importing 2 messages into thread._id 26
Trying to match conversation (7/12) (type: private)
- Importing 28129 messages into thread._id 106
[Warning]: Failed to get number of attachments in quoted message. Skipping
[Warning]: Failed to get number of attachments in quoted message. Skipping
[Warning]: Failed to get number of attachments in quoted message. Skipping
[Warning]: Failed to get number of attachments in quoted message. Skipping
Trying to match conversation (8/12) (type: private)
- Importing 1 messages into thread._id 38
Trying to match conversation (9/12) (type: private)
- Importing 4 messages into thread._id 244
Trying to match conversation (10/12) (type: private)
- Importing 99 messages into thread._id 246
Trying to match conversation (11/12) (type: private)
[Warning]: Failed to find recipient for uuid: 376fxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: <redacted>)
Trying to match conversation (12/12) (type: private)
[Warning]: Failed to find recipient for uuid: a8f8xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: <redacted>)
[Warning]: Profile data empty. Not updating group recipient.
[Warning]: Failed to update profile data.
reorderMmsSmsIds
updateThreadsEntries
Dealing with thread id: 48, 5, 13, 53, 54, 45, 136, 10, 72, 75, 30, 46, 86, 76, 68, 49, 62, 93, 95, 114, 96, 79, 6, 97, 102, 103, 98, 19, 85, 91, 26, 50, 64, 94, 90, 81, 99, 69, 88, 87, 12, 89, 100, 82, 83, 1, 60, 101, 84, 158, 147, 16, 92, 25, 144, 28, 154, 105, 40, 27, 17, 174, 161, 182, 37, 203, 188, 31, 208, 70, 56, 117, 32, 29, 204, 65, 195, 148, 34, 145, 143, 38, 7, 159, 175, 23, 177, 197, 21, 184, 183, 24, 234, 181, 178, 109, 11, 116, 73, 15, 71, 194, 22, 199, 33, 160, 146, 198, 217, 20, 55, 149, 106, 104, 108, 212, 225, 176, 187, 110, 205, 111, 112, 113, 210, 151, 201, 118, 119, 120, 121, 122, 123, 180, 124, 125, 126, 127, 219, 128, 129, 130, 131, 132, 133, 186, 134, 171, 202, 137, 140, 141, 142, 196, 150, 213, 139, 138, 135, 2, 14, 42, 4, 41, 35, 39, 8, 36, 9, 57, 80, 78, 52, 51, 47, 44, 3, 67, 155, 172, 173, 152, 153, 156, 157, 179, 162, 163, 164, 165, 166, 167, 168, 169, 170, 185, 189, 190, 191, 192, 193, 249, 200, 241, 206, 207, 209, 211, 214, 215, 216, 223, 218, 252, 220, 228, 221, 222, 224, 226, 227, 229, 230, 231, 232, 233, 235, 236, 237, 238, 244, 239, 240, 242, 243, 245, 246, 247, 248, 250, 251
Checking foreign key constraints... ok
Checking database integrity (full)... ok
Exporting backup to 'signal-merged.backup'
Writing HeaderFrame...
Writing DatabaseVersionFrame...
Writing SqlStatementFrame(s)...
Dealing with table 'avatar_picker'... 0/0 entries...done
Dealing with table 'recipient'... 485/485 entries...done
Dealing with table 'thread'... 241/241 entries...done
Dealing with table 'message'... 85193/85193 entries...done
Dealing with table 'call'... 4/4 entries...done
Dealing with table 'call_link'... 0/0 entries...done
Dealing with table 'cds'... 408/408 entries...done
Dealing with table 'chat_colors'... 0/0 entries...done
Dealing with table 'distribution_list'... 1/1 entries...done
Dealing with table 'distribution_list_member'... 0/0 entries...done
Dealing with table 'donation_receipt'... 0/0 entries...done
Dealing with table 'drafts'... 0/0 entries...done
Dealing with table 'emoji_search'... 0/0 entries...done
Dealing with table 'group_membership'... 12/12 entries...done
Dealing with table 'group_receipts'... 50/50 entries...done
Dealing with table 'groups'... 4/4 entries...done
Dealing with table 'identities'... 34/34 entries...done
Dealing with table 'kyber_prekey'... 0/0 entries...done
Dealing with table 'mention'... 3/3 entries...done
Dealing with table 'msl_payload'... 1700/1700 entries...done
Dealing with table 'msl_message'... 1857/1857 entries...done
Dealing with table 'msl_recipient'... 1778/1778 entries...done
Dealing with table 'notification_profile'... 0/0 entries...done
Dealing with table 'notification_profile_allowed_members'... 0/0 entries...done
Dealing with table 'notification_profile_schedule'... 0/0 entries...done
Dealing with table 'part'... 1/8315 entries...
[Warning]: Attachment data not found (rowid: 1, uniqueid: 0)
Dealing with table 'part'... 8315/8315 entries...done
Dealing with table 'payments'... 0/0 entries...done
Dealing with table 'pending_pni_signature_message'... 0/0 entries...done
Dealing with table 'pending_retry_receipts'... 0/0 entries...done
Dealing with table 'push'... 0/0 entries...done
Dealing with table 'reaction'... 1994/1994 entries...done
Dealing with table 'remapped_recipients'... 0/0 entries...done
Dealing with table 'remapped_threads'... 0/0 entries...done
Dealing with table 'remote_megaphone'... 5/5 entries...done
Dealing with table 'sender_key_shared'... 0/0 entries...done
Dealing with table 'sender_keys'... 0/0 entries...done
Dealing with table 'sticker'... 103/103 entries...done
Dealing with table 'storage_key'... 4/4 entries...done
Dealing with table 'story_sends'... 0/0 entries...done
Writing SharedPrefFrame(s)...
Writing KeyValueFrame(s)...
Writing Avatars...
Writing EndFrame...
Done! Wrote 4971651282 bytes.
C:\Junk\signal>Thanks for making this indispensible tool, you saved my day! Honestly, I sometimes wonder why I keep using Signal when it repeatedly spits in my face like this, but whatever. I'll be missing SMS support, but at least I didn't lose any messages.
bepaald
Hi,
I saw other users using the GitHub issues as a support/discussion forum and I hope I'm not out of line by doing the same thing.
The story is as follows. I was happily using Signal-JW-JK, which is a fork of a fork - the JW fork is probably well-known, but the JK fork adds back SMS functionality. Unfortunately, the fork stopped being maintained about half a year ago and the latest build stopped working with the Signal servers since yesterday. I decided to bite the bullet and forgo SMS functionality, and switch to regular Signal-JW.
I was a bit in a hurry (which you should never be with these things), and I made a quick backup with Swift Backup so I knew I'd at least have the db and keystore values backed up just in case. I had a screenshot with the backup key, from the last time I enabled backups. Unfortunately, I didn't verify that this key was still working before beginning. I did make a fresh backup, but that is now worthless.
I couldn't update the app from JW-JK to JW because the APK signature didn't match, so being rooted, I considered my options in forcing the update. I figured, if I keep the data, but just replace the APK, it should work. Apparently not, because messing with the APK/data in any way causes the app to force close and renders it completely useless.
This is the current predicament I find myself in. I am on the same device, with the full data present, but no version of Signal (not even the Signal-JW-JK version I was using before) will run, it just force closes. I have a recent backup, but the passphrase changed since the last time I imported it (in July 2023), and I was not notified of that change, so I have no screenshot with the new passphrase (skimming /r/signal on Reddit, this is apparently common).
I have the old backup from July 2023 that can be opened with the passphrase I have saved, but it only includes messages up to July 2023. I have Signal Desktop on a Windows PC linked since late 2022 that has all messages between 2022 and now. I have not relinked my phone, or changed numbers. I can still use Signal Desktop to send and receive messages.
I found Decrypting Signal DB for Android which seemed to suggest it's still possible to decrypt and recover the signal.db I have which Signal itself crashes on, although I don't know how to access the keystore (I don't see a
/data/keystore/folder in Solid Explorer). If I could do that, I could probably use signalbackup-tools to repackage it into a fresh backup file, and restore that, right? That seems to be the least complicated way, even though it's the most technical one.Alternatively, I saw that signalbackup-tools can import from the Signal Desktop database, which would be great, as between the July 2023 backup and the linked Signal Desktop database there is an 8+ month overlap where both databases contain the same data. However, reading the documentation, it looks like the intended use case is for when the Signal phone backup is newer than the Desktop database. In my case it's the other way around, the Signal Desktop database is up-to-date, but only stretches back to sometime in 2022, and my phone backup stretches from the beginning of time (when Signal was still called SecureSMS) to July 2023. Is there a simple way to set the correct date cutoff given my scenario, or, even easier, will duplicate entries simply be overwritten if I merge the two without a cutoff? I don't mind waiting longer if this causes the process to take longer, I just don't want any duplicate messages.
Thanks for reading my essay and sorry to bother you, I really should have verified my backup passphrase before starting to mess with APKs. I just wasn't having a great day, and that has a tendency to lead to carelessness which snowballs into having a worse day.