Merge failed successfully

[RobotCaleb]

First of all, this has gone quite pleasantly. Nice work and thank you.

Two backups. The first (A) is version 180 and has a contact and messages that have since been deleted using Signal. The second (B) is latest version 211 or 221 made yesterday, I don't remember exactly).

A is 14 gigabytes B is 16.2 gigabytes

Using this tool I cropped A to contain only the thread from thread list that was deleted. This results in a 474 megabyte backup that I can successfully restore from.

Restore A, create backup C to bring A into the same backup version as B.

Merge C and B using B as the first and C as the source. New backup file D, 16.6 gigabytes.

list threads shows the restored thread.

Restore backup to new install of Signal, works fine except the desired thread is absent.

Since this contact and the thread was deleted in B's history, do I need to flag something before I restore?

[bepaald]

Reading this, it sounds like it should have worked. Not entirely sure what happened, I might need to see the output of the operations.

But first, maybe you could run signalbackup-tools [D] [passphrase] --runsqlquery "SELECT * FROM thread WHERE _id = x", where x is the _id of the desired thread as shown by --listthreads. I have a theory. Feel free to censor any personal data (there may be a snippet of the last message in there).

Thanks!

[RobotCaleb]

Thanks for the prompt response! Will get back to you when I've had the chance to try it

[RobotCaleb]

_id	date	meaningful_messages	recipient_id	read	type	error	snippet	snippet_type	snippet_uri	snippet_content_type	snippet_extras	unread_count	archived	status	has_delivery_receipt	has_read_receipt	expires_in	last_seen	has_sent	last_scrolled	pinned	unread_self_mention_count	active	snippet_message_extras
82	1680132912000	12680	233	1	0	0	Lastmessagegoeshere	10485783	(NULL)	(NULL)	(NULL)	0	0	0	0	0	0	1705134211989	0	0	0	0	0	(NULL)

[bepaald]

Thanks! My theory might be correct (though I can't really explain it).

Between database version 180 and 221 (at 199 to be precise), the thread table gained a new column named active. I believe when a thread is marked inactive, it is hidden from view. The default value for this column is 0 (false). The only thing I don't understand is as far as I can see, the older database should have gained the active column and have it set to the correct value (1, or true) when it was migrated at your second step (turning A into C). But this didn't happen as active = 0 for this thread.

Possible solutions are:

• I think simply receiving or sending a message in this thread would make it active again, if still possible.
• Using this tool: signalbackup-tools [D] [passphrase] --runsqlquery "UPDATE thread SET active = 1 WHERE _id = 82" -o [E].

Let me know if this works and if the thread looks all right.

EDIT: I now do get why the thread is inactive. The thread was deleted from the [B] database. For a modern database, this means the contents of the thread were deleted, but the thread itself was just set to inactive. When importing [C] into [B], this existing (albeit inactive) thread was found and the messages were imported into it. So the active status (=0) of [B] was maintained. I think I should actually make some code changes for cases like this specifically. Thanks!

[RobotCaleb]

Thanks for your help so far.

Upon restore I see the restored thread as an entry with a snippet in the messages list. Opening that thread shows an empty new conversation. Searching doesn't find any messages from this thread.

Am curious about swapping which is the first and which is the source when merging. But I assume there's still some latent data regarding the deletion that is throwing this out of sync.

Sending a message did not rehydrate the thread, but I did it offline so as to not disturb with test messages.

[RobotCaleb]

I dumped both cropped A and active-marked E sqlite to directories. Messages appearing in A cannot be found in E. It seems that the messages didn't survive the merge.

[RobotCaleb]

I went through the entire process again. This time verifying that the messages were present before restoring. It all seems to be working quite well now.

Unfortunately, I don't have prior output because I had to restart the computer. I wish I knew what had gone awry so that I could provide you some feedback in that regard.

At this point, I think that you can put another successful merge notch on your column.

Thank you so much!

[bepaald]

Thank you for your feedback.

When the backup is restored, Signal actually deletes the contents of threads marked inactive. I suspect you ran the command I gave on a freshly exported [D] backup? This would indeed not have contained the messages anymore. The command would only have worked on [D] if it was run immediately after generating it, without restoring it in Signal. Similarly, my other suggested solution (receiving a new message in the thread) would also not have worked.

I went through the entire process again. This time verifying that the messages were present before restoring. It all seems to be working quite well now.

So this time I assume you used a backup in which the thread was already active (either offline, or by sending a message), or you updated this program since yesterday (it should now mark threads as active if messages are imported into it).

Do you think this could all be correct? If so, I think all mysteries here are solved. Thanks!

[RobotCaleb]

I honestly can't say. I know I never transferred back from the phone after restoring.

I wish I had a better recall of the process so I could know where it went weird.

[bepaald]

Ok, that's fine. The important thing is it all worked in the end. I also made some code improvements thanks to your report so that's always good.

Thank you so much for the feedback!