[DB/MERGE] snippet_extras and individualRecipient leading to signal crash

[MartB]

I had one entry in the merged thread table that was referencing an unknown "individualRecipient" in its snippet_extras field. I merged together a backup from molly as a secondary linked device and my main phone as the primary.

Might be worth taking a look at? https://github.com/bepaald/signalbackup-tools/blob/998880c0bd374ad591310cbee32122f827c6e684/signalbackup/updatethreadsentries.cc#L273

Was easy to fix manually but i wonder what went wrong. The recipient should have pointed to my own account in both cases.

--------- beginning of crash
05-13 15:13:24.226 30971 31092 E AndroidRuntime: FATAL EXCEPTION: signal-FixedSizePagingController
05-13 15:13:24.226 30971 31092 E AndroidRuntime: Process: im.molly.app, PID: 30971
05-13 15:13:24.226 30971 31092 E AndroidRuntime: org.thoughtcrime.securesms.database.RecipientTable$MissingRecipientException: Failed to find recipient with ID: RecipientId::2
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at org.thoughtcrime.securesms.database.RecipientTable.findRemappedIdRecord(RecipientTable.kt:750)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at org.thoughtcrime.securesms.database.RecipientTable.getRecord(RecipientTable.kt:738)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at org.thoughtcrime.securesms.recipients.LiveRecipient.fetchAndCacheRecipientFromDisk(LiveRecipient.java:194)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at org.thoughtcrime.securesms.recipients.LiveRecipient.resolve(LiveRecipient.java:154)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at org.thoughtcrime.securesms.recipients.Recipient$Companion.resolved(Recipient.kt:885)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at org.thoughtcrime.securesms.recipients.Recipient$Companion.resolvedList(Recipient.kt:891)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at org.thoughtcrime.securesms.recipients.Recipient.resolvedList(Recipient.kt:0)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at org.thoughtcrime.securesms.conversationlist.ConversationListDataSource.load(ConversationListDataSource.java:103)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at org.signal.paging.FixedSizePagingController.lambda$onDataNeededAroundIndex$1(FixedSizePagingController.java:111)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at org.signal.paging.FixedSizePagingController.$r8$lambda$pQWvrV6w7QQq3SnkCgnHNDTtP_I(FixedSizePagingController.java:0)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at org.signal.paging.FixedSizePagingController$$ExternalSyntheticLambda2.run(R8$$SyntheticClass:0)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at java.lang.Thread.run(Thread.java:1012)
05-13 15:13:24.226 30971 31092 E AndroidRuntime:    at org.signal.core.util.concurrent.SignalExecutors$1.run(SignalExecutors.java:32)

Offending line

25  1715358001000   59  96  1   0   0   📷 Photo 10485783    content://org.thoughtcrime.securesms/part/-1/22839  image/jpeg  {"isRevealable":false,"isSticker":false,"stickerEmoji":null,"isAlbum":false,"isRemoteDelete":false,"isMessageRequestAccepted":true,"isGv2Invite":false,"groupAddedBy":null,"individualRecipientId":"2","bodyRanges":"","isScheduled":false,"isRecipientHidden":false}   0   0   0   1   1   0   1715358002086   1   0   0   0   1   {null}

[bepaald]

Thanks for reporting!

Was easy to fix manually but i wonder what went wrong.

Looking at the code I update thread.snippet_extras by querying the database for the correct recipient_id, so if the id doesn't exist, it should also be a problem in the message table (where I'm retrieving this id from). I do only update the snippet_extras if the query is successful, so I'm thinking maybe the query fails for some reason and an old, invalid, individualRecipientId might remain. (However, I can't completely explain it in that case either: any old value should be valid as it comes from the input database whose recipient_id's are not altered.)

I've now added some output when the query fails, would you be able (and willing) to re-run the same merge command with the same input and source backups again? And watch out for any warning messages at the end (Not updating thread.snippet_extras: failed to get sender)?

What were the database-versions of each backup (always the first thing printed after opening a backup)? You mention a 'secondary device' and your 'main phone', are the merged backups technically owned by two separate Signal accounts (possibly with separate phone numbers)?

Also, maybe you could use --listrecipients on both backups and check if either contain a recipient with _id = 2, and if it points to the correct recipient (yourself) on either one?

The recipient should have pointed to my own account in both cases.

What do you mean exactly with both cases, there was only one incorrect id in one thread, right? Or do you mean the snippet_extras for that thread were pointing to your own account in the two backups before merging?

Sorry for all the questions, I may have more while trying to figure out what went wrong. Thanks again!

EDIT Just thought of one way this could happen, but only if the thread is imported from the source and does not exist in the target beforehand. And the query fails. Not sure if that is what you are experiencing but I will fix that case anyway, hopefully tomorrow I had a quick go, so this case may be fixed.

[bepaald]

So, from the 'offending line' you posted, it is clear the snippet_extras was not updated by this tool (if it was, it would only have a individualRecipientId field). Since the id is normally overwritten when the snippet is updated, it is not updated correctly during the import (to reflect changes in ids), which would cause this crash.

This should now be fixed, in that the individualRecipientId should now be correctly updated while merging, so it will always point to a valid recipient. However, it is still unknown why the updating of the snippet fails in the first place. Unfortunately it will remain so without help. So: the bug should be fixed, but its root cause is unknown.

Just closing this since it's stalled without any more feedback. If you ever want to get back to this, please do not hesitate, it can easily be reopened.

Thanks!

[MartB]

Thanks for taking a look at this and the valuable insights!

I got carried away by life and did not get to reply. If i can still find the files i used while merging, i can retry that with your latest fix.