Possible Workaround for failed merge

[carow-it]

This is, at the time of writing, a non issue I solved, but wanted to share for documentation. I had two backups with database version 171 (1,1 GB) and 179 (60 MB) and attempted a merge. This failed immediately, so I imported the older backup into Signal and created a new backup, which now became version 179. Afterward, I attempted to merge the big file into the small one, which also failed. The final successful attempt was small into big. The resulting file also imported without issues. I still have all the erroneous files if anyone needs me to recreate the exact error messages for further improvements.

[bepaald]

Hi, thanks for your feedback.

Merging different database versions is a known possible problem. While many database versions work perfectly fine together, there are also many that have incompatible SQL schema's, making it impossible to merge them without upgrading one of them. Something like this is actually mentioned in the README, although it then mentions some ancient database versions (which were problematic at the time of writing, but there are many more now).

https://github.com/bepaald/signalbackup-tools/blob/1babe5f4d26b7165a97c499e1be0e4c9a4e8ca41/README.md?plain=1#L402

I shall update this part of the README sometime in the next few days.

Afterward, I attempted to merge the big file into the small one, which also failed.

I am curious about this. I have heard this before — that one way works and the other does not — and I do always recommend merging the smaller file into the larger one, but, honestly I don't know why this difference exists. Could you tell what failed exactly? Did the program end with an error, or was the resulting backup not able to be restored by Signal? If it's not too much trouble, I would love the output for the failed big-into-small attempt.

Thanks!

[carow-it]

The merge itself resulted in an error, here is the output starting from the beginning of the failed thread. Keep in mind that the big backup file was previously 171 and just quickly converted to 179 with an import and rebackup. So only the small/newer file was actually a true 179. Maybe signals own db upgrade process while importing is missing features?

Importing thread 31 (29/59) from source file: .\signal-2023-02-17-14-04-21.backup
IV: (hex:) 97 f8 ff 54 44 d5 90 a4 66 cd c6 32 ad c9 ab 84 (size: 16)
SALT: (hex:) 4b cb 43 55 d0 a9 2d 97 06 33 3e 41 5f b8 0f 07 f0 aa 7a 9b c4 db 52 f2 49 e5 69 f2 71 61 55 3b (size: 32)
BACKUPKEY: (hex:) 46 a2 ab 7b 93 c9 3c b5 b6 6c 7c 8b 01 e9 78 d0 34 31 59 d0 33 d5 d2 79 9f d1 bc 0e 61 2e 11 3e (size: 32)
CIPHERKEY: (hex:) ed d5 33 38 ff 52 3c 21 fc b9 14 65 d9 cd 61 ac 78 c1 05 f0 bf 51 7d d8 78 24 30 16 ec 9d 79 0b (size: 32)
MACKEY: (hex:) 3f b2 c4 7f ba 52 d8 45 8b 85 c4 98 3d e4 e3 00 ef d2 52 fc 83 fc 98 4a 55 cd ca 83 8e d1 15 37 (size: 32)
COUNTER: 2549677908
Reading backup file...
FRAME 29780 (100.0%)... Read entire backup file...

done!
importThread
WARNING: Source database contains 'remapped_recipients'. This case may not yet be handled correctly by this program!
1 : 314 -> 878

Old id:
(no results)

New id:
_id|uuid|username|phone|email|group_id|group_type|blocked|message_ringtone|message_vibrate|call_ringtone|call_vibrate|notification_channel|mute_until|color|seen_invite_reminder|default_subscription_id|message_expiration_time|registered|system_display_name|system_photo_uri|system_phone_label|system_phone_type|system_contact_uri|system_info_pending|profile_key|profile_key_credential|signal_profile_name|profile_family_name|profile_joined_name|signal_profile_avatar|profile_sharing|last_profile_fetch|unidentified_access_mode|force_sms_selection|storage_service_key|dirty|mention_setting|storage_proto|capabilities|last_gv1_migrate_reminder|last_session_reset|wallpaper|wallpaper_file|about|about_emoji|system_family_name|system_given_name|extras|groups_in_common|chat_colors|custom_chat_colors_id|badges|pni|distribution_list_id|needs_pni_signature|unregistered_timestamp|hidden|reporting_token
878|8838cef7-beb1-4346-a080-5d95865ff3d7|(NULL)|Redacted|(NULL)|(NULL)|0|0|(NULL)|0|(NULL)|0|(NULL)|0|C130|0|-1|0|1|Redacted|content://com.android.contacts/display_photo/6|(NULL)|2|content://com.android.contacts/contacts/lookup/833i2b2994b78f22e3cd.3789r420-382A44442A2C2A4C5C.2399r773-382A44442A2C2A4C5C/4889|0|TaEX0ZcmXuIPiJbRtpmSWLVUYoWCz/mXWChkVDsrcNs=|CiBNoRfRlyZe4g+IltG2mZJYtVRihYLP+ZdYKGRUOytw2xKZAQBC+C3iT11vGKTMgCH+EFQJiS9rh7tfaLJ942tiyswuCp76S8IWyPAczayxYWvFD1vzQL379ffAOl7S5v58j1VFbijBNlLRoxEv5vwoUPjGDsGPPmeRzej0P9G6DaPOOV2IOM73vrFDRqCAXZWGX/PXTaEX0ZcmXuIPiJbRtpmSWLVUYoWCz/mXWChkVDsrcNsA/dpjAAAAAA==|Redacted||Redacted|(NULL)|0|1674750377501|2|0|1LNmW+WGWgMqrOhXUi1ZAQ==|0|0|(NULL)|169300|0|(NULL)|(NULL)|(NULL)|||Redacted|Redacted|(NULL)|1|(NULL)|0|(hex:) |(NULL)|(NULL)|(NULL)|0|0|(NULL)

2 : 734 -> 882

Old id:
(no results)

New id:
_id|uuid|username|phone|email|group_id|group_type|blocked|message_ringtone|message_vibrate|call_ringtone|call_vibrate|notification_channel|mute_until|color|seen_invite_reminder|default_subscription_id|message_expiration_time|registered|system_display_name|system_photo_uri|system_phone_label|system_phone_type|system_contact_uri|system_info_pending|profile_key|profile_key_credential|signal_profile_name|profile_family_name|profile_joined_name|signal_profile_avatar|profile_sharing|last_profile_fetch|unidentified_access_mode|force_sms_selection|storage_service_key|dirty|mention_setting|storage_proto|capabilities|last_gv1_migrate_reminder|last_session_reset|wallpaper|wallpaper_file|about|about_emoji|system_family_name|system_given_name|extras|groups_in_common|chat_colors|custom_chat_colors_id|badges|pni|distribution_list_id|needs_pni_signature|unregistered_timestamp|hidden|reporting_token
882|28063edd-3360-491e-b0d9-56f0b608e990|(NULL)|Redacted|(NULL)|(NULL)|0|0|(NULL)|0|(NULL)|0|(NULL)|0|C110|0|-1|0|1|Redacted|content://com.android.contacts/display_photo/5|(NULL)|2|content://com.android.contacts/contacts/lookup/833i3881bb028ee3d29b.3789r520-402A524C2A2C2A4C5C.2399r774-402A524C2A2C2A4C5C/4893|0|dMkMfbhATpqvLlxnKqjsN4nATQ6ll09U+SosL/sRL1g=|CiB0yQx9uEBOmq8uXGcqqOw3icBNDqWXT1T5Kiwv+xEvWBKZAQDoSEovpHQfL28HFQScSp+4kH0o+30WUl/crnWMIGCxCzSKZ2tVm/Z9OW+GpJPnsKkr+x6bGYj8inUHvaoyALU5up+uxTVopw0DV2r4+/e1Kzz7SJ8F+12OzGDPYNdD3V0oBj7dM2BJHrDZVvC2COmQdMkMfbhATpqvLlxnKqjsN4nATQ6ll09U+SosL/sRL1iAwtFjAAAAAA==|Redacted||Redacted|profiles/8MZeCheJ48XYWtzZB5uIfA|0|1674666889174|2|0|2i8yQYYpo8qdfN+5Wk6fOw==|0|0|(NULL)|169300|0|(NULL)|(NULL)|(NULL)|||Redacted|Redacted|(NULL)|1|(NULL)|0|(hex:) |(NULL)|(NULL)|(NULL)|0|0|(NULL)

3 : 979 -> 2

Old id:
(no results)

New id:
_id|uuid|username|phone|email|group_id|group_type|blocked|message_ringtone|message_vibrate|call_ringtone|call_vibrate|notification_channel|mute_until|color|seen_invite_reminder|default_subscription_id|message_expiration_time|registered|system_display_name|system_photo_uri|system_phone_label|system_phone_type|system_contact_uri|system_info_pending|profile_key|profile_key_credential|signal_profile_name|profile_family_name|profile_joined_name|signal_profile_avatar|profile_sharing|last_profile_fetch|unidentified_access_mode|force_sms_selection|storage_service_key|dirty|mention_setting|storage_proto|capabilities|last_gv1_migrate_reminder|last_session_reset|wallpaper|wallpaper_file|about|about_emoji|system_family_name|system_given_name|extras|groups_in_common|chat_colors|custom_chat_colors_id|badges|pni|distribution_list_id|needs_pni_signature|unregistered_timestamp|hidden|reporting_token
2|2eeade49-a9c2-4ecb-a441-743efe064bad|(NULL)|Redacted|(NULL)|(NULL)|0|0|(NULL)|0|(NULL)|0|(NULL)|0|A190|0|-1|0|1|Redacted|content://com.android.contacts/contacts/644/photo|(NULL)|2|content://com.android.contacts/contacts/lookup/833i76899c658a864faa.3789r644-443A404E4E2E3832443E3240.2399r735-443A404E4E2E3832443E3240/4737|0|d708Juxe2Vn/WfxwhW5R2tS+ZWX8vAuLjSvbf6skQng=|CiB3vTwm7F7ZWf9Z/HCFblHa1L5lZfy8C4uNK9t/qyRCeBKZAQBpUrzC4YOgHzxVnEueDmqLejX8Ne6WGZsICgQNnWF8Dnr1kYORBr7Pl0xMF18M0f2Ph6lbZuBjbqo7fms/tHorxDYcrCZisdl0KSEHPVPp/hi8pOqyNg9s7cT3yQTufEQu6t5JqcJOy6RBdD7+Bkutd708Juxe2Vn/WfxwhW5R2tS+ZWX8vAuLjSvbf6skQngAKHRjAAAAAA==|Redacted||Redacted|profiles/D9zqobh3W7ZjFrLUPJhKBQ==|1|1668496178601|2|0|fCwpP1vZpTKKzi1UF7AqCA==|0|0|(NULL)|38228|0|(NULL)|(NULL)|(NULL)|(NULL)|(NULL)|Redacted|Redacted|(NULL)|1|(NULL)|(NULL)|(hex:) |(NULL)|(NULL)|(NULL)|0|0|(NULL)

4 : 1072 -> 1084

Old id:
(no results)

New id:
_id|uuid|username|phone|email|group_id|group_type|blocked|message_ringtone|message_vibrate|call_ringtone|call_vibrate|notification_channel|mute_until|color|seen_invite_reminder|default_subscription_id|message_expiration_time|registered|system_display_name|system_photo_uri|system_phone_label|system_phone_type|system_contact_uri|system_info_pending|profile_key|profile_key_credential|signal_profile_name|profile_family_name|profile_joined_name|signal_profile_avatar|profile_sharing|last_profile_fetch|unidentified_access_mode|force_sms_selection|storage_service_key|dirty|mention_setting|storage_proto|capabilities|last_gv1_migrate_reminder|last_session_reset|wallpaper|wallpaper_file|about|about_emoji|system_family_name|system_given_name|extras|groups_in_common|chat_colors|custom_chat_colors_id|badges|pni|distribution_list_id|needs_pni_signature|unregistered_timestamp|hidden|reporting_token
1084|f609052f-a32f-4d02-8c9f-d99f4327b1c9|(NULL)|Redacted|(NULL)|(NULL)|0|0|(NULL)|0|(NULL)|0|(NULL)|0|A130|0|-1|0|1|Redacted|content://com.android.contacts/contacts/673/photo|(NULL)|2|content://com.android.contacts/contacts/lookup/833i77a5494c88f3b49f.3789r673-422A3A3E2C462E3E403A4436.2399r741-422A3A3E2C462E3E403A4436/4761|0|oKx+62lj5x7Eo+I+3PfYyZtzguvD6DXsuVRYnC6KU0I=|CiCgrH7raWPnHsSj4j7c99jJm3OC68PoNey5VFicLopTQhKZAQBI4E/zA3sg7JsvN+OubpcKjcG+uJveqcU9Eo7ZlUtzBXDQG8WrI9j0PBaDED2tqseDbxB5dqG6bvvhSHBBjfc3pk4YwpjY146z0DtOVJipzDzYrwyPAUnuVtjtlBVFCnz2CQUvoy9NAoyf2Z9DJ7HJoKx+62lj5x7Eo+I+3PfYyZtzguvD6DXsuVRYnC6KU0IA/dpjAAAAAA==|Redacted|Redacted|Redacted|(NULL)|0|1674730482427|2|0|jGE3/DreZrOPHbfvSMRhlA==|0|0|(NULL)|169300|0|(NULL)|(NULL)|(NULL)|||Redacted|Redacted|(hex:) |1|(NULL)|(NULL)|(hex:) |(NULL)|(NULL)|(NULL)|0|0|(NULL)

5 : 1087 -> 1086

Old id:
(no results)

New id:
_id|uuid|username|phone|email|group_id|group_type|blocked|message_ringtone|message_vibrate|call_ringtone|call_vibrate|notification_channel|mute_until|color|seen_invite_reminder|default_subscription_id|message_expiration_time|registered|system_display_name|system_photo_uri|system_phone_label|system_phone_type|system_contact_uri|system_info_pending|profile_key|profile_key_credential|signal_profile_name|profile_family_name|profile_joined_name|signal_profile_avatar|profile_sharing|last_profile_fetch|unidentified_access_mode|force_sms_selection|storage_service_key|dirty|mention_setting|storage_proto|capabilities|last_gv1_migrate_reminder|last_session_reset|wallpaper|wallpaper_file|about|about_emoji|system_family_name|system_given_name|extras|groups_in_common|chat_colors|custom_chat_colors_id|badges|pni|distribution_list_id|needs_pni_signature|unregistered_timestamp|hidden|reporting_token
1086|f8a69822-2ff6-4c71-88bc-306e87a29634|(NULL)|Redacted|(NULL)|(NULL)|0|0|(NULL)|0|(NULL)|0|(NULL)|0|A140|0|-1|0|1|Redacted|(NULL)|(NULL)|2|content://com.android.contacts/contacts/lookup/833i5fde38b3082cc53b.3789r684-442A303A44324C323A2E382A4C3050.2399r713-442A303A44324C323A2E382A4C3050/4649|0|wFej1/mjL+eZ62X/6NXv+GcF+fBrrRitSTyhNc7q69k=|CiDAV6PX+aMv55nrZf/o1e/4ZwX58GutGK1JPKE1zurr2RKZAQBFQ/sJGDGu/k/m5NmZSL+r4Vy5mTA7a7r3nRGtMnv+AG7G2HX31uI213BzzRSJxjrrZ28QH+zqxI8gr1KPf3Ir9rhVNyBc5TsYoViBuwgu27zA9vX0FN4Kb248oTWmYQ74ppgiL/ZMcYi8MG6HopY0wFej1/mjL+eZ62X/6NXv+GcF+fBrrRitSTyhNc7q69kAt9VjAAAAAA==|Redacted||Redacted|(NULL)|0|1674730482427|2|0|PWLxeOJ39Z/oX9cCAtq0uA==|0|0|(NULL)|169300|0|(NULL)|(NULL)|(NULL)|(NULL)|(NULL)|Redacted|Redacted|(hex:) |1|(NULL)|(NULL)|(hex:) |(NULL)|(NULL)|(NULL)|0|0|(NULL)

6 : 1088 -> 1082

Old id:
(no results)

New id:
_id|uuid|username|phone|email|group_id|group_type|blocked|message_ringtone|message_vibrate|call_ringtone|call_vibrate|notification_channel|mute_until|color|seen_invite_reminder|default_subscription_id|message_expiration_time|registered|system_display_name|system_photo_uri|system_phone_label|system_phone_type|system_contact_uri|system_info_pending|profile_key|profile_key_credential|signal_profile_name|profile_family_name|profile_joined_name|signal_profile_avatar|profile_sharing|last_profile_fetch|unidentified_access_mode|force_sms_selection|storage_service_key|dirty|mention_setting|storage_proto|capabilities|last_gv1_migrate_reminder|last_session_reset|wallpaper|wallpaper_file|about|about_emoji|system_family_name|system_given_name|extras|groups_in_common|chat_colors|custom_chat_colors_id|badges|pni|distribution_list_id|needs_pni_signature|unregistered_timestamp|hidden|reporting_token
1082|e65183f2-3318-44b0-847b-696d4d48372b|(NULL)|Redacted|(NULL)|(NULL)|0|0|(NULL)|0|(NULL)|0|(NULL)|0|A130|0|-1|0|1|Redacted|content://com.android.contacts/contacts/685/photo|(NULL)|2|content://com.android.contacts/contacts/lookup/833ic67fa0c8e31601c.3789r685-3C52403A2A38522C324C.2399r767-3C52403A2A38522C324C/4865|0|JbUCbjK+AWR62nmwvXGZvGt/n5GTHZoerCgUszQEtE0=|CiAltQJuMr4BZHraebC9cZm8a3+fkZMdmh6sKBSzNAS0TRKZAQDNYSDddtlpqileVZ/f+81IgmIrwwqE02GV4NBqzcLEAF77yXP5kOrAiOxr3u0FA+aCCzHiRDjEaUFRxZwjGRNj2LSxuB5Ad3N14/Wk633QdfmVL90n3yvhh1IvniPTsgzmUYPyMxhEsIR7aW1NSDcrJbUCbjK+AWR62nmwvXGZvGt/n5GTHZoerCgUszQEtE0At9VjAAAAAA==|Redacted|Redacted|Redacted|(NULL)|1|1674730482427|2|0|yGjOg+UI5ZfY/eTZghuBsQ==|0|0|(NULL)|169300|0|(NULL)|(NULL)|(NULL)|||Redacted|Redacted|(hex:) |1|(NULL)|(NULL)|(hex:) |(NULL)|(NULL)|(NULL)|0|0|(NULL)

7 : 1091 -> 1085

Old id:
(no results)

New id:
_id|uuid|username|phone|email|group_id|group_type|blocked|message_ringtone|message_vibrate|call_ringtone|call_vibrate|notification_channel|mute_until|color|seen_invite_reminder|default_subscription_id|message_expiration_time|registered|system_display_name|system_photo_uri|system_phone_label|system_phone_type|system_contact_uri|system_info_pending|profile_key|profile_key_credential|signal_profile_name|profile_family_name|profile_joined_name|signal_profile_avatar|profile_sharing|last_profile_fetch|unidentified_access_mode|force_sms_selection|storage_service_key|dirty|mention_setting|storage_proto|capabilities|last_gv1_migrate_reminder|last_session_reset|wallpaper|wallpaper_file|about|about_emoji|system_family_name|system_given_name|extras|groups_in_common|chat_colors|custom_chat_colors_id|badges|pni|distribution_list_id|needs_pni_signature|unregistered_timestamp|hidden|reporting_token
1085|fb9fdf88-1d9c-4296-b65c-5c29a53271e7|(NULL)|Redacted|(NULL)|(NULL)|0|0|(NULL)|0|(NULL)|0|(NULL)|0|A200|0|-1|0|1|Redacted|(NULL)|(NULL)|2|content://com.android.contacts/contacts/lookup/833i3cd904ef0aa20af0.3789r680-302A443A32402A4E2E385238422A2E38324C.2399r695-302A443A32402A4E2E385238422A2E38324C/4577|0|FxvZdVPiVJtghn4guEeKO7qagiYoTElR5A6UAcs3+rc=|CiAXG9l1U+JUm2CGfiC4R4o7upqCJihMSVHkDpQByzf6txKZAQDhUeO2+rnZhh9HmL+mTOR/KNKGTnHWOIcOC9ieHUtfBJB+78X/6DxA6TmPbFAkpyErZpTQhFsiRDLaqxFPuMtOpLTuyo7FKNuTAJaIspGGtQa5pTMFATbTF72E8iGbURr7n9+IHZxClrZcXCmlMnHnFxvZdVPiVJtghn4guEeKO7qagiYoTElR5A6UAcs3+rcAt9VjAAAAAA==|Redacted|Redacted|Redacted|(NULL)|0|1674730482427|2|0|CS7hDghLiXPuiQEh+sfuBQ==|0|0|(NULL)|169300|0|(NULL)|(NULL)|(NULL)|||Redacted|Redacted|(hex:) |1|(NULL)|(NULL)|(hex:) |(NULL)|(NULL)|(NULL)|0|0|(NULL)

8 : 1089 -> 1092

Old id:
(no results)

New id:
_id|uuid|username|phone|email|group_id|group_type|blocked|message_ringtone|message_vibrate|call_ringtone|call_vibrate|notification_channel|mute_until|color|seen_invite_reminder|default_subscription_id|message_expiration_time|registered|system_display_name|system_photo_uri|system_phone_label|system_phone_type|system_contact_uri|system_info_pending|profile_key|profile_key_credential|signal_profile_name|profile_family_name|profile_joined_name|signal_profile_avatar|profile_sharing|last_profile_fetch|unidentified_access_mode|force_sms_selection|storage_service_key|dirty|mention_setting|storage_proto|capabilities|last_gv1_migrate_reminder|last_session_reset|wallpaper|wallpaper_file|about|about_emoji|system_family_name|system_given_name|extras|groups_in_common|chat_colors|custom_chat_colors_id|badges|pni|distribution_list_id|needs_pni_signature|unregistered_timestamp|hidden|reporting_token
1092|8796377b-3b2f-4b3a-87ec-ca7cb179f4e3|(NULL)|Redacted|(NULL)|(NULL)|0|0|(NULL)|0|(NULL)|0|(NULL)|0|A110|0|-1|0|1|Redacted|(NULL)|(NULL)|2|content://com.android.contacts/contacts/lookup/833i465bb07a8f2f7135.3789r682-4E2A44304C2A503846423A.2399r729-4E2A44304C2A503846423A.2638r1144-/4713|0|PlINCnb5vrDpTPtCTKQAa6Dr3KATURFBDyEgTtAyr6w=|CiA+Ug0Kdvm+sOlM+0JMpABroOvcoBNREUEPISBO0DKvrBKZAQAlPxo/Khpt/UxQzmMlIgXSo7+l+zcPRJFgB7VsqHvdBjyqN9z9vehPUVSFBThXBn7/vNLde7yl9ohskuRArRQDnA+M4wXYLB3dWhZJ6JZQbKpm8rmWihOuO9cTAV8xZxSHljd7Oy9LOofsynyxefTjPlINCnb5vrDpTPtCTKQAa6Dr3KATURFBDyEgTtAyr6yACNdjAAAAAA==|Redacted|Redacted|Redacted|(NULL)|0|1674730482427|2|0|W/AUqGcSqWJdrLTrwEUfmw==|0|0|(NULL)|169300|0|(NULL)|(NULL)|(NULL)|(NULL)|(NULL)|Redacted|Redacted|(hex:) |1|(NULL)|(NULL)|(hex:) |(NULL)|(NULL)|(NULL)|0|0|(NULL)

9 : 1090 -> 1093

Old id:
(no results)

New id:
_id|uuid|username|phone|email|group_id|group_type|blocked|message_ringtone|message_vibrate|call_ringtone|call_vibrate|notification_channel|mute_until|color|seen_invite_reminder|default_subscription_id|message_expiration_time|registered|system_display_name|system_photo_uri|system_phone_label|system_phone_type|system_contact_uri|system_info_pending|profile_key|profile_key_credential|signal_profile_name|profile_family_name|profile_joined_name|signal_profile_avatar|profile_sharing|last_profile_fetch|unidentified_access_mode|force_sms_selection|storage_service_key|dirty|mention_setting|storage_proto|capabilities|last_gv1_migrate_reminder|last_session_reset|wallpaper|wallpaper_file|about|about_emoji|system_family_name|system_given_name|extras|groups_in_common|chat_colors|custom_chat_colors_id|badges|pni|distribution_list_id|needs_pni_signature|unregistered_timestamp|hidden|reporting_token
1093|faeacf76-c6d7-456e-97a9-4a2e3f414fe1|(NULL)|Redacted|(NULL)|(NULL)|0|0|(NULL)|0|(NULL)|0|(NULL)|0|A200|0|-1|0|1|Redacted|(NULL)|(NULL)|2|content://com.android.contacts/contacts/lookup/833i43548a370c54a1a1.3789r683-4E3A4246443250323A2E38324C.2399r756-4E3A4246443250323A2E38324C/4821|0|5p7XAQf1AzAGzXCABk7BgAHInIbCgGsHBqiT5GxkV0M=|CiDmntcBB/UDMAbNcIAGTsGAAcichsKAawcGqJPkbGRXQxKZAQB64fSA9AZVgw7sX2CePVnHY7Ned148a586uxjM5XjrD9AygdG8gE3okmWT45RTjPERsk7WVwyx1vk2NM8IZnZm+hAwtxWCE71fLELDMtT4WQMOe9LfKaPmh8RHfCTYeBz66s92xtdFbpepSi4/QU/h5p7XAQf1AzAGzXCABk7BgAHInIbCgGsHBqiT5GxkV0MA/dpjAAAAAA==|Redacted|Redacted|Redacted|profiles/AaWKIT4ZK78vdIz5wkPmqA==|0|1674730482427|2|0|rAfxvABT/t6WEe5aNUh2zQ==|0|0|(NULL)|169300|0|(NULL)|(NULL)|(NULL)|||Redacted|Redacted|(hex:) |1|(NULL)|(NULL)|(hex:) |(NULL)|(NULL)|(NULL)|0|0|(NULL)

 REMAP RECIPIENTS!
  Mapping 314 -> 878 (up)
  Mapping 734 -> 882 (up)
  Mapping 979 -> 2 (up)
  Mapping 1072 -> 1084 (up)
  Mapping 1087 -> 1086 (up)
  Mapping 1088 -> 1082 (up)
  Mapping 1091 -> 1085 (up)
  Mapping 1089 -> 1092 (up)
  Mapping 1090 -> 1093 (up)
Deleted releasechannel recipient from source database (_id: 938)
Failed to find recipient._id matching uuid/phone/group_id in target database
cropToThread
  Deleting messages not belonging to requested thread(s) from 'mms'
cleanDatabaseByMessages
  Deleting attachment entries from 'part' not belonging to remaining message entries
  Deleting other threads from 'thread'...
updateThreadsEntries
  Dealing with thread id: 31
  Deleting entries from 'mention' not belonging to remaining mms entries
  Deleting removed groups...
  Deleting unneeded MessageSendLog entries...
  Deleting reactions to non-existing messages
  Deleting call details from non-existing messages
  Deleting unreferenced recipient entries...
  Deleted 1089 unreferenced recipients
Deleted 1 unneeded distribution_lists
  Deleting unneeded notification profiles entries...
  Deleting unused avatars...
  Deleting unused attachments...
  Delete others from 'identities'
  Deleting group_receipts entries from deleted messages...
  Deleting group_receipts from non-existing recipients
  Deleting drafts from deleted threads...
  Deleting remapped recipients for non existing recipients
  Vacuuming database
  Deleted 21 existing storage_keys
  Deleted 3 existing remote_megaphone's
  Deleted 1036 existing cds's
makeIdsUnique
  Adjusting indexes in tables...
setMinimumId thread
  Adjusting 'message.thread_id' to match changes in 'thread'
  Adjusting 'drafts.thread_id' to match changes in 'thread'
  Adjusting 'mention.thread_id' to match changes in 'thread'
setMinimumId message
  Adjusting 'part.mid' to match changes in 'message'
  Adjusting 'group_receipts.mms_id' to match changes in 'message'
  Adjusting 'mention.message_id' to match changes in 'message'
  Adjusting 'msl_message.message_id' to match changes in 'message'
  Adjusting 'reaction.message_id' to match changes in 'message'
  Adjusting 'story_sends.message_id' to match changes in 'message'
  Adjusting 'call.message_id' to match changes in 'message'
compactIds
  Compacting table: message (_id)
setMinimumId part
  Adjusting 'message.link_previews' to match changes in 'part'
compactIds
  Compacting table: part (_id)
setMinimumId recipient
  Adjusting 'message.recipient_id' to match changes in 'recipient'
  Adjusting 'message.quote_author' to match changes in 'recipient'
  Adjusting 'sessions.address' to match changes in 'recipient'
  Adjusting 'group_receipts.address' to match changes in 'recipient'
  Adjusting 'thread.recipient_id' to match changes in 'recipient'
  Adjusting 'groups.recipient_id' to match changes in 'recipient'
  Adjusting 'remapped_recipients.old_id' to match changes in 'recipient'
  Adjusting 'remapped_recipients.new_id' to match changes in 'recipient'
  Adjusting 'mention.recipient_id' to match changes in 'recipient'
  Adjusting 'msl_recipient.recipient_id' to match changes in 'recipient'
  Adjusting 'reaction.author_id' to match changes in 'recipient'
  Adjusting 'notification_profile_allowed_members.recipient_id' to match changes in 'recipient'
  Adjusting 'payments.recipient' to match changes in 'recipient'
  Adjusting 'identities.address' to match changes in 'recipient'
  Adjusting 'distribution_list.recipient_id' to match changes in 'recipient'
  Adjusting 'distribution_list_member.recipient_id' to match changes in 'recipient'
  Adjusting 'story_sends.recipient_id' to match changes in 'recipient'
  Adjusting 'pending_pni_signature_message.recipient_id' to match changes in 'recipient'
  Adjusting 'call.peer' to match changes in 'recipient'
  Adjusting 'group_membership.recipient_id' to match changes in 'recipient'
setMinimumId groups
compactIds
  Compacting table: groups (_id)
setMinimumId identities
compactIds
  Compacting table: identities (_id)
setMinimumId group_receipts
compactIds
  Compacting table: group_receipts (_id)
setMinimumId drafts
compactIds
  Compacting table: drafts (_id)
setMinimumId sticker
compactIds
  Compacting table: sticker (_id)
setMinimumId msl_payload
  Adjusting 'msl_recipient.payload_id' to match changes in 'msl_payload'
  Adjusting 'msl_message.payload_id' to match changes in 'msl_payload'
compactIds
  Compacting table: msl_payload (_id)
setMinimumId msl_recipient
compactIds
  Compacting table: msl_recipient (_id)
setMinimumId msl_message
compactIds
  Compacting table: msl_message (_id)
setMinimumId group_call_ring
compactIds
  Compacting table: group_call_ring (_id)
setMinimumId remapped_recipients
compactIds
  Compacting table: remapped_recipients (_id)
setMinimumId remapped_threads
compactIds
  Compacting table: remapped_threads (_id)
setMinimumId mention
compactIds
  Compacting table: mention (_id)
setMinimumId reaction
compactIds
  Compacting table: reaction (_id)
setMinimumId notification_profile
  Adjusting 'notification_profile_allowed_members.notification_profile_id' to match changes in 'notification_profile'
  Adjusting 'notification_profile_schedule.notification_profile_id' to match changes in 'notification_profile'
compactIds
  Compacting table: notification_profile (_id)
setMinimumId notification_profile_allowed_members
compactIds
  Compacting table: notification_profile_allowed_members (_id)
setMinimumId notification_profile_schedule
compactIds
  Compacting table: notification_profile_schedule (_id)
setMinimumId payments
compactIds
  Compacting table: payments (_id)
setMinimumId storage_key
compactIds
  Compacting table: storage_key (_id)
setMinimumId sender_key_shared
compactIds
  Compacting table: sender_key_shared (_id)
setMinimumId sender_keys
compactIds
  Compacting table: sender_keys (_id)
setMinimumId pending_retry_receipts
compactIds
  Compacting table: pending_retry_receipts (_id)
setMinimumId avatar_picker
compactIds
  Compacting table: avatar_picker (_id)
setMinimumId distribution_list
  Adjusting 'recipient.distribution_list_id' to match changes in 'distribution_list'
  Adjusting 'distribution_list_member.list_id' to match changes in 'distribution_list'
compactIds
  Compacting table: distribution_list (_id)
setMinimumId distribution_list_member
compactIds
  Compacting table: distribution_list_member (_id)
setMinimumId donation_receipt
compactIds
  Compacting table: donation_receipt (_id)
setMinimumId story_sends
compactIds
  Compacting table: story_sends (_id)
setMinimumId cds
compactIds
  Compacting table: cds (_id)
setMinimumId remote_megaphone
compactIds
  Compacting table: remote_megaphone (_id)
setMinimumId pending_pni_signature_message
compactIds
  Compacting table: pending_pni_signature_message (_id)
setMinimumId call
compactIds
  Compacting table: call (_id)
setMinimumId group_membership
compactIds
  Compacting table: group_membership (_id)
  updateRecipientIds
  Mapping 987 -> 2 (up)
Importing statements from source table 'avatar_picker'...0 entries...
Importing statements from source table 'recipient'...1 entries...
  NOTE: Dropping column 'recipient.dirty' from source : Column not present in target database
  NOTE: Dropping column 'recipient.last_gv1_migrate_reminder' from source : Column not present in target database
Error : After sqlite3_step(): UNIQUE constraint failed: recipient.phone
  Query: "INSERT INTO recipient (_id,uuid,username,phone,email,group_id,group_type,blocked,message_ringtone,message_vibrate,call_ringtone,call_vibrate,notification_channel,mute_until,color,seen_invite_reminder,default_subscription_id,message_expiration_time,registered,system_display_name,system_photo_uri,system_phone_label,system_phone_type,system_contact_uri,system_info_pending,profile_key,profile_key_credential,signal_profile_name,profile_family_name,profile_joined_name,signal_profile_avatar,profile_sharing,last_profile_fetch,unidentified_access_mode,force_sms_selection,storage_service_key,mention_setting,storage_proto,capabilities,last_session_reset,wallpaper,wallpaper_file,about,about_emoji,system_family_name,system_given_name,extras,groups_in_common,chat_colors,custom_chat_colors_id,badges,pni,distribution_list_id,needs_pni_signature,unregistered_timestamp,hidden,reporting_token) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)"
Importing statements from source table 'thread'...1 entries...
Importing statements from source table 'message'...62 entries...
Importing statements from source table 'call'...0 entries...
Importing statements from source table 'cds'...0 entries...
Importing statements from source table 'chat_colors'...0 entries...
Importing statements from source table 'distribution_list'...0 entries...
Importing statements from source table 'distribution_list_member'...0 entries...
Importing statements from source table 'donation_receipt'...0 entries...
Importing statements from source table 'drafts'...0 entries...
Importing statements from source table 'emoji_search'...0 entries...
Importing statements from source table 'group_call_ring'...0 entries...
Importing statements from source table 'group_membership'...0 entries...
Importing statements from source table 'group_receipts'...0 entries...
Importing statements from source table 'groups'...0 entries...
Importing statements from source table 'identities'...0 entries...
Importing statements from source table 'mention'...0 entries...
Importing statements from source table 'msl_payload'...0 entries...
Importing statements from source table 'msl_message'...0 entries...
Importing statements from source table 'msl_recipient'...0 entries...
Importing statements from source table 'notification_profile'...0 entries...
Importing statements from source table 'notification_profile_allowed_members'...0 entries...
Importing statements from source table 'notification_profile_schedule'...0 entries...
Importing statements from source table 'part'...11 entries...
Importing statements from source table 'payments'...0 entries...
Importing statements from source table 'pending_pni_signature_message'...0 entries...
Importing statements from source table 'pending_retry_receipts'...0 entries...
Importing statements from source table 'push'...0 entries...
Importing statements from source table 'reaction'...0 entries...
Importing statements from source table 'remapped_recipients'...0 entries...
Importing statements from source table 'remapped_threads'...0 entries...
Importing statements from source table 'remote_megaphone'...0 entries...
Importing statements from source table 'sender_key_shared'...0 entries...
Importing statements from source table 'sender_keys'...0 entries...
Importing statements from source table 'sticker'...1509 entries...
Importing statements from source table 'storage_key'...0 entries...
Importing statements from source table 'story_sends'...0 entries...
updateThreadsEntries
  Dealing with thread id: 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31
Checking foreign key constraints...
ERROR Foreign key constraint violated. This will not end well, aborting.
      Please report this error to the program author.
------------------------------
| table   | parent    | fkid |
------------------------------
| message | recipient | 0    |
| thread  | recipient | 0    |
------------------------------
A fatal error occurred while trying to import thread 31. Aborting

[bepaald]

Thank you so much for this info. I'm wondering if you would be willing to help me try to further diagnose this issue by occasionally re-testing or even running some custom functions and reporting back the result?

In fact, I already added a small custom function to test a hypothesis. If you want to help (and whenever you have time of course), could you run signalbackup-tools [smallbackup] [smallbackuppassphrase] --carowit [bigbackup] [bigpackuppassphrase]? It might give me a bigger clue where things are going wrong. There will probably be a phone number (or two) in the output that you'll want to censor, but I would like to be able to tell whether they were identical or different (same with the uuid's, but I'm not sure there is a need to censor them).

Thanks again!

[Meteor0id]

I still have all the erroneous files if anyone needs me to recreate the exact error messages for further improvements.

These files contain all your Signal chats and contacts, you might not want to share those files. And if you don't share them, than only you can run these tests.

[carow-it]

Sorry for the late reply, I have been in ill for the last two weeks and I still want to help out. I could purge the logs from personal data with regex. But they are not my daily business so it would take me a while to build the right one. Y'all don't happen to have one ready for this specific case?