Some media are not exported

[jbaker6953]

For some reason a small number of attachments are not exported. For example, see this message that definitely as an attachment and the attachment is present.

 * Executing query: SELECT * from mms where _id = 37864
_id|thread_id|date|date_received|date_server|msg_box|read|body|part_count|ct_l|address|address_device_id|exp|m_type|m_size|st|tr_id|delivery_receipt_count|mismatched_identities|network_failures|subscription_id|expires_in|expire_started|notified|read_receipt_count|quote_id|quote_author|quote_body|quote_attachment|quote_missing|quote_mentions|shared_contacts|unidentified|previews|reveal_duration|reactions|reactions_unread|reactions_last_seen|remote_deleted|mentions_self|notified_timestamp|viewed_receipt_count|server_guid|receipt_timestamp|ranges|is_story|parent_story_id|quote_type|export_state|exported
37864|1|1670890796153|1670890804118|1670890802739|10485780|1|(NULL)|1||31|(NULL)|(NULL)|132|(NULL)|1|(NULL)|0|(NULL)|(NULL)|-1|0|0|0|0|0|(NULL)|(NULL)|-1|0|(NULL)|(NULL)|1|(NULL)|0|(NULL)|0|1670891314822|0|0|1670891234954|0|fdfb615f-4d06-42fe-badf-28b9b791f644|-1|(NULL)|0|0|0|(NULL)|0

 * Executing query: select * from part where mid = 37864
_id|mid|seq|ct|name|chset|cd|fn|cid|cl|ctt_s|ctt_t|encrypted|pending_push|_data|data_size|file_name|thumbnail|aspect_ratio|unique_id|digest|fast_preflight_id|voice_note|data_random|thumbnail_random|width|height|quote|caption|sticker_pack_id|sticker_pack_key|sticker_id|data_hash|blur_hash|transform_properties|transfer_file|display_order|upload_timestamp|cdn_number|borderless|sticker_emoji|video_gif
8161|37864|0|video/mp4|(NULL)|(NULL)|jbIgrzEzZY85tYFH+CLl/mhn6HndFp8iFdMdoqJB4qusdXFav1K5FSdANKs4pm6dN7bW2uIX4DGn4TsydyJapA==|(NULL)|(NULL)|0BCWGJXPD_ixwwc-tqAD|(NULL)|(NULL)|(NULL)|2|(NULL)|2148004|(NULL)|(NULL)|(NULL)|1670890804128|(hex:) 02 2e 0f 94 fc 25 41 0a dc da 3f ad 90 dd aa f3 09 5e 73 e0 d5 6c f1 77 73 db 38 bf e8 9a 94 a1|(NULL)|0|(NULL)|(NULL)|720|1280|0|(NULL)|(NULL)|(NULL)|-1|(NULL)|LQDv4hNHM{xZ~WR-Rjniozt8j]NG|{"skipTransform":false,"videoTrim":false,"videoTrimStartTimeUs":0,"videoTrimEndTimeUs":0,"sentMediaQuality":0,"videoEdited":false}|(NULL)|0|1670890799813|2|0|(NULL)|0

However, --output produces no corresponding "Attachment[_id][unique_id]" in the output. In this case I expect an Attachment_8161_1670890804128.bin and Attachment_8161_1670890804128.sbf. Neither are created. In my Signal database there are 56 instances of attachments that are not exported.

[jbaker6953]

There doesn't appear to be any readily apparent commonality between the missing attachments.

_id|mid|seq|ct
752|1883|0|image/jpeg
794|1940|0|image/png
795|1941|0|image/png|
1083|3091|0|image/jpeg
1202|4123|0|image/jpeg
1222|4288|0|image/jpeg
1240|4491|0|image/png
1252|4587|0|image/jpeg
1364|4998|0|image/jpeg
1750|7005|0|image/jpeg
1921|7628|0|image/png
2325|9379|0|image/jpeg
2389|9590|0|image/jpeg
2756|10935|0|image/jpeg
2986|11486|0|image/jpeg
3106|11893|0|image/jpeg
3436|13190|0|image/jpeg
3471|13327|0|image/jpeg
3472|13328|0|image/jpeg
4264|17075|0|image/jpeg
5440|22831|0|image/jpeg
5623|23552|0|image/png
5624|23553|0|image/png
7192|32014|0|image/png
7193|32015|0|image/png
7194|32016|0|image/png
7195|32017|0|image/png
7196|32018|0|image/png
7197|32019|0|image/png
7198|32020|0|image/png
7199|32021|0|image/png
7200|32022|0|image/png
7201|32023|0|image/png
7202|32024|0|image/png
7203|32025|0|image/png
7204|32026|0|image/png
7205|32027|0|image/png
7206|32028|0|image/png
7208|32030|0|image/png
7209|32031|0|image/png
7210|32032|0|image/png
7211|32033|0|image/png
7212|32034|0|image/png
7213|32035|0|image/png
7214|32036|0|image/png
7215|32037|0|image/png
7217|32039|0|image/png
7218|32040|0|image/png
7293|32539|0|image/jpeg
7473|33241|0|image/jpeg
7492|33348|0|video/mp4
7693|34520|0|image/jpeg
7694|34521|0|image/jpeg
7779|35112|0|image/pngg
7934|36497|0|image/png
8070|37479|0|image/png
8161|37864|0|video/mp4

[bepaald]

Hm, strange. Have you actually confirmed there are attachments missing? If you still have this database on a working Signal install on a phone, you could try to locate these messages in a conversations on the phone. Selecting the mms.body and DATETIME(ROUND(mms.date_received / 1000), 'unixepoch') should help you find them quite easily.

Another simple way to test: if you use --output to dump the backup to a directory and then pack that directory back up (with signalbackup-tools [DIRECTORY] -o [output] -op [passphrase]), compare its size to the original: if there were 56 attachments missing from the directory (that were in the original backup) the new backup would be significantly smaller than the original. I expect it to be no more than about 50 bytes in difference (or even bit identical, depending on a few factors).

[bepaald]

Lastly, you could run with --scanmissingattachments to see if the attachments are expected to be missing. Of course, if there is a bug in finding the attachments, there could also be a bug in this function, but if you get 'expected' 56 times, you can be fairly sure there is no problem.

Thanks!

[bepaald]

Any news? I'm thinking there was no bug here...

[bepaald]

If you have more info, feel free to post more. Just closing this for cleanup now.