(binary) MMS and SMS restore to separate threads

[williamtheaker]

MMS from the same sender/receiver conversations are restored to two separate threads: one containing the SMS and the other containing the binary attachment. This means the actual text of the conversation is unattached from the attachment, so you end up with a thread of just images.

[tmo1]

Hi,

I'm not sure I understand exactly what you're describing. MMS messages do not have SMS parts; they have one or more parts, each of which may be either text or binary (e.g., images, videos, etc.). Do you mean that MMS text parts are being separated from their corresponding binary parts, or that SMS and MMS messages between the same parties are being separated?

Furthermore, what to you mean by two separate threads? If you select a particular Contact or phone number, do you not see all messages, both SMS and MMS, to and from that Contact or number?

I appreciate your bringing this to my attention, and I don't mean to harass you; I'm just trying to get a better understanding of what you're seeing.

[tmo1]

Here's another report of what is likely the problem reported here, that SMS and MMS messages between the same contact are restored to separate threads. This is due to the way Android stores such messages internally. I suppose the app should be fixed to somehow create one unified thread upon import - if Android initially displayed the messages that way, this should be doable in a relatively straightforward way - but I'm going to have to figure out how to do it.

[slowscript]

My observation: MMS that I've sent restore to the correct thread, just the ones that I received end up separate in a thread that also has my phone number as a member. This gave me the idea to manually edit those messages in the JSON file and remove the recipient_addresses field (that contained my number) and after this it restored correctly. Probably not the right fix but could help somebody with the same problem.

[tmo1]

My observation: MMS that I've sent restore to the correct thread, just the ones that I received end up separate in a thread that also has my phone number as a member. This gave me the idea to manually edit those messages in the JSON file and remove the recipient_addresses field (that contained my number) and after this it restored correctly. Probably not the right fix but could help somebody with the same problem.

That is very interesting - thank you. I'm still trying to understand why removing one's own recipient_address would solve the problem, but if we can figure that out, it may lead to a proper solution, and your solution may be helpful to some in the meantime, as you note.

[tmo1]

I suspect that the issue is related to the fact that for SMS messages, Android only stores the address of "the other party", and not the local address (sender for sent messages, recipient for received messages), whereas for MMS messages, the addresses of both senders and recipients are stored. This may be causing Android to separate SMS and received MMS messages, since the latter have an associated address (the user's) not present in the former, and that could be why removing the user's address from the MMS JSON fixes the problem.

As you note, though, removing a recipient address doesn't seem to be the ideal solution, and in any event, if Android itself (on the original system) knows to associate the SMS and MMS messages, then there should be some way for SMS I/E to trigger this association on the new system as well. We just have to figure out what that is :)

[dece]

I encountered the same issue as OP, with every MMS misplaced, not just others' as slowscript observed, and I tried to play with the JSON to no avail. For what it's worth, the export was made on Android 11 and the import on Android 13.

[tmo1]

I encountered the same issue as OP, with every MMS misplaced, not just others' as slowscript observed, and I tried to play with the JSON to no avail.

Thank you.

I have an idea of a possible fix for this problem; I'll implement and test it when I get a chance, and report back here.

[Philippe3869]

Same behavior from Android 9 to Android 11 (both with /e/ OS). However it's a nice app, thanks ! Waiting for this fix. Note that I complete french translation using Weblate.

[tmo1]

Same behavior from Android 9 to Android 11 (both with /e/ OS). However it's a nice app, thanks ! Waiting for this fix. Note that I complete french translation using Weblate.

Thank you for the translation help and the kind words!

I have confirmed the problem with SMS and MMS messages being separated upon import. I thought I had a fix, but I was wrong and it did not work, so I'm back to the drawing board.

I need to figure out the mechanism Android uses to group SMS and MMS messages together. I though that it would do this by giving them the same thread_id, but this apparently turns out to not be the case. In my test collection of messages, there are SMS and MMS messages from and to a particular other party, and while on the original system, they are properly grouped together, despite apparently having different thread_ids, when they are imported to a new system, they appear separately.

I have another idea or two that I'm going to experiment with, and I'll report any progress here.

[tmo1]

I finally have a solution that seems to fix this problem, at least in my testing. I ask anyone who has experienced the problem to please test this version of the app and report the results.

[iiPythonx]

From what I can tell, that seems to of fixed most of the issue although (at least on my system) some two-way conversations appear as a group chat with my phone number being treated as another member. I didn't change any JSON, this backup was taken from the F-Droid version and reimported with the apk you sent.

Regardless, that change made my messages app actually usable so thank you!

[tmo1]

From what I can tell, that seems to of fixed most of the issue

Good!

although (at least on my system) some two-way conversations appear as a group chat with my phone number being treated as another member.

Are the messages SMS or MMS? Is your phone number on the new phone the same as that of the old one?

Regardless, that change made my messages app actually usable so thank you!

You're welcome!

[Philippe3869]

Same conclusion as @iiPythonx: SMS and MMS from/to same contacts are now in a unique conversation, so it seems to be corrected. For conversations with several recipients (messages I sent to several recipients), my phone number and my name are in the list of participants. That is not the case on the source phone, nor for one-to-one conversation. Notes:

• I used data exported with the tool taken from F-Droid (1.5.1) as I supposed it is only an import issue
• from Android 9 to Android 11 (both with /e/ OS).
• destination phone has no SIM card, but contacts are the same as on source phone (I don't know if contacts are involved in importing messages and in this issue)

Thank you ! If you find the solution to the residual problem, that would be great! I am available to test again. Greetings

[iiPythonx]

Is your phone number on the new phone the same as that of the old one?

It is, although another thing I found is the moment someone from one of said groups sends a message android recreates the group with my "duplicate" removed from it. The group it creates has no message history, but at least android is trying to do something about it.

Like @Philippe3869 said:

my phone number and my name are in the list of participants. That is not the case on the source phone, nor for one-to-one conversation.

Seems like that's the only issue that still resides.

[tmo1]

For conversations with several recipients (messages I sent to several recipients), my phone number and my name are in the list of participants. That is not the case on the source phone, nor for one-to-one conversation.

destination phone has no SIM card, but contacts are the same as on source phone (I don't know if contacts are involved in importing messages and in this issue)

SMS I/E does not look at Contacts on import; I don't know what Android's Messaging app does.

If the destination phone does not have the your number associated with it, then it makes sense that Android would include your phone number in the message participant list, since it has no way of knowing that it shouldn't (as opposed to the original phone, where Android might know to exclude the the number associated with the phone from the list).

some two-way conversations appear as a group chat with my phone number being treated as another member.

Is your phone number on the new phone the same as that of the old one?

It is

In this case, I'm not sure why it's being treated differently upon import. I suppose it's only a minor annoyance, but I'll leave this issue open as long as it remains unresolved.

In any event, thanks for testing. I'm going to commit, merge, and release the fix soon.

[Philippe3869]

I have tested again with version 1.5.3 (13), import and export, this time with the same SIM card. I arrive at the same conclusion: all the conversations, between me and several people, and only in this direction (I have no group conversation) show my name as participant. I don't think I'm wrong in saying that the messages that constitute them are therefore MMS (because to several people). I also notice that another conversation, from me to a single recipient and consisting of a single MMS, also shows me as a participant. From the JSON data, it is possible to know who is the local number with "msg_box" (2 for sent messages, 1 for received), but I don't know if you can use this information when importing...

[farfromrefug]

@tmo1 I am facing this issue with latest 2.4.1 while importing on android 14 Not sure I understood the whole thread here but would you have any idea why

[tmo1]

@tmo1 I am facing this issue with latest 2.4.1 while importing on android 14 Not sure I understood the whole thread here but would you have any idea why

Please be more specific - what exactly are you seeing? Additionally, please provide a minimal reproducible example.

[farfromrefug]

@tmo1 sorry wasn't clear enough. I have the exact same issue as this original issue. MMS and SMS from the same contact appear separate. I am on Los 21 with the default message app . I used default settings to export the messages (and ibdo sée the MMS being exported). Any more info I can give you ?

[tmo1]

As I wrote above:

please provide a minimal reproducible example.

What do you mean by "MMS and SMS from the same contact" - outgoing, incoming, both, etc.?

[farfromrefug]

@tmo1 both incoming and out I think. Not easy to say for sure what the separation really is because I dont see the data behind each message (if MMS and SMS). What I can tell for sure is that all images are on one side with some texts (which I think were sent with the images MMS) and all other contact discussions appear in a separated thread with the same exact contact name (if I click on the thread list item contact avatar it links to the same contact).

[tmo1]

One more time:

please provide a minimal reproducible example.

Please try to track down at least one message from each of the separately displayed threads, locate their JSON, and post redacted version of it here.

[farfromrefug]

@tmo1 tried to get a small sample as explained in the doc but was not working with my setup, had to go over 100 to get some that fails. But then i changed the import settings and it made it work! I disabled message and call log entry deduplication and the import worked correctly. So i guess for me it is fixed, but definitely still an issue somewhere. If i manage to reproduce it with a small sample i will share it here