Closed Thieb67k closed 2 years ago
Was it longer than 160 chars?
Nope, in both cases. And everyday I reboot the phone hence that sounds like recurrent, or random. Plus I do have the end of the text. The beginning is missing and I would expect the opposite if the text is too long... Is there a link with the issue #976 ? Because in that case, I cannot send/receive a MMS and even the associated text is not send/received.
Were they simply out of order multiple SMS messages? When an SMS is > 160 characters, it is split into multiple messages. I have seen these arrive and/or display out of order before.
The point is I do not have multiple messages. I have only one, truncated. Not only one word but a significant part. By the way, even if that is not systematic, that happens only with my wife, which is seriously problematic...
My wife has two problems that are occurring frequently:
@ntorquet still having that issue ?
@ntorquet still having that issue ?
Yes! It is very uncool because you can't evn know that someone send you a message. And the case of truncated messages is... very strange
yes very strange, never experienced that. on OTA 11 stable ?
Yes.
What can be done, is to check if message are effectively truncated in history database, you can retrieve it there: /home/phablet/.local/share/history-service/history.sqlite
and with a sqlite client (e.g DB Browser ) see messages in "textevent" table.
@lduboeuf Thanks for your help. I will take a look a come back to you.
@lduboeuf I had a look in the history database. The text messages are stored exactly as they appear on the screen. They are truncated and mixed up with old messages from the same sender (only from Orange/Josh users as far as I know).
ok, just to know, do you have lots of messages stored ?. I think i saw a comment in code about message overload.
Right now, 11861 messages stored :o)
Ok, would be interesting to see if this can be the culprit. What can be done: backup of history db, remove it and see if it happen again. And tell us if you still have the pb (or not).
My wife removed a lot of sms. But, she still doesn't receive some sms sometime. Very annoying because if no body tell you that he send a message to you, you, you will never know... It happens to me also sometime.
On Nexus 5, and devel channel (20191218) I've got truncated message as well. Affected message are mms as i can see from text_events table (/org/ofono/mms/208150116711414/2b4eaaff472848ef67964b89e41754ac) Into the message column I've got NULL
i don't know how are handled mms for now. Do you have this behavior by removing the database and started again with a clean db ? ( don't forget backup )
Update: That issue does not happen every times even though I feel it is getting worse (most frequent, begining and end of messages are truncated). The text contains some usual french characters that are not problematic with others messages. Sending the exact same message does not help. I even noticed that one or two messages are missing, yet the sender's messages notifications show that is allright... DB browser does not show the complete message. That happens on mako but if someone has a spare compatible device... ;)
Hello, Unfortunately, that still happens with OTA-17. Is there any news for this issue?
@Thieb67k do you confirmed that your message that is stored in the db (~/.local/share/history-service/history.sqlite) table text_event is the same as it is displayed ? Do you know if it only happens from the same contact, or maybe same provider ? If your contact re-send it, does it display the same ?
If you can reproduce, before the contact send you the message, you can monitor the sms recevied with that:
dbus-monitor --system "type='signal',sender='org.ofono',interface='org.ofono.MessageManager'"
You will have something a bit noisy but you should read the content ( starts after "string' ): e.g:
signal time=1621517592.684904 sender=:1.25 -> destination=(null destination) serial=3892 path=/ril_0; interface=org.ofono.MessageManager; member=IncomingMessage
string "Du 07/05 au 07/06 votre conso hors forfait est de 0.00€.
Voix : 00:03:57
SMS : 14
Données : 301.33 Mo
Retrouvez plus d'infos sur http://mobile.free.fr/moncompte/
Identifiant client: XXXX"
array [
dict entry(
string "LocalSentTime"
variant string "2021-05-20T15:33:11+0200"
)
dict entry(
string "SentTime"
variant string "2021-05-20T15:33:11+0200"
)
dict entry(
string "Sender"
variant string "555"
)
]
Hello, -Messages stored into the db are the same as they are displayed on the messaging App. -I know that happens with 2 contacts and their provider are differents. -If the contact resend the text, it looks identical.
Because I know absolutely nothing about IT programming, should I paste the command (dbus-monitor --system "type='signal',sender='org.ofono',interface='org.ofono.MessageManager'") as is into the terminal? Thanks
Because I know absolutely nothing about IT programming, should I paste the command (dbus-monitor --system "type='signal',sender='org.ofono',interface='org.ofono.MessageManager'") as is into the terminal? Thanks
Yes, connect with adb or ssh to your device and paste that command, then press enter. You should see the incoming sms and see if it is truncated as it is displayed in messaging-app
Well, I am not comfortable with those tools (Terminal, adb, ssh...). Anyway, I managed to get that screenshot. "Unable", "error"... I need to spend more time on using that I guess...
@Thieb67k yes, noticed that when doing it from the terminal, it won't work.
If you can connect your device to a PC, then do adb shell
on your PC and then the paste command above
Ok, Thanks! That works. I got a text from my provider and that looks similar to yours. Next step is to do the same with a problematic text.
Ok, I have checked the command (dbus-monitor --system "type='signal',sender='org.ofono',interface='org.ofono.MessageManager'"). I see the incoming sms and each time it appears like as it is displays in the Messaging-App: full or truncated.
When I have the "Oups" notification because I cannot see the message (here, a sms, not a MMS) into the Messaging-App, the Terminal show me that text: The difference starts at line5 "member=MessageAdded object path "ril/0/message" instead of "member=IncomingMessage..." Then line18 ("member=MessageRemoved..."). I have not deleted anything.
I hope that helps.
humm, so if you said that they appear like it is in Messaging-app, it means that ofono component have the issue. Thanks for reporting
@Thieb67k or anyone willing to help testing. I can't provide any guarantee but i want to try things at lower level. But i need a tester in devel channel and that can reproduce the issue
Thanks for your help. I hope that story will conclude with an happy end :) I will see what I can do. Thanks again.
@Thieb67k are you on Telegram ? I would like to first try some stuffs locally If so please pm me on Telegram or Forum ( lduboeuf )
so as @dobey says, the message is not truncated but out of order ( at least for what you've sent me @Thieb67k )
So we can eliminate the modem, i'm looking into Ofono, that should be there but i'm a noob in clang, sms spec, glib, etc... Probably a long journey for me. I'm trying to look around if no fix has been already done upstream
oh not fixed in sailfish too https://together.jolla.com/question/161920/bug-sms-are-sentreceived-in-bad-order/ could be related: https://issuetracker.google.com/issues/36944392#c5
@lduboeuf well the last comment there is from 2017, so maybe it was fixed silently. Such an old report would need reconfirmation with a current version before we make this assumption. You could comment there asking for a state ;)
I'm trying to look around if no fix has been already done upstream
Well, nothing exist to fix that afaik, looked in sailfish repo and usptream ofono. I've tried to dig the issue, but it is really hard to get into it. Could be an encoding issue, bad character somewhere, some buffer not freed correctly... I admit i will not be able to go further, sorry.
As the issue is on non UT sender phone, they seems to not respect the sms protocol correctly, so not a UT Ofono issue i would say. Or we could make a hack, but honestly it can takes days for me at least to figure out what is the issue and more importantly without breaking things. For users who face that issue, i invite you to ask users to update their phone if this possible.
cheers
OK, maybe i've found something, i need to give some few hours more... :)
So i've tried to put logs everywhere, the sms chunks are received in proper sequence but for some reason, they are mixed. I can't go further sorry. Hopefully someone with better low level ubderstanding can spot something.
Can you point me to some code? I can take a look. Sometimes thats a minor thing, easy to fix, but with great consequences.
@Flohack74 So here are some debug logs, but you won't find all them in the code: The use case here is a sms that is splitted in 2 parts, received in the proper order 1,2 , but the output becomes 2, 1
phablet ofonod[2005]: drivers/rilmodem/sms.c:ril_sms_notify() req: 1003; data_len: 676
May 29 17:07:44 ubuntu-phablet ofonod[2005]: [0,UNSOL]< UNSOL_RESPONSE_NEW_SMS {XXXX}
May 29 17:07:44 ubuntu-phablet ofonod[2005]: drivers/rilmodem/sms.c:ril_sms_notify() smsc_len is 8
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:ofono_sms_deliver_notify() len 167 tpdu len 159
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:handle_deliver()
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:handle_deliver() ref: 141 max: 2 seq: 1
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_assembly_add_fragment_backup() 1 inserted at position 0, bit 2, num_fragments 1
May 29 17:07:44 ubuntu-phablet ofonod[2005]: [0,0176]> RIL_REQUEST_SMS_ACKNOWLEDGE (1,0)
May 29 17:07:44 ubuntu-phablet ofonod[2005]: drivers/rilmodem/sms.c:ril_sms_notify() req: 1003; data_len: 180
May 29 17:07:44 ubuntu-phablet ofonod[2005]: [0,UNSOL]< UNSOL_RESPONSE_NEW_SMS {XXXX}
May 29 17:07:44 ubuntu-phablet ofonod[2005]: drivers/rilmodem/sms.c:ril_sms_notify() smsc_len is 8
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:ofono_sms_deliver_notify() len 43 tpdu len 35
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:handle_deliver()
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:handle_deliver() ref: 141 max: 2 seq: 2
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_assembly_add_fragment_backup() 2 inserted at position 1, bit 4, num_fragments 2
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:sms_dispatch()
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:sms_dispatch() dst -1 src -1
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:sms_dispatch() dst -1 src -1
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_decode_text() Guess size 160
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_decode_text() decode_text charset other than 7bit
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_decode_text() decode_text charset 7bit, max chars: 11
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_decode_text() converted after: emi épaisse
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_decode_text() utf16
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_decode_text() converted after: emi épaisseSteack burger ( on a du pain ) , bleu ou roquefort, crème fraîche s
so here: https://github.com/ubports/ofono/blob/xenial/src/smsutil.c#L2214 the GSList *sms_list
is already out of order, was wondering if the for loop guarantee the order.
this list is filled here https://github.com/ubports/ofono/blob/xenial/src/smsutil.c#L2651
I've used that tool to make sure sms PDU was received ok: https://www.diafaan.com/sms-tutorials/gsm-modem-tutorial/online-sms-pdu-decoder/
@Thieb67k Would be interesting to see if it occurs on a Device Android >=9 too or Pinephone as it doesn't use the same ofono.
Humm interesting I totally forgot about this. Yes the for loop guarantees the order, thats a usual iterator, and there is nothing that can manipulate those positions. So I assume the insert is done with the wrong positions. Also, if you
Probably that decoding has a bug:
it could, idk :) , i've search for any updates in ofono uptream and mer branch and haven't found anything related to that kind of issue
well the position seems to be fine because the sequence is : First chunk: seq 1 inserted at position 0, bit 2, num_fragments 1 Second Chunk: seq 2 inserted at position 1, bit 4, num_fragments 2
I can't find where i've put the logs before but it was nearby that https://github.com/ubports/ofono/blob/xenial/src/smsutil.c#L2651-L2654
Do we have test data that we can capture to manually check whats going on? Also, I think multipart SMS was always fine for me, maybe its happening only for some providers, so it will be hard to debug.
Yes we have to found someone that send bad sms. I personally never had that. I used to test it a bit by receiving sms from the suspected phone directly from @Thieb67k
I have the raw sms data received (PDU) locally here, but as it is sensitive, i did not share it here.
I don't see updates here too: https://together.jolla.com/question/161920/bug-sms-are-sentreceived-in-bad-order/ and don't know where to find the bug tracker for that issue
Ok if its sensitive can you just replace the message data by garbage strings?
like the one above: received in proper sequence 1,2 and decoded has 2,1
May 29 17:07:44 ubuntu-phablet ofonod[2005]: drivers/rilmodem/sms.c:ril_sms_notify() req: 1003; data_len: 676
May 29 17:07:44 ubuntu-phablet ofonod[2005]: [0,UNSOL]< UNSOL_RESPONSE_NEW_SMS {##PDU SEQUENCE 1##}
May 29 17:07:44 ubuntu-phablet ofonod[2005]: drivers/rilmodem/sms.c:ril_sms_notify() smsc_len is 8
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:ofono_sms_deliver_notify() len 167 tpdu len 159
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:handle_deliver()
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:handle_deliver() ref: 141 max: 2 seq: 1
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_assembly_add_fragment_backup() 1 inserted at position 0, bit 2, num_fragments 1
May 29 17:07:44 ubuntu-phablet ofonod[2005]: [0,0176]> RIL_REQUEST_SMS_ACKNOWLEDGE (1,0)
May 29 17:07:44 ubuntu-phablet ofonod[2005]: drivers/rilmodem/sms.c:ril_sms_notify() req: 1003; data_len: 180
May 29 17:07:44 ubuntu-phablet ofonod[2005]: [0,UNSOL]< UNSOL_RESPONSE_NEW_SMS {##PDU SEQUENCE 2##}
May 29 17:07:44 ubuntu-phablet ofonod[2005]: drivers/rilmodem/sms.c:ril_sms_notify() smsc_len is 8
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:ofono_sms_deliver_notify() len 43 tpdu len 35
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:handle_deliver()
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:handle_deliver() ref: 141 max: 2 seq: 2
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_assembly_add_fragment_backup() 2 inserted at position 1, bit 4, num_fragments 2
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:sms_dispatch()
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:sms_dispatch() dst -1 src -1
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:sms_dispatch() dst -1 src -1
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_decode_text() Guess size 160
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_decode_text() decode_text charset other than 7bit
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_decode_text() decode_text charset 7bit, max chars: 11
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_decode_text() converted after: emi épaisse
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_decode_text() utf16
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/smsutil.c:sms_decode_text() converted after: emi épaisseSteack burger ( on a du pain ) , bleu ou roquefort, crème fraîche s
May 29 17:07:44 ubuntu-phablet ofonod[2005]: src/sms.c:sms_dispatch() dispatch_text_message
As you notice "emi épaisse" should be the second part, but is inserted at the beginning
humm interesting, when using that tool: https://www.diafaan.com/sms-tutorials/gsm-modem-tutorial/online-sms-pdu-decoder/ if you check the PDUs, the first one have: Data coding: Unicode the second one: Data coding: | SMS Default Alphabet
see comment about charset here : https://github.com/ubports/ofono/blob/xenial/src/sms.c#L1308
Description of the feature The 1st sms of the day, for one contact, is truncated: I got only the end of the sms. That happened today and yesterday with the same contact. Following sms are complete.
Expected behavior To get a complete text...
Actual behavior incomplete sms even if "Status" in properties is "received".