search

4IceG / luci-app-sms-tool-js

Simple user interface to handle SMS / USSD / AT commands via sms_tool application. (LuCI JS) | OpenWrt >= 21.02
GNU General Public License v3.0
31 stars 14 forks source link

no SMS Messages if some of them contain From: field with special symbols #9

Closed bam80 closed 9 months ago

bam80 commented 1 year ago

After installing the new version, tinkering with SIM selection and rebooting the router, can't see my massages any more:

image

Powered by LuCI openwrt-23.05 branch (git-23.118.79121-6fb185f) / OpenWrt 23.05.0-rc2 (r23228-cd17d8df2a)

How could I diagnose this?

bam80 commented 1 year ago

The messages seem are receiving because I see increasing inbox counter, just don't see the messages themselves.

4IceG commented 1 year ago

There is probably some message that breaks the json structure. Show the result of the command:

sms_tool -d /dev/ttyUSB2 recv

Please note that the sms-tool package does not support all available types of messages.

bam80 commented 1 year ago

Hi @4IceG, thanks for the tip.

I made a full dump of the messages, but they contain sensitive information so I can't put if here.

It was a lot of messages received since I switched the SIM, so I started to delete one by one from the end with sms_tool, and refresh the page with the messages.

The problematic massage was this:

MSG: 42
From: DITMOS
Date/Time: 08/28/23 18:06:35
Reference number: 10
SMS segment 3 of 6
участие в электронном голосовании и розыгрыше призов можно с 8 по 1

After I deleted it, all the messages show up in your app.

The same message in RAW format:

MSG: 42
07919762020041F7600DD0C42435D27C4E010008328082816053218C0500030A06030443044704300441044204380435002004320020044D043B0435043A04420440043E043D043D043E043C00200433043E043B043E0441043E04320430043D043804380020043800200440043E0437044B04330440044B044804350020043F044004380437043E04320020043C043E0436043D043E00200441002000380020043F043E00200031

Can we make it work for the future?

4IceG commented 1 year ago

Hi @bam80 , Cezary is the maintainer of the sms-tool package and he will probably need the entire content of the problematic SMS sent to an e-mail.

@obsy Cezary co o tym myslisz?

obsy commented 1 year ago

But - what kind of problem? Looks like normal message.

obsy commented 1 year ago

@4IceG: wygląda na normalną wiadomość, bez znaków specjalnych czy innych takich.

bam80 commented 1 year ago

Any way I could reproduce the problem, as long as I have all the dump?

bam80 commented 1 year ago

Does the RAW contain my phone number? If no, I could wipe only the messages with sensitive data and provide full the dump.

obsy commented 1 year ago

Show me:

/usr/bin/sms_tool -s SM -d /dev/ttyUSB2 -f  "%Y-%m-%d %H:%M " -j recv

Change SM and /dev/ttyUSB2 as you have.

No, only sender number.

bam80 commented 1 year ago

@obsy The message actually contains some not-plain symbols, see ^Q in From: field: 

From: DIT^QMOS

Here is the hex dump:


00001e80  d0 b0 d1 82 d0 bd d0 be  2e 0a 0a 4d 53 47 3a 20  |...........MSG: |
00001e90  34 32 0a 46 72 6f 6d 3a  20 44 49 54 11 4d 4f 53  |42.From: DIT.MOS|
00001ea0  0a 44 61 74 65 2f 54 69  6d 65 3a 20 30 38 2f 32  |.Date/Time: 08/2|
00001eb0  38 2f 32 33 20 31 38 3a  30 36 3a 33 35 0a 52 65  |8/23 18:06:35.Re|
00001ec0  66 65 72 65 6e 63 65 20  6e 75 6d 62 65 72 3a 20  |ference number: |
00001ed0  31 30 0a 53 4d 53 20 73  65 67 6d 65 6e 74 20 33  |10.SMS segment 3|
00001ee0  20 6f 66 20 36 0a d1 83  d1 87 d0 b0 d1 81 d1 82  | of 6...........|
00001ef0  d0 b8 d0 b5 20 d0 b2 20  d1 8d d0 bb d0 b5 d0 ba  |.... .. ........|
00001f00  d1 82 d1 80 d0 be d0 bd  d0 bd d0 be d0 bc 20 d0  |.............. .|
00001f10  b3 d0 be d0 bb d0 be d1  81 d0 be d0 b2 d0 b0 d0  |................|
00001f20  bd d0 b8 d0 b8 20 d0 b8  20 d1 80 d0 be d0 b7 d1  |..... .. .......|
00001f30  8b d0 b3 d1 80 d1 8b d1  88 d0 b5 20 d0 bf d1 80  |........... ....|
00001f40  d0 b8 d0 b7 d0 be d0 b2  20 d0 bc d0 be d0 b6 d0  |........ .......|
00001f50  bd d0 be 20 d1 81 20 38  20 d0 bf d0 be 20 31 0a  |... .. 8 .... 1.|
00001f60  0a 4d 53 47 3a 20 34 33  0a 46 72 6f 6d 3a 20 44  |.MSG: 43.From: D|
obsy commented 1 year ago

Show us json output. Probably the json is badly formatted - sender field.

bam80 commented 1 year ago

@obsy as I said, I already deleted the badly formed message. How could I reproduce the problem now?

obsy commented 1 year ago

Can you get a similar message again? Or just wait until sms are not visible again and then show the result in json.

bam80 commented 1 year ago

It was advertisement SMS and I see this problem for the first time. Cant take a while to show up again.

obsy commented 1 year ago

ASCII char 11, vertical tab. How it should by displayed on phone? Like'

DIT
   MOS

?

bam80 commented 1 year ago

Show me:

/usr/bin/sms_tool -s SM -d /dev/ttyUSB2 -f  "%Y-%m-%d %H:%M " -j recv

Change SM and /dev/ttyUSB2 as you have.

When I use -s SM I see only this until I don't refresh the page with the messages again:

# /usr/bin/sms_tool -s SM -d /dev/ttyUSB2 -f  "%Y-%m-%d %H:%M " -j recv
{"msg":[]}

Without -s I see the output then, is the -s necessary and what are the variants?

obsy commented 1 year ago

@4IceG: o ile treść w jsonie jest eskejpowana i znaki poza zakresu są w utf'ie, o tyle sender na chwilę obecną nie. Jeżeli to wynika z tych dodatkowych znaków w senderze to trzeba pomyśleć co z tym fantem zrobić z ogóle - wycinać to czy coś innego. Bo nie pokażesz pewnie kilku liniowego sendera czy jak w/w przykładzie znaku z kodem 11 ASCII.

obsy commented 1 year ago

Without -s I see the output then, is the -s necessary and what are the variants?

-s SM -s ME -s TE

depending on modem and sim card.

4IceG commented 1 year ago

@obsy , rozumiem Cezary. To wiele wyjasnia.

Ciekawe jak to rozwiazali dedykowani producenci fw, u Nas ciezko o taki nietypowy sms, więc tego nawet nie widzimy.

bam80 commented 1 year ago

-s SM -s ME -s TE

@obsy -s SM|ME|TE could probably be in the help output of the app maybe?

bam80 commented 1 year ago

ASCII char 11, vertical tab. How it should by displayed on phone?

I found an article about these messages, apparently they look like: image https://devicess.ru/prishla-sms-ot-dit_moslog-ili-dit_mos-eto-chto-znachit/

obsy commented 1 year ago

Underline.

bam80 commented 1 year ago

I found a way to reproduce the problem, just need to try to login to https://www.mos.ru/ and they send a SMS with code:

$ ssh root@_gateway sms_tool -d /dev/ttyUSB2 -f '"%Y-%m-%d %H:%M"' recv|hexdump -C
...
00001e60  0a 4d 53 47 3a 20 34 32  0a 46 72 6f 6d 3a 20 44  |.MSG: 42.From: D|
00001e70  49 54 11 4d 6f 73 4c 6f  67 0a 44 61 74 65 2f 54  |IT.MosLog.Date/T|
00001e80  69 6d 65 3a 20 32 30 32  33 2d 30 38 2d 33 30 20  |ime: 2023-08-30 |
00001e90  31 37 3a 35 34 0a d0 98  d1 81 d0 bf d0 be d0 bb  |17:54...........|
00001ea0  d1 8c d0 b7 d1 83 d0 b9  d1 82 d0 b5 20 d0 ba d0  |............ ...|
00001eb0  be d0 b4 20 d0 bd d0 b0  20 6c 6f 67 69 6e 2e 6d  |... .... login.m|
00001ec0  6f 73 2e 72 75 2c 20 d0  bd d0 b5 20 d0 bf d0 b5  |os.ru, .... ....|
00001ed0  d1 80 d0 b5 d0 b4 d0 b0  d0 b2 d0 b0 d0 b9 d1 82  |................|
00001ee0  d0 b5 20 d0 bd d0 b8 d0  ba d0 be d0 bc d1 83 3a  |.. ............:|

Here is how the json output looks like:

$ ssh root@_gateway sms_tool -jd /dev/ttyUSB2 -f '"%Y-%m-%d %H:%M"' recv
...
{"index":42,"sender":"DITMosLog","timestamp":"2023-08-30 17:54","content":"\u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0439\u0442\u0435 \u043a\u043e\u0434 \u043d\u0430 login.mos.ru, \u043d\u0435 \u043f\u0435\u0440\u0435\u0434\u0430\u0432\u0430\u0439\u0442\u0435 \u043d\u0438\u043a\u043e\u043c\u0443: 111111"}]}

Show us json output. Probably the json is badly formatted - sender field.

@obsy the sender: field looks OK?

obsy commented 1 year ago

Yes.

Does luci display it or not?

bam80 commented 1 year ago

Oh, no, json actually contains the same badness with ASCII 11:


$ ssh root@_gateway sms_tool -jd /dev/ttyUSB2 -f '"%Y-%m-%d %H:%M"' recv|hexdump -C
...
00003cc0  2e 22 7d 2c 7b 22 69 6e  64 65 78 22 3a 34 32 2c  |."},{"index":42,|
00003cd0  22 73 65 6e 64 65 72 22  3a 22 44 49 54 11 4d 6f  |"sender":"DIT.Mo|
00003ce0  73 4c 6f 67 22 2c 22 74  69 6d 65 73 74 61 6d 70  |sLog","timestamp|
00003cf0  22 3a 22 32 30 32 33 2d  30 38 2d 33 30 20 31 37  |":"2023-08-30 17|
00003d00  3a 35 34 22 2c 22 63 6f  6e 74 65 6e 74 22 3a 22  |:54","content":"|
00003d10  5c 75 30 34 31 38 5c 75  30 34 34 31 5c 75 30 34  |\u0418\u0441\u04|
00003d20  33 66 5c 75 30 34 33 65  5c 75 30 34 33 62 5c 75  |3f\u043e\u043b\u|
00003d30  30 34 34 63 5c 75 30 34  33 37 5c 75 30 34 34 33  |044c\u0437\u0443|
00003d40  5c 75 30 34 33 39 5c 75  30 34 34 32 5c 75 30 34  |\u0439\u0442\u04|
00003d50  33 35 20 5c 75 30 34 33  61 5c 75 30 34 33 65 5c  |35 \u043a\u043e\|
00003d60  75 30 34 33 34 20 5c 75  30 34 33 64 5c 75 30 34  |u0434 \u043d\u04|
00003d70  33 30 20 6c 6f 67 69 6e  2e 6d 6f 73 2e 72 75 2c  |30 login.mos.ru,|
00003d80  20 5c 75 30 34 33 64 5c  75 30 34 33 35 20 5c 75  | \u043d\u0435 \u|
00003d90  30 34 33 66 5c 75 30 34  33 35 5c 75 30 34 34 30  |043f\u0435\u0440|
00003da0  5c 75 30 34 33 35 5c 75  30 34 33 34 5c 75 30 34  |\u0435\u0434\u04|
00003db0  33 30 5c 75 30 34 33 32  5c 75 30 34 33 30 5c 75  |30\u0432\u0430\u|
00003dc0  30 34 33 39 5c 75 30 34  34 32 5c 75 30 34 33 35  |0439\u0442\u0435|
00003dd0  20 5c 75 30 34 33 64 5c  75 30 34 33 38 5c 75 30  | \u043d\u0438\u0|
00003de0  34 33 61 5c 75 30 34 33  65 5c 75 30 34 33 63 5c  |43a\u043e\u043c\|
bam80 commented 1 year ago

Does luci display it or not?

No. But see my message above.

bam80 commented 1 year ago

So is it a problem with sms_tool or luci-app-sms-tool-js?

obsy commented 1 year ago

Ideologically - sms_tool (badly formatted json). But @4IceG - can you test does luci process such json or does it return an error?

4IceG commented 1 year ago

Ideologically - sms_tool (badly formatted json). But @4IceG - can you test does luci process such json or does it return an error?

I'm trying with this message, but LuCi is not showing anything at the moment. I'll try again.

bam80 commented 1 year ago

@4IceG please note I cut off a tail of the message with code in my dump, it's meaningless anyway

obsy commented 1 year ago 
44 49 54 11 4D 4F 53 
From:DIT_MOS
Date/Time:08/28/23 20:06:35
Reference number: 10
SMS segment 3 of 6
участие в электронном голосовании и розыгрыше призов можно с 8 по 1

I need to read the specs to see if that's how it's supposed to work.

obsy commented 1 year ago

@bam80 What router are you using sms-tool on?

bam80 commented 1 year ago

@obsy

Model | Kroks KNdRt31R19 Firmware Version | OpenWrt 23.05.0-rc2 r23228-cd17d8df2a / LuCI openwrt-23.05 branch git-23.118.79121-6fb185f

Modem: Quectel EP06-E

obsy commented 1 year ago

Mipsel. Try this version: https://dl.eko.one.pl/test/sms-tool_2022-03-21-f07699ab-1_mipsel_24kc.ipk

(uninstall old, install new, or unpack and replace binary in /usr/bin)

obsy commented 1 year ago

@bam80 : have you tested new sms_tool?

bam80 commented 1 year ago

I will soon

bam80 commented 1 year ago

Try this version: https://dl.eko.one.pl/test/sms-tool_2022-03-21-f07699ab-1_mipsel_24kc.ipk

@obsy Hey hey, sorry for a long wait, I was really busy with other things. But eventually I was in need to read my SMS and faced this problem again so I decided it's time now)

Thıs version solved a problem, could you publish the changes and make a bugfix release please? I would like to see it in openwrt packages feed also.

Thank you very much both of you!

obsy commented 1 year ago

Updated. You can do PR in package repo.

bam80 commented 1 year ago

The fix itself: https://github.com/obsy/sms_tool/commit/1b6ca03284fd65db8799dbf7c6224210093786b0

packages PR: https://github.com/openwrt/packages/pull/22204

bam80 commented 1 year ago

Could we please close it when the fix land in the owrt packages repo instead? Obviously, most of the users come from pre-built owrt and don't build stuff themselves..

bam80 commented 1 year ago

Thank you. I'll chase to fix it on owrt package repo side when I have chance. Will keep you informed here.

helllamer commented 9 months ago

packages PR: openwrt/packages#22204

@bam80 Looks like it is not merged since 25.09.2023, MR still in WiP status. Do you need any help?

bam80 commented 9 months ago

@helllamer it's just rebasing stuff on my side, the patch must be trivial - I would appreciate if somebody finish it. Let me know if you need me for that, thanks.