SMS send/receive in full available character set

[GoogleCodeExporter]

SMS should be sent with supporing all possible character set
Same for incoming SMS.

This assume send/receive with PDU and UCS2 (unicode) character set.

Original issue reported on code.google.com by bg_...@mail.ru on 30 Nov 2010 at 8:21

[GoogleCodeExporter]

Original comment by bg_...@mail.ru on 6 Dec 2010 at 12:36

• Changed state: Fixed

[GoogleCodeExporter]

k3520 and e1612 aren't sending correct latin caracter:

ñ, á, é   -> wrong character received in SMS  (using smsaspdu=yes or 
smsaspdu=no)

Also, note that e1612 sends different wrong character than k3520 ;(

Original comment by pag...@gmail.com on 6 Dec 2010 at 4:43

• Changed state: Started

[GoogleCodeExporter]

Thanks,  can i see PDUs from both send and receive side ?

Original comment by bg_...@mail.ru on 6 Dec 2010 at 6:52

[GoogleCodeExporter]

sent: 
001100099196899865F00008A9340048006F006C00610020007100750065002000740061006C0020
00650073007400FF00730020004500730070006100FF0061003F

received: Hola que tal estÿs Espaÿa?  (Hola que tal estás España?)

I used http://www.rednaxela.net/pdu.php to confirm original pdu is not 
working...

Original comment by pag...@gmail.com on 6 Dec 2010 at 7:33

[GoogleCodeExporter]

it's a bit strange. Two different systems are generating 2 different pdu (one 
is ok, and the other is wrong)

perhaps locales?

Original comment by pag...@gmail.com on 6 Dec 2010 at 8:00

[GoogleCodeExporter]

Required both sent and taken PDU.

We assume input is UTF-8 encoded, PDU build with Alpabet UCS-2
For recode used iconv(to="UCS-2BE" from="UTF-8");

Not sure used locale for this convertion or not.
More likely errors source: user input strictly in UTF-8.

Original comment by bg_...@mail.ru on 6 Dec 2010 at 8:50

[GoogleCodeExporter]

Linux debian-casa 2.6.32-5-686 #1 SMP Sat Oct 30 22:47:19 UTC 2010 i686 
GNU/Linux
Used k3520 and send to another k3520 in the same machine

sent pdu: 
001100099136442032F10008A9360048006F006C00610020007100750065002000740061006C0020
00650073007400FF00730020004500730070006100FF00610032003F

received pdu:
07914306070059F2040B914336442022F5000801216022013140360048006F006C00610020007100
750065002000740061006C002000650073007400FF00730020004500730070006100FF0061003200
3F

Original message: Hola que tal estás España2?

Original comment by pag...@gmail.com on 6 Dec 2010 at 9:16

[GoogleCodeExporter]

UDL 36
UD sent
0048006F006C00610020007100750065002000740061006C002000650073007400FF007300200045
00730070006100FF00610032003F

UDL 36
UD received
0048006F006C00610020007100750065002000740061006C002000650073007400FF007300200045
00730070006100FF00610032003F

is same. 

Error in UTF-8 -> UCS-2 -> UTF-8 transforms.
try 
$ echo "Hola que tal estás España2?" | iconv -f "UTF-8" -t "UCS-2BE" | iconv 
-f "UCS-2BE" -t "UTF-8"

Original comment by bg_...@mail.ru on 6 Dec 2010 at 10:57

[GoogleCodeExporter]

root@debian-casa:~# echo "Hola que tal estás España2?" | iconv -f "UTF-8" -t 
"UCS-2BE" | iconv -f "UCS-2BE" -t "UTF-8"
Hola que tal estás España2?

Original comment by pag...@gmail.com on 6 Dec 2010 at 11:24

[GoogleCodeExporter]

not understant yet why change symbols in chan_datacard :(

Original comment by bg_...@mail.ru on 6 Dec 2010 at 11:44

[GoogleCodeExporter]

Also one question, where you look at received SMS?

Original comment by bg_...@mail.ru on 6 Dec 2010 at 11:51

[GoogleCodeExporter]

I have more modems in other hardware and everything is ok ;)

Again, let's see what other people is reporting...

Original comment by pag...@gmail.com on 6 Dec 2010 at 11:52

[GoogleCodeExporter]

SMS decoded body passed to channel variable SMS
and body encoded in base64 to SMS_BASE64

Original comment by bg_...@mail.ru on 7 Dec 2010 at 12:01

[GoogleCodeExporter]

I enabled debug and get PDU information from console...

Original comment by pag...@gmail.com on 7 Dec 2010 at 12:04

[GoogleCodeExporter]

if PDUs for out/in such as you have written trouble not in modem.

Original comment by bg_...@mail.ru on 7 Dec 2010 at 12:04

[GoogleCodeExporter]

> Also one question, where you look at received SMS?
I say here about SMS text, not PDU.

Original comment by bg_...@mail.ru on 7 Dec 2010 at 12:05

[GoogleCodeExporter]

Received SMS is in my iphone.. I can receive SMS without problems...

Original comment by pag...@gmail.com on 7 Dec 2010 at 12:06

[GoogleCodeExporter]

I think may be message spoils on its way to you, because set to channel 
variable, read from channel variable, etc

Original comment by bg_...@mail.ru on 7 Dec 2010 at 12:07

[GoogleCodeExporter]

SMS text to send is the same: Hola que tal estás España2?. 

Hardware1 generates this PDU:
001100099196899865F00008A9360048006F006C00610020007100750065002000740061006C0020
00650073007400E100730020004500730070006100F100610032003F
Iphone shows:  Hola que tal estás España2?

Hardware 2 generates this PDU:
001100099196899865F00008A9360048006F006C00610020007100750065002000740061006C0020
00650073007400FF00730020004500730070006100FF00610032003F
Iphone shows:  Hola que tal estÿs Espaÿa2?

Also using k3520 on both systems.  It's a poltergeist XDD

Original comment by pag...@gmail.com on 7 Dec 2010 at 12:15

[GoogleCodeExporter]

Hardware not generate PDU, chan_datacard generate is and feeds to device, 
device update 1 fields and send to SC, and Service Center just convert PDU from 
SMS-SUBMIT to SMS-DELIVER by update headers not message body

Original comment by bg_...@mail.ru on 7 Dec 2010 at 12:27

[GoogleCodeExporter]

ok, may be i understand where is problem 

has only 2 difference 
... E1 ... F1 ...
... FF ... FF ...

looks like signed operation under unsigned data

Original comment by bg_...@mail.ru on 7 Dec 2010 at 12:30

[GoogleCodeExporter]

Sorry, I meant "chan_datacard in Hardware1"...  

The question is that both PDU are different before they are sent to SC... 
(obviously, chan_datacard is the same and modems are the same..)

Original comment by pag...@gmail.com on 7 Dec 2010 at 12:32

[GoogleCodeExporter]

Both devices on same host?

Original comment by bg_...@mail.ru on 7 Dec 2010 at 12:35

[GoogleCodeExporter]

no, different hosts (different hardware)

Original comment by pag...@gmail.com on 7 Dec 2010 at 12:37

[GoogleCodeExporter]

So, iconv() functions in this cases differ, on second host iconv() considers 
'á' and 'ñ' as invalid characters (not know how convert to UCS-2) and replace 
with -1 (0xFF)

Original comment by bg_...@mail.ru on 7 Dec 2010 at 12:45

[GoogleCodeExporter]

But, as I wrote before.. if a execute in the faulty system

root@debian-casa:~# echo "Hola que tal estás España2?" | iconv -f "UTF-8" -t 
"UCS-2BE" | iconv -f "UCS-2BE" -t "UTF-8"

I get the correct answer!!!

Hola que tal estás España2?

Original comment by pag...@gmail.com on 7 Dec 2010 at 12:50

[GoogleCodeExporter]

locales for asterisk and for root is same ?

Original comment by bg_...@mail.ru on 7 Dec 2010 at 12:54

[GoogleCodeExporter]

other case iconv linked agains other library with iconv instead of 
chan_datacard 
ldd iconv 
ldd chan_datacard.so

Original comment by bg_...@mail.ru on 7 Dec 2010 at 12:57

[GoogleCodeExporter]

There is no user asterisk. I run asterisk as root...

Original comment by pag...@gmail.com on 7 Dec 2010 at 12:57

[GoogleCodeExporter]

fixed in r87!!!  Now working well in both test systems.. ;)

Original comment by pag...@gmail.com on 7 Dec 2010 at 9:29

• Changed state: Fixed

[GoogleCodeExporter]

i don't  know how it happined, but i rewrite hex <-> binary converters for 
accelerating

Original comment by bg_...@mail.ru on 7 Dec 2010 at 12:03