search

mgaman / PDUlib

Encode/Decode PDU strings for use with most GSM modems. Both 7 bit and 16 bit alphabets are supported.
GNU Lesser General Public License v2.1
22 stars 10 forks source link

PDU decoding doesnt work reliable #11

Closed EliasA97 closed 2 years ago

EliasA97 commented 2 years ago

Hello, when I decode an sms with greek letters, it doesn't decode Sigma (Σ) or other characters. It just showing boxes instead.

mgaman commented 2 years ago

Did you test on a phone or on your computer? Send me a sample message that failed.

On Sun, Mar 6, 2022 at 7:02 AM EliasA97 @.***> wrote:

Hello, when I decode an sms with greek letters, it doesn't decode Sigma (Σ) or other characters. It just showing boxes instead.

— Reply to this email directly, view it on GitHub https://github.com/mgaman/PDUlib/issues/11, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA3KSTQGP7M6DGBXKK6SLHTU6Q343ANCNFSM5QAT4INQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

EliasA97 commented 2 years ago

I used it on an arduino mega with a SIM800 module. So. I got 4 sms's from my carrier that inform me for my remaining calling minutes, MB, sms etc. I got the PDU message from the module and I can decode it with an online decoder just fine. When I decode it with your library, it decodes only the characters that are same with the Lattin alphabet. I will give you the PDU of the 4 messages that I have in order for you to test them and I'll send the decoded message that I get with your library

char *inpdu[] = {"06910379010023040DD05774983EAFC20100002230603023928046456C318901D960B013C8027D62404FCA13447D3A41CB27A6F904519F59D01519A49EA6A02A146479664114E4E4997582A8482708167BC168A0984C078301", "06910379010023040DD05774983EAFC20100002230604054558046456C318901D960B013C8027D62404FCA13447D3A41CB27A6F904519F59D01519A49EA6A02A146479664114E4E4997582A8482708167BC168A0984C078301", "06910379010023040DD05774983EAFC20100002230608044508047456C31890161B2CE2725B90C8262B09BED04B2409F18D0931204518320487249CD064196671644414E9E5927888A748262B5176CF692C16432504CA683C100", "06910379010023040DD05774983EAFC2010000223060804401803C456C318901C564301868DA9C822CD0270654CC3893CB2008925C52B34190E5990551909367D609A2229DA058ED059B8164341D0C06", "06910379010023040DD05774983EAFC20100002230608044518049456C318901C570B51808528116834E5053089A2483200BE5899C2031481086FA04259DD4A2D459A4829A452C3409A226312098EC05A3BD6430990C04B3E96630"}; These are the sms's.

`Not using PM

+3097100032 WhatsUp@@@ò£¥ 22030603322908 EXEI? 600' ?PO? O?O TON KO?MO TOY WHAT'S UP ?OY ?H?OYN THN 01/04 12:00

+3097100032 WhatsUp@@@ò£¥ 22030604455508 EXEI? 600' ?PO? O?O TON KO?MO TOY WHAT'S UP ?OY ?H?OYN THN 01/04 12:00

+3097100032 WhatsUp@@@ò£¥ 22030608440508 EXEI? ?YNO?IKA 1076' ?PO? O?A TA ?IKTYA ?OY ?H?OYN THN 15/03/2022 12:00

+3097100032 WhatsUp@@@ò£¥ 22030608441008 EXEI? 1200 SMS ?PO? E?NIKA ?IKTYA ?OY ?H?OYN THN 15/03 24:00

+3097100032 WhatsUp@@@ò£¥ 22030608441508 EXEI? 1851 ??PEAN MB ?IA ??OH?H?H ?TO INTERNET MEXPI TI? 02/04/2022 06:30 ` This is my serial monitor

EliasA97 commented 2 years ago

The last message should be like this: image

mgaman commented 2 years ago

OK, let me look into it

On Sun, Mar 6, 2022 at 9:09 AM EliasA97 @.***> wrote:

The last message should be like this: [image: image] https://user-images.githubusercontent.com/56277356/156912959-ce79b720-0a06-40eb-aaf5-a3c3f55779a0.png

— Reply to this email directly, view it on GitHub https://github.com/mgaman/PDUlib/issues/11#issuecomment-1059908889, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA3KSTROK3ORFMKXEOIL253U6RK2ZANCNFSM5QAT4INQ . You are receiving this because you commented.Message ID: @.***>

mgaman commented 2 years ago

please send the pdu message as text, not a picture

On Sun, Mar 6, 2022 at 10:59 PM David Henry @.***> wrote:

OK, let me look into it

On Sun, Mar 6, 2022 at 9:09 AM EliasA97 @.***> wrote:

The last message should be like this: [image: image] https://user-images.githubusercontent.com/56277356/156912959-ce79b720-0a06-40eb-aaf5-a3c3f55779a0.png

— Reply to this email directly, view it on GitHub https://github.com/mgaman/PDUlib/issues/11#issuecomment-1059908889, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA3KSTROK3ORFMKXEOIL253U6RK2ZANCNFSM5QAT4INQ . You are receiving this because you commented.Message ID: @.***>

EliasA97 commented 2 years ago

I believe the message is sending as text, not image. Or do you mean the above text in photo?

mgaman commented 2 years ago

OK, I have a fix. There is a table to convert GSM 7 bit to Unicode 8 bit which ignores Greek as Greek has 16 bit Unicode values. I have made a patch which works OK. I am delaying updating Github as I do not yet have have a fix for encoding Greek. Right now I treat Greek as 16 bit Unicode instead of GSM 7 bit, it works but limits messages to 70 characters instead of 160. In the meantime you can patch your own sources

Add the following to pdulib.h

unsigned short lookupgreek7ToUnicode[] { 0x394, / 16 GREEK CAPITAL LETTER DELTA / 95, /* 17 LOW LINE / 0x3a6, / 18 GREEK CAPITAL LETTER PHI / 0x393, / 19 GREEK CAPITAL LETTER GAMMA / 0x39b, / 20 GREEK CAPITAL LETTER LAMBDA / 0x3a9, / 21 GREEK CAPITAL LETTER OMEGA / 0x3a0, / 22 GREEK CAPITAL LETTER PI / 0x3a8, / 23 GREEK CAPITAL LETTER PSI / 0x3a3, / 24 GREEK CAPITAL LETTER SIGMA / 0x398, / 25 GREEK CAPITAL LETTER THETA / 0x39e, / 26 GREEK CAPITAL LETTER XI */ };

Modify pdulib.cpp PDU::convert_7bit_to_ascii, line 324 replace w += buildUtf(C,&ascii[w]);

by if (C != NPC8) w += buildUtf(C, &ascii[w]); else { unsigned short S = lookup_greek7ToUnicode[a7bit[r]-16]; w += buildUtf(S, &ascii[w]); }

On Tue, Mar 8, 2022 at 1:34 PM EliasA97 @.***> wrote:

I believe the message is sending as text, not image. Or do you mean the above text in photo?

— Reply to this email directly, view it on GitHub https://github.com/mgaman/PDUlib/issues/11#issuecomment-1061684109, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA3KSTXSAZALOQQ3WZFKM3TU643KDANCNFSM5QAT4INQ . You are receiving this because you commented.Message ID: @.***>

EliasA97 commented 2 years ago

Thank you for the fix. I'll try it later and I'll inform you!!!

EliasA97 commented 2 years ago

Unfortunatelly I can't get it to work. Mabye I don't copy the lines correctly. Anyway, I'll wait for your updated library.

Thank's again!

mgaman commented 2 years ago

Check again, its a really simple fix. Does it all compile OK?

On Wed, Mar 9, 2022 at 1:23 AM EliasA97 @.***> wrote:

Unfortunatelly I can't get it to work. Mabye I don't copy the lines correctly. Anyway, I'll wait for your updated library.

Thank's again!

— Reply to this email directly, view it on GitHub https://github.com/mgaman/PDUlib/issues/11#issuecomment-1062366936, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA3KSTXRF3LEW33IXZCX2RLU67OO5ANCNFSM5QAT4INQ . You are receiving this because you commented.Message ID: @.***>

EliasA97 commented 2 years ago

Can you confirm that the copied lines are ok? image

image

image

I tried so many times and I still get compilation errors

mgaman commented 2 years ago

in pdulib.h change unsigned short lookup_greek7ToUnicode to const unsigned short lookup_greek7ToUnicode

On Wed, Mar 9, 2022 at 10:28 PM EliasA97 @.***> wrote:

Can you confirm that the copied lines are ok? [image: image] https://user-images.githubusercontent.com/56277356/157529920-047efc22-011e-4ec5-b9c5-5cdf7150b563.png

[image: image] https://user-images.githubusercontent.com/56277356/157530111-b6f6cbe3-75d7-4894-9dd7-09f746b7330e.png

`Arduino: 1.8.19 (Windows 10), Πλακέτα:"Arduino Uno"

C:\Program Files (x86)\Arduino\arduino-builder -dump-prefs -logger=machine -hardware C:\Program Files (x86)\Arduino\hardware -hardware C:\Users\anast\AppData\Local\Arduino15\packages -tools C:\Program Files (x86)\Arduino\tools-builder -tools C:\Program Files (x86)\Arduino\hardware\tools\avr -tools C:\Users\anast\AppData\Local\Arduino15\packages -built-in-libraries C:\Program Files (x86)\Arduino\libraries -libraries C:\Users\anast\OneDrive\Έγγραφα\Arduino\libraries -fqbn=arduino:avr:uno -vid-pid=1A86_7523 -ide-version=10819 -build-path C:\Users\anast\AppData\Local\Temp\arduino_build_441878 -warnings=none -build-cache C:\Users\anast\AppData\Local\Temp\arduino_cache_508896 -prefs=build.warn_data_percentage=75 -prefs=runtime.tools.avr-gcc.path=C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avr-gcc\7.3.0-atmel3.6.1-arduino7 -prefs=runtime.tools.avr-gcc-7.3.0-atmel3.6.1-arduino7.path=C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avr-gcc\7.3.0-atmel3.6.1-arduino7 -prefs=runtime.tools.arduinoOTA.path=C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\arduinoOTA\1.3.0 -prefs=runtime.tools.arduinoOTA-1.3.0.path=C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\arduinoOTA\1.3.0 -prefs=runtime.tools.avrdude.path=C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avrdude\6.3.0-arduino17 -prefs=runtime.tools.avrdude-6.3.0-arduino17.path=C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avrdude\6.3.0-arduino17 -verbose C:\Users\anast\AppData\Local\Temp\arduino_modified_sketch_272457\Decode.ino

C:\Program Files (x86)\Arduino\arduino-builder -compile -logger=machine -hardware C:\Program Files (x86)\Arduino\hardware -hardware C:\Users\anast\AppData\Local\Arduino15\packages -tools C:\Program Files (x86)\Arduino\tools-builder -tools C:\Program Files (x86)\Arduino\hardware\tools\avr -tools C:\Users\anast\AppData\Local\Arduino15\packages -built-in-libraries C:\Program Files (x86)\Arduino\libraries -libraries C:\Users\anast\OneDrive\Έγγραφα\Arduino\libraries -fqbn=arduino:avr:uno -vid-pid=1A86_7523 -ide-version=10819 -build-path C:\Users\anast\AppData\Local\Temp\arduino_build_441878 -warnings=none -build-cache C:\Users\anast\AppData\Local\Temp\arduino_cache_508896 -prefs=build.warn_data_percentage=75 -prefs=runtime.tools.avr-gcc.path=C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avr-gcc\7.3.0-atmel3.6.1-arduino7 -prefs=runtime.tools.avr-gcc-7.3.0-atmel3.6.1-arduino7.path=C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avr-gcc\7.3.0-atmel3.6.1-arduino7 -prefs=runtime.tools.arduinoOTA.path=C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\arduinoOTA\1.3.0 -prefs=runtime.tools.arduinoOTA-1.3.0.path=C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\arduinoOTA\1.3.0 -prefs=runtime.tools.avrdude.path=C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avrdude\6.3.0-arduino17 -prefs=runtime.tools.avrdude-6.3.0-arduino17.path=C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avrdude\6.3.0-arduino17 -verbose C:\Users\anast\AppData\Local\Temp\arduino_modified_sketch_272457\Decode.ino

Using board 'uno' from platform in folder: C:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5

Using core 'arduino' from platform in folder: C:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5

Detecting libraries used...

"C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avr-gcc\7.3.0-atmel3.6.1-arduino7/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -flto -w -x c++ -E -CC -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10819 -DARDUINO_AVR_UNO -DARDUINO_ARCH_AVR "-IC:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5\cores\arduino" "-IC:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5\variants\standard" "C:\Users\anast\AppData\Local\Temp\arduino_build_441878\sketch\Decode.ino.cpp" -o nul

Alternatives for pdulib.h: @.***

ResolveLibrary(pdulib.h)

-> candidates: @.***

"C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avr-gcc\7.3.0-atmel3.6.1-arduino7/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -flto -w -x c++ -E -CC -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10819 -DARDUINO_AVR_UNO -DARDUINO_ARCH_AVR "-IC:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5\cores\arduino" "-IC:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5\variants\standard" "-IC:\Users\anast\OneDrive\Έγγραφα\Arduino\libraries\PDUlib\src" "C:\Users\anast\AppData\Local\Temp\arduino_build_441878\sketch\Decode.ino.cpp" -o nul

"C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avr-gcc\7.3.0-atmel3.6.1-arduino7/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -flto -w -x c++ -E -CC -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10819 -DARDUINO_AVR_UNO -DARDUINO_ARCH_AVR "-IC:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5\cores\arduino" "-IC:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5\variants\standard" "-IC:\Users\anast\OneDrive\Έγγραφα\Arduino\libraries\PDUlib\src" "C:\Users\anast\OneDrive\Έγγραφα\Arduino\libraries\PDUlib\src\pdulib.cpp" -o nul

Generating function prototypes...

"C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avr-gcc\7.3.0-atmel3.6.1-arduino7/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -flto -w -x c++ -E -CC -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10819 -DARDUINO_AVR_UNO -DARDUINO_ARCH_AVR "-IC:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5\cores\arduino" "-IC:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5\variants\standard" "-IC:\Users\anast\OneDrive\Έγγραφα\Arduino\libraries\PDUlib\src" "C:\Users\anast\AppData\Local\Temp\arduino_build_441878\sketch\Decode.ino.cpp" -o "C:\Users\anast\AppData\Local\Temp\arduino_build_441878\preproc\ctags_target_for_gcc_minus_e.cpp"

"C:\Users\anast\AppData\Local\Arduino15\packages\builtin\tools\ctags\5.8-arduino11/ctags" -u --language-force=c++ -f - --c++-kinds=svpf --fields=KSTtzns --line-directives "C:\Users\anast\AppData\Local\Temp\arduino_build_441878\preproc\ctags_target_for_gcc_minus_e.cpp"

Μεταγλώττιση σχεδίου...

"C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avr-gcc\7.3.0-atmel3.6.1-arduino7/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -MMD -flto -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10819 -DARDUINO_AVR_UNO -DARDUINO_ARCH_AVR "-IC:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5\cores\arduino" "-IC:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5\variants\standard" "-IC:\Users\anast\OneDrive\Έγγραφα\Arduino\libraries\PDUlib\src" "C:\Users\anast\AppData\Local\Temp\arduino_build_441878\sketch\Decode.ino.cpp" -o "C:\Users\anast\AppData\Local\Temp\arduino_build_441878\sketch\Decode.ino.cpp.o"

Compiling libraries...

Compiling library "PDUlib"

"C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avr-gcc\7.3.0-atmel3.6.1-arduino7/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -MMD -flto -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10819 -DARDUINO_AVR_UNO -DARDUINO_ARCH_AVR "-IC:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5\cores\arduino" "-IC:\Users\anast\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.5\variants\standard" "-IC:\Users\anast\OneDrive\Έγγραφα\Arduino\libraries\PDUlib\src" "C:\Users\anast\OneDrive\Έγγραφα\Arduino\libraries\PDUlib\src\pdulib.cpp" -o "C:\Users\anast\AppData\Local\Temp\arduino_build_441878\libraries\PDUlib\pdulib.cpp.o"

Compiling core...

Using precompiled core: C:\Users\anast\AppData\Local\Temp\arduino_cache_508896\core\core_arduino_avr_uno_33479967ad576a9c7f34af6950090e43.a

Linking everything together...

"C:\Users\anast\AppData\Local\Arduino15\packages\arduino\tools\avr-gcc\7.3.0-atmel3.6.1-arduino7/bin/avr-gcc" -w -Os -g -flto -fuse-linker-plugin -Wl,--gc-sections -mmcu=atmega328p -o "C:\Users\anast\AppData\Local\Temp\arduino_build_441878/Decode.ino.elf" "C:\Users\anast\AppData\Local\Temp\arduino_build_441878\sketch\Decode.ino.cpp.o" "C:\Users\anast\AppData\Local\Temp\arduino_build_441878\libraries\PDUlib\pdulib.cpp.o" "C:\Users\anast\AppData\Local\Temp\arduino_build_441878/..\arduino_cache_508896\core\core_arduino_avr_uno_33479967ad576a9c7f34af6950090e43.a" "-LC:\Users\anast\AppData\Local\Temp\arduino_build_441878" -lm

C:\Users\anast\AppData\Local\Temp\arduino_build_441878\libraries\PDUlib\pdulib.cpp.o (symbol from plugin): In function `SPstart':

(.text+0x0): multiple definition of `lookup_greek7ToUnicode'

C:\Users\anast\AppData\Local\Temp\arduino_build_441878\sketch\Decode.ino.cpp.o (symbol from plugin):(.text+0x0): first defined here

collect2.exe: error: ld returned 1 exit status

Χρησιμοποιώντας την βιβλιοθήκη PDUlib στην έκδοση 0.4.7 στον φάκελο: C:\Users\anast\OneDrive\Έγγραφα\Arduino\libraries\PDUlib

exit status 1

Σφάλμα μεταγλώττισης για την πλακέτα Arduino Uno.

`

— Reply to this email directly, view it on GitHub https://github.com/mgaman/PDUlib/issues/11#issuecomment-1063334684, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA3KSTQC5OJPYHY6ASRUMITU7ECWPANCNFSM5QAT4INQ . You are receiving this because you commented.Message ID: @.***>

EliasA97 commented 2 years ago

Great. It works fine now. Thank's again!

mgaman commented 2 years ago

Fixed in V0.5.1