SMS send don't work in 4G-mode

pizdjuk commented 2 years ago

But works if the modem put in 2G mode only (I didnt test other variants). In 4G mode mmcli returns "timeout was reached".

pizdjuk commented 2 years ago

Just flashed the 003.003 modem firmware. Nothing changed. The issue still present in both (002 and 003) variants.

Biktorgj commented 2 years ago

Did it work in stock? That seems to either work or not depending on the carrier.

There's a command to force the modem to use a specific method to send SMS, and that is AT+CGSMS Query the current configuration by sending AT+CGSMS?, then you can set it to whatever you need:

AT+CGSMS=0 -> Always use GPRS to send messages (probably needs the correct APN settings for it)
AT+CGSMS=1 -> Always use legacy mode (via SMSC) to send messages
AT+CGSMS=2 -> Use GPRS by default and fallback to legacy if it fails
AT+CGSMS=3 -> Use Legacy by default and fallback to GPRS

Still, some users do have problems sending messages even after changing this setting. I have not been able to replicate this though, so if it works please let me know

pizdjuk commented 2 years ago

tried all of the setting without success. Do you have an e-mail that I could attach all modem-logs (for security reasons I would not post it here)?

pothos commented 2 years ago

@pizdjuk It's a problem with the provider and the workaound is to disable IMS in eg25-manager which means that calls and SMS will use 2G instead of VoLTE (and during the SMS sending or call there data transfer will not use LTE speed). I do not recommend to do it because the call quality is worse. I would rather manually switch to 2G in case of sending an SMS or issuing an USSD code.

That's the change to the eg25-manager config to disable IMS:

sudo nano /usr/share/eg25-manager/pine64,pinephone-1.2.toml
# change the last part of '{ cmd = "QCFG", subcmd = "ims", expect = "1" },' to 'subcmd = "ims", value = "2"'
sudo systemctl reboot

You may also try to set value = "0" instead of value = "2", one of them worked for me, but I forgot which. In any case you can check if with mmcli -m any --command='AT+QCFG="ims"' which should return the matching value as first digit of the output.

pizdjuk commented 2 years ago

Yeah, man! Thanks! It worked. I with value="1"

The call-quality of 2G is completely ok for me :)

pizdjuk commented 2 years ago

Dont work anymore :/

pothos commented 2 years ago

Didn't check the docs but with value = 1 I don't think it will disable IMS, try 0 or 2 and reboot, I would say.

pizdjuk commented 2 years ago

With value=0 didn't work. With value=2 works. Thanks!!

I could sent about 12-15 sms without problems till now.

I close the issue for now.

pizdjuk commented 2 years ago

Something strange is going on. I rebooted and cannot send SMS again. The file didnt change

mpodshivalin commented 2 years ago

I've also had this problem.

Data&Calls work across all networks, but when I'm on 4G, I can't send and receive SMS. mmcli -m 0 --messaging-list-sms shows this:

    /org/freedesktop/ModemManager1/SMS/1 (unknown)
    /org/freedesktop/ModemManager1/SMS/0 (unknown)

Original firmware also has this exact problem
It doesn't appear to be an OS problem since it can be reproduced in Manjaro KDE and PostmarketOS Phosh
Doesn't matter if I have ADSP Version 01.003.01.003 or 01.002.01.002
If I have an incoming SMS in "unknown" status, it seems that there is no way I can read it so I lose all incoming SMS while on 4G. So switching to 2G to send an SMS isn't an option

I've commented out this line in /usr/share/eg25-manager/pine64,pinephone-1.2.toml

{ cmd = "QCFG", subcmd = "ims", expect = "1" },

And the problem has gone away, I can reliably get SMS, do calls & 4G internet. Call quality seems ok to me

UltraBlackLinux commented 2 years ago

@pizdjuk It's a problem with the provider and the workaound is to disable IMS in eg25-manager which means that calls and SMS will use 2G instead of VoLTE (and during the SMS sending or call there data transfer will not use LTE speed). I do not recommend to do it because the call quality is worse. I would rather manually switch to 2G in case of sending an SMS or issuing an

That worked for me, thank you!

vap0rtranz commented 2 years ago

{ cmd = "QCFG", subcmd = "ims", value = "2" }

worked for my carrier, however ...

... how long will this workaround fix this?

"It's a problem with the provider and the workaound is to disable IMS in eg25-manager which means that calls and SMS will use 2G instead of VoLTE"

OK, but my carrier has announced that all remaining 2G & 3G fallback will be terminated by EOY 2022. Probably to coincide with 5G rollouts. So this will break for a lot of Pinephone users as fallback options go away.

Biktorgj commented 2 years ago

It's a tough question. Disabling IMS doesn't disable LTE, what it does is it stops trying to send SMS and calls over IP (IMS means IP multimedia subsystem), so you use the good ol' GSM way of sending messages.

That said, the problem is, to use IMS, you need to have a compatible profile for your carrier, and the Quectel modem only provides a few + a generic one, which works for some but doesn't for others. Here's a repo with MBN profiles extracted from various vendors: https://github.com/JohnBel/QualcommMBNs/tree/master/extracted/mbn

But pushing those profiles to the modem won't work either, because they're digitally signed and will only work for the vendor whose signature match.

So short of re-signing those files and hoping for the best, or asking Quectel to bundle profiles for major carriers everywhere, there's not much more I can do about it (I even made a tool to push the mbn profiles to the modem)

I want to, at some point, to try to sign those files with test keys to see if they load, but I don't know what's inside, so I don't really know if they're chipset dependent or only have some kind of apn, proxy or what, so no promises on whether it'll work or not

vap0rtranz commented 2 years ago

Yea, too many variations of carrier-modem configurations.

Or push this onto users but with a more more streamlined/standard way for users to re-config the modem, like pointing everyone to eg25-manager.

Mobian discussion basically re-iterated the carrier profile point for this same IMS issue with SMS: https://gitlab.com/mobian1/issues/-/issues/254.

The TOML profiles for eg25-manager are device specific (ex: pine64,pinephone-1.2), while the Quectel modem profiles are carrier specific (hVoLTE-Verizon) -- but just a few and generic. Seems like a marriage of these two approaches would line-up compatibility if users were guided on where to re-config.

Maybe it won't be a big deal. I think this year (2022) with Pinephone Pro purchases happening using the same Quectel modem and carriers everywhere terminating UMTS & CDMA we'll see how much carrier-modem compatibility was dependent on falling back to non-LTE or a specific configuration.

vap0rtranz commented 2 years ago

P.S.

A Mobian user posted LineageOS's list of carrier profiles in an eg25-manager issue, so x-posting that here b/c it looks to be a better option (to externalize the profiles in a common config): https://gitlab.com/mobian1/devices/eg25-manager/-/issues/25

Biktorgj commented 2 years ago

Hi everyone,

I'm working on a small tool to convert, unpack and repack IMS/VoLTE configuration files. It's not yet finished but it already seems to work for some files, but I have no idea if the modem is able to connect to IMS with them as I don't have access to those providers.

If your carrier is somewhere in this folder, can you link me the file and I'll try to provide you a converted mcfg_sw file so you can push it to the modem and tell me if it works?

pothos commented 2 years ago

Maybe this here: https://github.com/JohnBel/QualcommMBNs/tree/master/mbn/Lenovo/Zuk2/mcfg_sw/generic/eu/telefoni/volte/germany

Biktorgj commented 2 years ago

@pothos How confortable are you with command line stuff? Here's the source for MBNLoader: https://github.com/the-modem-distro/meta-qcom/tree/playing_with_diag/recipes-modem/mbnloader/files

And here the profile you linked, converted test.zip

How to try this:

Build MBNloader from source (save it to some dir and run make. You should only need gcc installed on your phone)
Unzip test.zip
Reboot the modem/the phone, and after a fresh boot, once everything is running, stop ModemManager (systemctl stop ModemManager
As root, run mbnloader -p /dev/ttyUSB2 -m mcfg_sw.mbn-mdm9607
If everything works, you should see a new profile when sending the AT command AT+QMBNCFG="List" called Germany-VoLTE-Telefonica
Reboot the modem and restart ModemManager, or reboot the entire phone, however you want :)

If Autosel is enabled (it is by default), it might choose it automatically on next reboot if your SIM belongs to that carrier or if the connected network matches. You can also select manually, but let's not get ahead of ourselves.

If you can give it a try, please post results here, especially the results of the command AT+QMBNCFG="List" after step 6. If you have the profile, and the list command returns something like this:

AT+QMBNCFG="List"
+QMBNCFG: "List",24,1,1,"Germany-VoLTE-Telefonica",0x05XXXXX,XXXXXXX

It will mean the profile is correctly enabled. Worst case scenario the modem will crash and you'll have to reboot the phone to get it back online

Biktorgj commented 2 years ago

Hi again, I tested the profile I posted before and it wasn't being correctly added (despite the modem saying everything was fine.... )

Attaching a new profile: Germany-VoLTE-Telefonica.zip

If everything works as expected, you should get this in QMBNCFG list: +QMBNCFG: "List",7,0,0,"Germany-VoLTE-Telefonica",0x05010C24,100731940 (well, you'll probably get 24,0,0 instead of 7,0,0 if you never deleted a profile in the modem...)

Small explanation for what you see in that list: +QMBNCFG: "COMMAND", Profile number (7), Profile selected (0), Profile active(0), "Name of the profile", Profile version (0x05010C24), Generation date (100731940)

Following my own example, if you have a SIM from Telefonica Germany, and you manually select the profile, you should get +QMBNCFG: "List",7,1,1,"Germany-VoLTE-Telefonica",0x05010C24,100731940 If Autoselect is enabled (default) and you use that provider, you should get something like this: +QMBNCFG: "List",7,0,1,"Germany-VoLTE-Telefonica",0x05010C24,100731940

pothos commented 2 years ago

Thanks for the detailed instructions and the update!

Immediate result after applying:

AT+QMBNCFG="List"
+QMBNCFG: "List",0,0,0,"ROW_Generic_3GPP",0x0501081F,202108201
+QMBNCFG: "List",1,0,0,"VoLTE-ATT",0x0501033C,202105171
+QMBNCFG: "List",2,0,0,"hVoLTE-Verizon",0x05010141,202011021
+QMBNCFG: "List",3,0,0,"Sprint-VoLTE",0x05010205,202010201
+QMBNCFG: "List",4,0,0,"Commercial-TMO_VoLTE",0x05010505,202106241
+QMBNCFG: "List",5,0,0,"Telus-Commercial_DO",0x0580F601,202105061
+QMBNCFG: "List",6,0,0,"Commercial-SBM",0x05011C18,202004211
+QMBNCFG: "List",7,0,0,"Commercial-DT",0x05011F1C,202106171
+QMBNCFG: "List",8,0,0,"Reliance_OpnMkt",0x05011B38,202003251
+QMBNCFG: "List",9,0,0,"TF_Spain_VoLTE",0x05010CFA,202106171
+QMBNCFG: "List",10,0,0,"Volte_OpenMkt-Commercial-CMCC",0x05012071,202102221
+QMBNCFG: "List",11,0,0,"VoLTE_OPNMKT_CT",0x050113FC,202105131
+QMBNCFG: "List",12,0,0,"CU-VoLTE",0x05011508,202109101
+QMBNCFG: "List",13,0,0,"Telstra-Commercial_VoLTE",0x0580079E,202105192
+QMBNCFG: "List",14,0,0,"Commercial-KDDI",0x0501071D,202011192
+QMBNCFG: "List",15,0,0,"Commercial-DCM",0x05010D17,202104121
+QMBNCFG: "List",16,0,0,"Commercial-SKT",0x05012715,202103121
+QMBNCFG: "List",17,0,0,"Commercial-KT",0x05012C0D,202108241
+QMBNCFG: "List",18,0,0,"Commercial-LGU",0x05012626,202108241
+QMBNCFG: "List",19,0,0,"Commercial-USCC",0x0504FC40,202011111
+QMBNCFG: "List",20,0,0,"Optus-Commercial_VoLTE",0x05800CA2,201910241
+QMBNCFG: "List",21,0,0,"STC_Saudi_VoLTE",0x0501FE01,201912231
+QMBNCFG: "List",22,0,0,"Commercial-Rogers",0x05018821,202108021
+QMBNCFG: "List",23,1,0,"Germany-VoLTE-Telefonica",0x05010C24,100731940

OK

After the modem reboots I get this here, however:

AT+QMBNCFG="List"
+QMBNCFG: "List",0,1,1,"ROW_Generic_3GPP",0x0501081F,202108201
+QMBNCFG: "List",1,0,0,"VoLTE-ATT",0x0501033C,202105171

+QMBNCFG: "List",2,0,0,"hVoLTE-Verizon",0x05010141,202011021
+QMBNCFG: "List",3,0,0,"Sprint-VoLTE",0x05010205,202010201
+QMBNCFG: "List",4,0,0,"Commercial-TMO_VoLTE",0x05010505,202106241
+QMBNCFG: "List",5,0,0,"Telus-Commercial_DO",0x0580F601,202105061
+QMBNCFG: "List",6,0,0,"Commercial-SBM",0x05011C18,202004211
+QMBNCFG: "List",7,0,0,"Commercial-DT",0x05011F1C,202106171
+QMBNCFG: "List",8,0,0,"Reliance_OpnMkt",0x05011B38,202003251
+QMBNCFG: "List",9,0,0,"TF_Spain_VoLTE",0x05010CFA,202106171
+QMBNCFG: "List",10,0,0,"Volte_OpenMkt-Commercial-CMCC",0x05012071,202102221
+QMBNCFG: "List",11,0,0,"VoLTE_OPNMKT_CT",0x050113FC,202105131
+QMBNCFG: "List",12,0,0,"CU-VoLTE",0x05011508,202109101
+QMBNCFG: "List",13,0,0,"Telstra-Commercial_VoLTE",0x0580079E,202105192
+QMBNCFG: "List",14,0,0,"Commercial-KDDI",0x0501071D,202011192
+QMBNCFG: "List",15,0,0,"Commercial-DCM",0x05010D17,202104121
+QMBNCFG: "List",16,0,0,"Commercial-SKT",0x05012715,202103121
+QMBNCFG: "List",17,0,0,"Commercial-KT",0x05012C0D,202108241
+QMBNCFG: "List",18,0,0,"Commercial-LGU",0x05012626,202108241
+QMBNCFG: "List",19,0,0,"Commercial-USCC",0x0504FC40,202011111
+QMBNCFG: "List",20,0,0,"Optus-Commercial_VoLTE",0x05800CA2,201910241
+QMBNCFG: "List",21,0,0,"STC_Saudi_VoLTE",0x0501FE01,201912231
+QMBNCFG: "List",22,0,0,"Commercial-Rogers",0x05018821,202108021
+QMBNCFG: "List",23,0,0,"Germany-VoLTE-Telefonica",0x05010C24,100731940

OK

pothos commented 2 years ago

With AT+QMBNCFG="Select","Germany-VoLTE-Telefonica" I am back to +QMBNCFG: "List",23,1,0,"Germany-VoLTE-Telefonica",0x05010C24,100731940

pothos commented 2 years ago

Now it seems to switch by itself to 2G when I do an USSD code, previously it didn't work and I had to switch manually to 2G for it to work. I wonder whether it switches back to 4G eventually because currently it's stuck on 2G even after making a call (the call was 2G as I could hear a lot of noise). I see this here:

AT+QCFG="ims"
+QCFG: "ims",1,0

Edit: I guess it means that the profile isn't working correctly

Biktorgj commented 2 years ago

So... Selected but not active. I have to ask just in case, your carrier is Telefonica or is it a mvno operating under their network? And... By any chance, won't you have an Android phone available to check if VoLTE works with it?

This is the same result we had with a custom built profile the other day, but there are so many things involved here and so little debugging capacity I have to ask the dumb questions just in case

pothos commented 2 years ago

Yeah, not directly Telefonica… I'm pretty sure that VoLTE worked under Android. It also works in the PinePhone. What doesn't work is sending SMS/USSD with the PinePhone under 4G.

pothos commented 2 years ago

I found some old log from 2021 where I had played with the profiles there I had +QMBNCFG: "List",9,1,0,"TF_Germany_VoLTE",0x05010C1B,202004151 and also no success.

Biktorgj commented 2 years ago

Makes sense, IMS profiles are made for the specific carrier. You use the network as a transport, but all the endpoints where the phone needs to connect to belong to your actual provider, not the network you're connected to. That's why you might also have a different APN config that the one provided by O2/Telefonica. Since it can't activate the profile (9,1,0 and not 9,1,1) it automatically changes network to 2/3G to make a call, or request a USSD code. This though, +QMBNCFG: "List",0,1,1,"ROW_Generic_3GPP",0x0501081F,202108201, tells me the modem could correctly connect with the default profile. Don't VoLTE calls work for you?

pothos commented 2 years ago

With the default profile VoLTE calls work, just SMS/USSD not until I switch to 2G/3G

bircoph commented 2 years ago

Hi!

If your carrier is somewhere in this folder, can you link me the file and I'll try to provide you a converted mcfg_sw file so you can push it to the modem and tell me if it works?

Mine is absent in the Lenovo folder, but is available in others, e.g. here

Biktorgj commented 2 years ago

@bircoph Test profile here: mts_vowifi_v5.zip

Tested in one of my modems and it gets correctly loaded, but that's as far as I can test:

+QMBNCFG: "List",8,0,0,"MTSVoWiFi_v5",0x05012805,134293509

Let me know if it works!

bircoph commented 2 years ago

+QMBNCFG: "List",8,0,0,"MTSVoWiFi_v5",0x05012805,134293509

Wow! This is something amazing. By accident this is not a common VoLTE profile, but VoWIFI profile just recently announced by the provider for "a limited number of phone models". It allows voice phone calls via wifi access points without mobile network at all. I will test it when I'll be in an area of such point.

Meanwhile while looking how to enable VoWIFI support I found that for both VoLTE and VoWIFI I need to enable "internet calls, technological mark" service option in my personal account at provider. It is free of charge, but disabled by default. After enabling it 4G-only VoLTE calls work on my pinephone even on default VoLTE profile. SMS over 4G (technically over IP intranet) also work. Only USSD requests does not work in 4G-mode.

I found another suitable profile in the repo here. Could you please convert it for testing as well? From its dump it looks like this is a common VoLTE profile, not VoWIFI, but it has a special option ussdAcceptContact — maybe USSD requests will work with this profile over 4G.

Thanks!

Biktorgj commented 2 years ago

Even if the profile says "VoWifi" it should work without wifi, the baseband uses only one profile for a carrier so they won't have two different profiles for the same operator and change them on the fly (it would need to disconnect a IMS session and reconnect to do that). You won't be able to use VoWifi in the Pinephone though, as the wlan adapter isn't connected directly to the modem, so the modem can't route through it

Here's the profile you linked to: mts_test2.zip You'll need to first remove the previous profile or it won't load it, since it seems to be older.

Send AT+QMBNCFG="Delete","MTSVoWiFi_v5" to remove the profile, and then add it with mbnloader, it should get you something like this:

+QMBNCFG: "List",13,0,0,"MTS RUS_Commercial",0x0501A202,134324737

I made the profile conversion as seamless as I could with mcfg tools. If you find some other profile and want to check it out, it's just two commands:

Unpack the original with -f flag (it fixes the carrier ID): ./unpack_mcfg -i ORIGINAL_MBN_FILE -o FOLDER_TO_UNPACK_IT_TO -f
Repack the file without touching anything: ./pack_mcfg -i FOLDER_WHERE_YOU_UNPACKED_IT/dump.list -o NEW_FILE_NAME.mbn

Then just load it with mbnloader as with the other profiles

bircoph commented 1 year ago

Hi! I was finally able to test this. Sorry for a delay, but pinephone is used as my main phone, so I have limitations for time slots when I can conduct dangerous experiments.

Both profiles load and work, but worse than default ROW_Generic_3GPP in 4G-only mode:

On VoWifi profile voice calls and ipv4 internet don't work, but ipv6 does.
On older profile voice calls also don't work, but both ipv4 and ipv6 work fine.

For now I reverted on ROW_Generic_3GPP where in 4G-only mode everything except USSD requests and calls to short service numbers work.

I'll try to test other profiles, at least it should be possible to bisect why ipv4 works on one profile and don't work on another.

vmaurin commented 1 year ago

@Biktorgj shared a bunch of commands not to use IMS for SMS, and for me it works :

I have IMS enabled
Voice is using it
Data is working
I can send SMS

Here the process

Reading the current value

echo 'AT+QNVFR="/nv/item_files/modem/mmode/sms_domain_pref"' | atinout - /dev/EG25.AT -

if there is an error, echo 'AT+QNVFD="/nv/item_files/modem/mmode/sms_domain_pref"' | atinout - /dev/EG25.AT - to delete the file

Values are : -1 [UNSET], 00 [NOT ALLOWED OVER IMS], 01 [PREFERRED OVER IMS], 02 [MAXVAL]

Writing the new value

echo 'AT+QNVFW="/nv/item_files/modem/mmode/sms_domain_pref", 00' | atinout - /dev/EG25.AT -

Checking it was saved

echo 'AT+QNVFR="/nv/item_files/modem/mmode/sms_domain_pref"' | atinout - /dev/EG25.AT -

Reboot

Thank you @Biktorgj

pothos commented 1 year ago

Great stuff, I did

echo 'AT+QNVFW="/nv/item_files/modem/mmode/sms_domain_pref", 00'  | sudo socat - /dev/ttyUSB2,crnl

and it directly worked plus it is preserved over reboots :) I guess the riddle is solved now :) Sad that these workarounds are needed - this setting probably can't be the default for everyone? Then I suggest to close this issue here it should be documented in a new section under https://github.com/the-modem-distro/pinephone_modem_sdk/blob/kirkstone/docs/SETTINGS.md

miklosakos commented 1 year ago

@bircoph sorry to necrobump this but could you convert these: Pannon/Telenor/Yettel HU, Vodafone HU profiles?

the-modem-distro / pinephone_modem_sdk