Clarification on how TinyGsm works and its connections to Networks. Twilio, SMS mixed with LTE-M?

[X ] I have read the Troubleshooting section of the ReadMe

What type of issues is this?

[ ] Request to support a new module

[ ] Bug or problem compiling the library [ ] Bug or issue with library functionality (ie, sending data over TCP/IP) [X ] Question or request for help

Can someone clarify how the TinyGsm library works in regards to the network paths it establishes. (I am looking through the library code but just not understanding it yet). A GPRS and/or EPS connection I understand is established. GPRS might use 2G and while SMS might use 2G, I’m pretty sure they are not the same… but would there be scenario where an SMS network might get used? I think my application uses the LTE-M network with TinyGsm, but I’m not 100% sure. I limited the mode in the code to LTE-M (AT_CNMP=38) and it works fine so that reinforces my belief it goes through the LTE-M network . Am I correct?

Alternatively, does anyone use the TinyGsm library in code that communicates through Twilio’s Super SIM and have you had problems lately?

What are you working with?

I’m using a LilyGO T-SIM7000G controller using a SuperSIM from Twilio to help send sensor data to a MongoDB database. The TinyGsm library is the heart. I based the code on the TinyGsm examples and use the Arduino IDE to develop.

Why the questions?

I have successfully sent the data in daily bursts to the database for days at a time, it looks really good. I’m very grateful for the TinyGsm library. However, I’ve experienced days of outage throughout August. Network connections fail for hours, actually whole days, but if I wait long enough, things will connect and go back to working. Looking at Twilio, AT&T and T-Mobile sites for information on outages has been challenging. (The Twilio Network carrier options set for my area are ATT & T-Moblie.)

Chats with Twilio support says that there were no Twilio outages when I experienced it. They are researching AT&T and TMobile.
The Twilio site HOWEVER, listed their SMS network having multiple maintenance windows roughly at the times I experienced the outages . Twilio points out the SMS and LTE-M networks are separate and so it’s a coincidence. A couple days ago I had a third outage again, again roughly matching those SMS maintenance windows, I’m doubting it’s just coincidence. I really like to know what’s going on…a full day of outage is terrible. There's an implication its the way I connect to the network and since I use TinyGsm to do that, all these questions get generated. Are the TinyGSM gprsConnect(), gprsConnect(), waitForNetwok,(), or isNetworkConnected() functions dependent on something dealing with the SMS network?

A side question is I don’t know exactly what to look for regarding the status of the LTE-M networks. Guidance? Is it just the 4G/5 network as a whole?

I use a LilyGO T-SIM7600G-H with a Consumer SIM which disconnects 23-24hrs and to date doesn't reconnect so I modified the code to check then reconnect which works. I looked into M2M SIMs as they don't disconnect but don't have the same features of a consumer SIM, call, SMS etc.

Revised code here

What is SuperSIM from Twilio? ""SuperSIM" is a product offered by Twilio, a cloud communications platform, to help businesses manage their cellular connectivity needs for Internet of Things (IoT) devices and applications"

droidblastnz Your issue is a little different than mine but I appreciate the contribution. We both seem to be struggling on finding details with respect to the networks we are using. Especially the path like the LTE-M connection I'm assuming I'm making the connection through. You are speculating your network operator has time limits you need to overcome, You fortunately seemed to have improved things with a re-try of the connection when you detect an outage and re connecting in a slightly different way to make the connection. I've coded similar things, escalating the actions and eventually getting to a hard reset depending on the number of iterations through the connection logic. In my case, during the time frame of outage, the connections just never make it no matter what I do. When the window 'reopens', everything goes backs to working perfectly.

The Twilio site has a couple of pages with some manual AT commands to try to test the network manually (https://www.twilio.com/docs/iot/supersim/cellular-modem-knowledge-base/simcom-supersim#sim7000g-cat-m1-nb-iot-gsm). I have had mixed experiences doing that. Sometimes that sequence failed...convincing me my problem was a network issue. A couple of other times, that sequence allowed me to connect to a Network (wish I understood which one) and do a simple HTTP GET request from a canned location but my application code based on TinyGsm still would NOT connect. The Twilio AT command sequence is far simpler than what I see in the TinyGsm library so I'm not too surprised that there could be a difference. This also reinforces my thought there is something in the library sequence that I could change to sometimes make a "better" connection, that is IF I understood the library better and what was really going on in the network. I'm trying to write my own "library", merging or optioning the two techniques, but I'm failing spectacularly, it is currently a little out of my knowledge reach.

@stardad60 are you using the TinyGSM library version or LilyGO-T-SIM7000G?

Document is here https://github.com/Xinyuan-LilyGO/LilyGO-T-SIM7000G/tree/master/docs

SIM7000 Series_AT Command Manual_V1.06.pdf

and

SIM7000系列_HTTP(S)_应用文档_V1.03.pdf

Check in and add a issue to LilyGO-T-SIM7000G's Github and ask lewisxhe to comment.

In the end I had to go through AT commands step by step until I saw the outcome, bit hard for http but in my case it was issues with MQTT, network reconnects and also FTP. I ended up installing docker/HiveMQ and also Filezilla locally so I could see the logs as I complete AT commands and then also as code. At times hit and miss hence took me close to 12 months to finish my project.

Also note in my case I had old firmware that didn't support modern authentication ie 1.2TLS for MQTTS

Again in my case with the SIM7600

AT+CNMP: <mode>

2 – Automatic
13 – GSM Only
14 – WCDMA Only
38 – LTE Only
59 – TDS-CDMA Only
9 – CDMA Only
10 – EVDO Only
19 – GSM+WCDMA Only
22 – CDMA+EVDO Only
48 – Any but LTE
60 – GSM+TDSCDMA Only
63 – GSM+WCDMA+TDSCDMA Only
67 – CDMA+EVDO+GSM+WCDMA+TDSCDMA Only
39 – GSM+WCDMA+LTE Only
51 – GSM+LTE Only
54 – WCDMA+LTE Onlyeadline

TinyGSM

  /*  Preferred mode selection : AT+CNMP
        2 – Automatic
        13 – GSM Only
        14 – WCDMA Only
        38 – LTE Only
        59 – TDS-CDMA Only
        9 – CDMA Only
        10 – EVDO Only
        19 – GSM+WCDMA Only
        22 – CDMA+EVDO Only
        48 – Any but LTE
        60 – GSM+TDSCDMA Only
        63 – GSM+WCDMA+TDSCDMA Only
        67 – CDMA+EVDO+GSM+WCDMA+TDSCDMA Only
        39 – GSM+WCDMA+LTE Only
        51 – GSM+LTE Only
        54 – WCDMA+LTE Only
  */
  String ret;
  ret = modem.setNetworkMode(2);
  DBG("setNetworkMode:", ret);

@droidblastnz I'm currently using the TinyGsm library and choosing the TINY_GSM_MODEM_SIM7000 part of the library. I might be misinterpreting your question...but I don't see a 'library' at the LilyGO-T-SIM7000G site. Certainly there is a lot of information on the AT commands and different examples there, but I don't see a library per se. Point me closer if I missed. I think you are providing good references for the AT command sequences for my particular controller that can help me in trying a different sequence.

Your lists of modes is a reminder there are multiple network connections possible, more ways to get confused. I'm keeping the command in my code to use option 38 and limit the connection LTE-M only for now to minimize the variables.

@stardad60 I found there were differences in the https://github.com/Xinyuan-LilyGO/LilyGO-T-SIM7000G to TinyGSM library and or the examples.

Also I remember having weird issues so I reset the modem often. Also how are you powering the SIM? You need a good power supply of 2A or more. For a while I had a USB Power Monitor to monitor the 5v line.

How much signal are you getting, all my testing was indoors but I added a large external pigtail higher gain antenna. Check with AT+CSQ +CSQ: <rssi>,<ber>

0 – - 113 dBm or less
1 – - 111 dBm
2...30 – - 109... - 53 dBm
31 – - 51 dBm or greater
99 – not known or not detectable
100 – - 116 dBm or less
101 – - 115 dBm
102…191 – - 114... - 26dBm
191 – - 25 dBm or greater
199 – not known or not detectable
100…199 – expand to TDSCDMA, indicate RSCP received

Reset Modem

Sometimes (especially if you played with AT commands), your module configuration may become invalid. This may result in problems such as:

Can't connect to the GPRS network
Can't connect to the server
Sent/received data contains invalid bytes
etc. To return module to Factory Defaults, use the ResetModem sketch

Also re reading the TinyGSM notes as these are important to remember. I noted in my case I had several external sensors attached such as a shock sensor, PIR sensor, ESP32CAM running on the T-SIM7600 hence suspected power or interference issues. The wiring i.e., Berg connector - later known as a DuPont connector arent good so post my code checks I soldered - hard wired, when I placed into a box I spaced all devices away from the T-SIM7600.

Why - I was seeing funny characters in my serial coms. The T-SIM7600 runs at 115200 to the WRover ESP, tried to slow the speed down but wasnt successful. Some of my code needed to be changed to remove these characters, so my point here and above is go back to the basics and check. I never got to the bottom of the issue but saw threads and comments about bad design and interference at the board level. Consider the SIM/Modem is transmitting place the antenna away from the ESP etc.

Are you running the serial com port monitor? What do you see when the T-SIM7000 fails? Add the signal check command into the loop and start checking the basics.

This library is blocking in all of its communication. Depending on the function, your code may be blocked for a long time waiting for the module responses. Apart from the obvious (ie, waitForNetwork()) several other functions may block your code for up to several minutes. The gprsConnect() and client.connect() functions commonly block the longest, especially in poorer service regions.

Ensure stable data & power connection

Most modules require as much as 2A to properly connect to the network. This is 4x what a "standard" USB will supply! Improving the power supply actually solves stability problems in many cases!

Read about powering your module. Keep your wires as short as possible Consider soldering them for a stable connection Do not put your wires next to noisy signal sources (buck converters, antennas, oscillators etc.) If everything else seems to be working but you are unable to connect to the network, check your power supply!

Failed connection or no data received

The first connection with a new SIM card, a new module, or at a new location/tower may take a LONG time - up to 15 minutes or even more, especially if the signal quality isn't excellent. If it is your first connection, you may need to adjust your wait times and possibly go to lunch while you're waiting.

If you are able to open a TCP connection but have the connection close before receiving data, try adding a keep-alive header to your request. Some modules (ie, the SIM7000 in SSL mode) will immediately throw away any un-read data when the remote server closes the connection - sometimes without even giving a notification that data arrived in the first place. When using MQTT, to keep a continuous connection you may need to reduce your keep-alive interval (PINGREQ/PINGRESP).

Hope this helps.

@droidblastnzThe major point of using the LilyGOTTSIM7000G module is its inclusion of the typical periphery circuits. It has a built-in battery holder, over/underload protection circuitry, and solar power connections with its form of power protection. So the only power intended is a 18650 style LiPo battery. I’m using a 9900mAH battery. There are voltage monitors on the battery and solar panel, those readings are stored along with the data and I have not seen degradation before any outage, granted that is not a current measurement. I have read about the 2 amp needs for the modem and stable power but I can’t measure it without hacking deeply into the board. If the module can’t handle it as designed, there is no point in the module existing in its current form so I haven’t been overly concerned with the power.

That said, the very first outages I experienced I worried a bit about the power and put a fully charged battery in right away but saw no immediate differences . When the data connection window reopens, it’s usually after hours of reconnect tries which drains the batteries a bit so the connection resumed at slightly lower voltages so that further steered me away from power issues.

I do have a keep alive header . I also have been adding to the code print outs of responses of AT commands , so now I have to wait for another outage. Thanks for the guidance.

vshymanskyy / TinyGSM