Library not working with ESP32 hardware Serial2

[kennythum]

Hello there. Firstly, i would say great work and thank you for the wonderful work! I encountered a problem that the library is not working with the ESP32 on hardware Serial2. Somehow I try sending AT command via Serial2 and it works! But running with tinyGsm library it gives unknown characters feedback, or no response sometimes. I did try to use Serial1 (UART1), but Serial1 give nothing at all, no responding. Hence, only Serial2 were able to communicate with the GSM module...just when using with tinyGsm library, it can't work. Anyone could help? Really appreciate.

[hubaksis]

@kennythum What GSM module do you use?

[kennythum]

@hubaksis Sorry for not providing any details: ESP32 board : ESP-WROOM-32 (revision :1) GSM Module : SIM800C IDE : Arduino 1.8.4

[hubaksis]

Can you please show your connection between the module and ESP32?

[kennythum]

I don't think its the hardware or the connection problem. Because I can communicate it with other devices... But here you go: https://ibb.co/m3OE8H

HardwareSerial (2) is on GPIO_16(RX) GPIO_17(TX)

[hubaksis]

@kennythum Actually, it can be, I am just not sure yet :)

Thank you for the connections. Now I can see that it's the same as mine.

I have the same issue when I used communication through TX and RX pins on ESP8266. ESP8266 and ESP32 have 3.3V logic level. SIM modules (probably, just my guess) are using a different logic level.

So, I checked the communication and found out that the modem can read and understand what the controller is sending, it responds back, but the controller can't understand the response.

So, there are two options you can check:

1. Use different software pins and use SoftwareSerial library. It worked for me without any issues.
2. Use TTL logic level shifter. I am going to do it later to check why the communication is not working on TX RX. Voltage divider didn't work for me.

[hubaksis]

I did a test and connected a modem to ESP8266's Serial using TTL logic level converter. It works. Without level shifter - no. By the way, ESP doen't start if TX pin is connected. Probably it should be pulled up high.

[kennythum]

Hello @hubaksis , thanks for testing out! Thumbs up to you. By the way, are you using TTL logic level IC? If yes, which chip did you use? And FYI, the GSM module that I used was integrated with TTL logic level converter, it can support 3.3-5V range, so no problem at all. And I did tried on my ESP8266, it work as well...and even my ESP32! I can send AT command to the GSM module directly, and get feedback from it. I did tried manually send AT command to connect into the network, and it works! Just that when I use with tinygsm library, it return gibberish feedback, and hence my network status is 4 (unknown)...I'm totally out of clue.

[hubaksis]

@kennythum GSM module can understand 3.3-5V logic, but ESP can't. I did this test on ESP8266 and modules SIM808 and 5320. TTL logic converter looks like this My connections are:

Modem RX - RXI
Modem TX - TXO

ESP TX - RXO
ESP RX - TXI

I had to disconnect ESP TX for the module to start and then connect it back.

If tinyGSM return gibberish - it means that the library chose a wrong connection speed. What is the result of the TinyGsmAutoBaud?

If you can send commands and see the answer in the console window, then the library also will be able to do so. It's really simple and this part of the code can't work.

What the library does when you call int modemSpeed = TinyGsmAutoBaud(SerialAT);

1. It selects the next speed from the array { 115200, 57600, 38400, 19200, 9600, 74400, 74880, 230400, 460800, 2400, 4800, 14400, 28800 };
2. It sets the port speed by calling SerialAT.begin(rate);
3. It sends command "AT" to the modem and waits for an answer. If the speed is modem working speed, then the modem will understand the command and will respond to it.
4. If the answer is "OK" then the chosen speed is correct.

It's not a rocket science and I am 90% sure that this code can't fail. It's really simple and there are not vital dependencies.

So,

1. Add a delay to the setup to wait until the module starts or power it up separately (on different modules I noticed a delay until it start to respond).
2. Can you please provide the code that you are using to communicate with the modem
3. The output from this code.
4. Try to load AT_Debug sketch from tools to the ESP and check whether you can communicate with the modem.

[bionicbone]

I had an issue with it and ended up replacing the Return 0 at the end with Return 19200. I didn't dig in, I'm using a nano and could have been memory issues or something.

[ankgt]

@hubaksis, @kennythum

I have a similar configuration like yours where I have connected an ESP32 Doit Devkit V1 board to a SIM800L modem using HardwareSerial (Uart2 connected to pins 16,17).

Issuing AT commands manually using the AT_Debug works fine, although I had to manually set the baud to 9600. The auto detect selected 115200, which did not work.

Using the AT commands I can successfully make a GET request using the correct sequence of commands.

However, running the TinyGSM Diagnostics example results in garbled text results.

I have seen other issues like this, but not seen any solutions. I am thinking that it should not be a TTL logic level issue, because in that case it should not work with issuing manual AT commands either, am I right?

Any suggestions on what I can try next will be appreciated. Thanks!

Some of the output on the serial console looks like this:

AT+CREG?
AT+CREG?
cG⸮⸮]⸮⸮AT+CREG?

+CREG: 0,1 OK
Connecting to airtelgprs.comAT+CIPSHUT

AT+SAPBR=3,1,"Contype","GPRS"
AT+SAPBR=3,1,"APN","airtelgprs.com"
AT+SAPBR=3,1,"USER",""
AT+SAPBR=3,1,"PWD",""

OK
AT+CGDCONT=1,"IP","airtelgprs.com"
Z⸮⸮g⸮
ERROR
AT+CGACT=1,1
n⸮R_⸮⸮⸮⸮⸮⸮%⸮K,⸮C⸮⸮⸮⸮DO⸮I>R3AT+SAPBR=1,1
AT+SAPBR=2,1
 fail

[vshymanskyy]

This is covered in README: https://github.com/vshymanskyy/TinyGSM#esp32-hardwareserial

[ankgt]

@vshymanskyy Thanks. I should have mentioned that I have already read this and included the extra parameters in Serial.begin() as suggested. It does not help.

I have not been able to try shifting to Uart1 though, because my USB serial port is connected to it.

If it is a hardware or Arduino HardwareSerial problem, then should it not work if I am manually sending the AT commands using AT_Debug?

Thanks.

[vshymanskyy]

@ankgt when you enter manually, the speed of typing is so slow ;)

[vshymanskyy]

@ankgt @kennythum I would suggest posting this bug to ESP32, most probably the issue is there: https://github.com/espressif/arduino-esp32

[ankgt]

I will do that.

I still am wondering about the manual commands though. While I agree I cannot type so fast :) if I manually issue the AT commands to make an HTTP GET request, the modem retrieves the data from the site and sends it back to the ESP32 at full speed, and that also works without any errors. Also when I issue the commands through the Arduino serial monitor, it does not send them character by character, but instead sends the entire string once the enter key is pressed.

So could the issue be the timing between the individual commands being sent then?

[ankgt]

Just a quick followup. I tried using another library (https://github.com/carrascoacd/ArduinoSIM800L) and modified it slightly as it was designed for use with SoftwareSerial. It works correctly with my ESP32 and HardwareSerial UART2. I can successfully make HTTP GET and POST requests.

So I don't think the problem is linked to the ESP32 HardwareSerial implementation.

[vshymanskyy]

@ankgt You seem to be making wrong conclusions ;) It means that the library you mentioned does not cause the same the problem. But TinyGSM does not use anything but correct, platform-agnostic Arduino API. And it is not causing troubles in many other platforms.

[ioticos]

I have esp32 and sim800 working perfect. if somebody need some help...

[bionicbone]

I may do soon, although I'll be setting it up to do FTP in a couple of weeks. I've got it working with a nano so hopefully it won't be so difficult.

[kennythum]

The funny thing is, I have couples of ESP32 board, 1 is the older version (which i believe, CE1313 marking engraved on it), and 3 was the later version...they all are the revision 1. The older version ESP32 could make connection with the ESP32 hardware serial1 port (with pin 16,17 <--I know pin 16,17 is for serial2, but serial2 could not work, so i reroute serial1 to pin 16,17). Meanwhile, the 3 later version ESP32 chip could not use with the tinyGSM library at all, but they all could make a good AT response by sending commands to the SIM800C! It even can establish connection, HTTP GET/POST without a problem.

[ankgt]

@vshymanskyy It's possible! I am fairly new to embedded development and still trying to figure things out!

@kennythum I guess my board is a newer one, since I can to establish proper connections using other libraries. I would really like to use TinyGSM though, since its very well tested and has wide functionality.

If you connected using Serial1 port using 16,17, I am assuming you could no longer monitor the running of the program using the terminal from your PC? Or did you try connecting Serial2 to the built in USB-UART converter pins?

[ankgt]

@mdc4u Yes would love some help! Can you confirm if you are using HardwareSerial with UART2? Is there anything specific you did to get it working?

[ioticos]

Well, in my case, I have my project distributed as follows: UART0 -> usb to debug UART1 -> (gpio 4 and 2) SIM800 (115200 baud) UART2 -> (gpio 16 and 17) GPS

in HardwareSerial.cpp

if (_uart_nr == 0 && rxPin <0 && txPin <0) {
         rxPin = 3;
         txPin = 1;
     }
     if (_uart_nr == 1 && rxPin <0 && txPin <0) {
         rxPin = 4;
         txPin = 2;
     }
     if (_uart_nr == 2 && rxPin <0 && txPin <0) {
         rxPin = 16;
         txPin = 17;
     }

everything works correctly. I have even managed to operate OTA through SIM800 instead of using wifi.

My big problem is that the client, when I move (for example in a car) is randomly disconnected.

Using the same code in a poor nano arduino, the disconnections do not occur. I hope to solve this mystery soon,

I hope I can be useful regards and thanks!

Paul

[vshymanskyy]

@mdc4u

managed to operate OTA through SIM800 instead of using wifi

Could you share some code for that? I would like to check it, and maybe optimize the scenario

[ioticos]

Well, it's complex, but not at all impossible, and no problem in illustrating with a bit of code, At this moment the OTA through SIM800 I have implemented with a private project, give me some time to extract that, and make it more universal.

[vshymanskyy]

@mdc4u would appreciate that!

[ioticos]

@vshymanskyy

sorry for the untidiness I have quickly assembled this example to download .bin server saves it in spiffs and then apply ota.

this works without any problem.

where if I have a problem is to keep a tcp client connected, (in another part of my project of course nothing to do with OTA) in my .ino, check connection periodically as any typical tnygsm script, in case the socket has been closed, I try to connect again, if in several attempts does not happen. well, restart both esp and sim800, this, applied to an arduino nano or an arduino one, works well for me but, B U T this same using an esp32, becomes unstable when I move in a vehicle, is disconnected every time. This does not happen with a nano, what exactly brought me to esp32 is to be able to use their nucleuses separately, and the blessed OTA

What is promised is debt, here goes

#include <Update.h>
#define TINY_GSM_MODEM_SIM800
// Increase RX buffer
#define TINY_GSM_RX_BUFFER 1030

const char apn[] = "";
const char user[] = "";
const char pass[] = "";

HardwareSerial SerialAT(1);

#include <TinyGsmClient.h>
#include <CRC32.h>

#include "FS.h"
#include "SPIFFS.h"

#ifdef DUMP_AT_COMMANDS
#include <StreamDebugger.h>
StreamDebugger debugger(SerialAT, Serial);
TinyGsm modem(debugger);
#else
TinyGsm modem(SerialAT);
#endif
TinyGsmClient client(modem);

const char server[] = "here the server like sample.com.ar";
const int port = 80;

const char resource[] = "/updates/v1_0.bin"; //here de bin file

uint32_t knownCRC32 = 0x6f50d767;
uint32_t knownFileSize = 1024; // In case server does not send it

void setup()
{
    // Set console baud rate
    Serial.begin(115200);
    delay(10);

    if (!SPIFFS.begin(true))
    {
        Serial.println("SPIFFS Mount Failed");
        return;
    }
    SPIFFS.format();
    listDir(SPIFFS, "/", 0);

    // Set GSM module baud rate
    SerialAT.begin(57600);
    delay(3000);

    // Restart takes quite some time
    // To skip it, call init() instead of restart()
    Serial.println("Initializing modem...");
    modem.restart();

    String modemInfo = modem.getModemInfo();
    Serial.print("Modem: ");
    Serial.println(modemInfo);

    // Unlock your SIM card with a PIN
    //modem.simUnlock("1234");
}

void printPercent(uint32_t readLength, uint32_t contentLength)
{
    // If we know the total length
    if (contentLength != -1)
    {
        Serial.print("\r ");
        Serial.print((100.0 * readLength) / contentLength);
        Serial.print('%');
    }
    else
    {
        Serial.println(readLength);
    }
}

void loop()
{
    Serial.print("Waiting for network...");
    if (!modem.waitForNetwork())
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");

    Serial.print("Connecting to ");
    Serial.print(apn);
    if (!modem.gprsConnect(apn, user, pass))
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");

    Serial.print("Connecting to ");
    Serial.print(server);

    // if you get a connection, report back via serial:
    if (!client.connect(server, port))
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");
    // Make a HTTP request:
    client.print(String("GET ") + resource + " HTTP/1.0\r\n");
    client.print(String("Host: ") + server + "\r\n");
    client.print("Connection: close\r\n\r\n");

    long timeout = millis();
    while (client.available() == 0)
    {
        if (millis() - timeout > 5000L)
        {
            Serial.println(">>> Client Timeout !");
            client.stop();
            delay(10000L);
            return;
        }
    }

    Serial.println("Recibiendo Header");
    uint32_t contentLength = knownFileSize;

    File file = SPIFFS.open("/update.bin", FILE_APPEND);

    while (client.available())
    {
        String line = client.readStringUntil('\n');
        line.trim();
        //Serial.println(line);    // Uncomment this to show response header
        line.toLowerCase();
        if (line.startsWith("content-length:"))
        {
            contentLength = line.substring(line.lastIndexOf(':') + 1).toInt();
        }
        else if (line.length() == 0)
        {
            break;
        }
    }

    Serial.println("Recibiendo respuesta");
    timeout = millis();
    uint32_t readLength = 0;
    CRC32 crc;

    unsigned long timeElapsed = millis();
    printPercent(readLength, contentLength);

    while (readLength < contentLength && client.connected() && millis() - timeout < 10000L)
    {
        int i = 0;
        while (client.available())
        {

            if (!file.print(char(client.read())))
            {
                Serial.println("Fallo Append");
            }
            //Serial.print((char)c);       // Uncomment this to show data
            //crc.update(c);
            readLength++;

            if (readLength % (contentLength / 13) == 0)
            {
                printPercent(readLength, contentLength);
            }
            timeout = millis();
        }
    }

    file.close();

    printPercent(readLength, contentLength);
    timeElapsed = millis() - timeElapsed;
    Serial.println();

    client.stop();
    Serial.println("Desconectado del Servidor");

    modem.gprsDisconnect();
    Serial.println("GPRS Desconectado");
    Serial.println();

    float duration = float(timeElapsed) / 1000;

    Serial.print("Tamaño de Archivo: ");
    Serial.println(contentLength);
    Serial.print("Leido:  ");
    Serial.println(readLength);
    Serial.print("Calculado. CRC32:    0x");
    Serial.println(crc.finalize(), HEX);
    Serial.print("Conocido CRC32:    0x");
    Serial.println(knownCRC32, HEX);
    Serial.print("Bajado en:       ");
    Serial.print(duration);
    Serial.println("s");

    Serial.println("Se genera una espera de 3 segundos");
    for (int i = 0; i < 3; i++)
    {
        Serial.print(String(i) + "...");
        delay(1000);
    }

    //readFile(SPIFFS, "/update.bin");

    updateFromFS();

    // Do nothing forevermore
    while (true)
    {
        delay(1000);
    }
}

void appendFile(fs::FS &fs, const char *path, const char *message)
{
    Serial.printf("Appending to file: %s\n", path);

    File file = fs.open(path, FILE_APPEND);
    if (!file)
    {
        Serial.println("Failed to open file for appending");
        return;
    }
    if (file.print(message))
    {
        Serial.println("APOK");
    }
    else
    {
        Serial.println("APX");
    }
}

void readFile(fs::FS &fs, const char *path)
{
    Serial.printf("Reading file: %s\n", path);

    File file = fs.open(path);
    if (!file || file.isDirectory())
    {
        Serial.println("Failed to open file for reading");
        return;
    }

    Serial.print("Read from file: ");
    while (file.available())
    {
        Serial.write(file.read());
        delayMicroseconds(100);
    }
}

void writeFile(fs::FS &fs, const char *path, const char *message)
{
    Serial.printf("Writing file: %s\n", path);

    File file = fs.open(path, FILE_WRITE);
    if (!file)
    {
        Serial.println("Failed to open file for writing");
        return;
    }
    if (file.print(message))
    {
        Serial.println("File written");
    }
    else
    {
        Serial.println("Write failed");
    }
}

void listDir(fs::FS &fs, const char *dirname, uint8_t levels)
{
    Serial.printf("Listing directory: %s\n", dirname);

    File root = fs.open(dirname);
    if (!root)
    {
        Serial.println("Failed to open directory");
        return;
    }
    if (!root.isDirectory())
    {
        Serial.println("Not a directory");
        return;
    }

    File file = root.openNextFile();
    while (file)
    {
        if (file.isDirectory())
        {
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if (levels)
            {
                listDir(fs, file.name(), levels - 1);
            }
        }
        else
        {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("  SIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}

void deleteFile(fs::FS &fs, const char *path)
{
    Serial.printf("Deleting file: %s\n", path);
    if (fs.remove(path))
    {
        Serial.println("File deleted");
    }
    else
    {
        Serial.println("Delete failed");
    }
}

void updateFromFS()
{
    File updateBin = SPIFFS.open("/update.bin");
    if (updateBin)
    {
        if (updateBin.isDirectory())
        {
            Serial.println("Error, en el directorio");
            updateBin.close();
            return;
        }

        size_t updateSize = updateBin.size();

        if (updateSize > 0)
        {
            Serial.println("Intentando comenzar Actualización");
            performUpdate(updateBin, updateSize);
        }
        else
        {
            Serial.println("Error, archivo vacío");
        }

        updateBin.close();

        // whe finished remove the binary from sd card to indicate end of the process
        //fs.remove("/update.bin");
    }
    else
    {
        Serial.println("No se puede cargar el archivo");
    }
}

void performUpdate(Stream &updateSource, size_t updateSize)
{
    if (Update.begin(updateSize))
    {
        size_t written = Update.writeStream(updateSource);
        if (written == updateSize)
        {
            Serial.println("Escritos : " + String(written) + " successfully");
        }
        else
        {
            Serial.println("Solamente escritos : " + String(written) + "/" + String(updateSize) + ". Retry?");
        }
        if (Update.end())
        {
            Serial.println("OTA realizado!");
            if (Update.isFinished())
            {
                Serial.println("Ota exitoso, reiniciando!");
                ESP.restart();
            }
            else
            {
                Serial.println("Ota no terminó? Algo salió mal!");
            }
        }
        else
        {
            Serial.println("Ocurrió Error #: " + String(Update.getError()));
        }
    }
    else
    {
        Serial.println("Sin espacio suficiente para hacer OTA");
    }
}

[vshymanskyy]

@mdc4u From the first glance, it looks like we can skip the FS part at all, and feed The GSM Client directly to the updater. But! I'll have to check several things ;) Thanks!

[ioticos]

@vshymanskyy Hello Yes, you are absolutely right, feed the updater, it would be great. Due to my little knowledge about the free updater, my direct feed attempts did not work. And what you saw is the only way I found to make it work. Totally inefficient by the way.

Well ... going back to the topic that brought us. At last I was able to make the client not disconnect after several hours of operation (sending GPS data) using sim800 ...

in my case the solution was simple. short cables and well welded. It's more ... now to make sure I've soldier what could be said ... pin with pin

now ... no more problems ... my suspicion started when I inspected the talk between the esp and the sim800 ...

the famous "?" they appeared from time to time, and that's when the disaster happened ...

then it appeared, my amateur radio spirit ... that has to be noise.

I quickly removed the cables and welded pin with pin

ready. in my case. the problem was solved

I hope it serves someone.

Thank you!!!!

Pablo

[vshymanskyy]

@kennythum there are multiple confiramtions that this is working. Closing the issue for now. Please reopen if still have issues, and report it to ESP32 core as well

[ankgt]

@mdc4u Thank you very much for the tip. I have managed to get the library working on Serial1!

[kamrulroky]

@enerbits OTA through GSM looks awesome, I am trying to do same thing on esp8266 (4mb flash chip version) with sim800l .the code is for esp32,and i am not sure for "update.h" what is the corresponding library on esp8266. will it work if i change file system objects for esp8266?

[vshymanskyy]

@kamrulroky probably won't work for ESP8266 this way. I'm preparing an article on how to make this working in an optimal way for ESP8266 and ESP32

[kamrulroky]

@vshymanskyy thank you, that would be very helpful

[mcr-ksh]

same here

[its-me-vicky]

@vshymanskyy

sorry for the untidiness I have quickly assembled this example to download .bin server saves it in spiffs and then apply ota.

this works without any problem.

where if I have a problem is to keep a tcp client connected, (in another part of my project of course nothing to do with OTA) in my .ino, check connection periodically as any typical tnygsm script, in case the socket has been closed, I try to connect again, if in several attempts does not happen. well, restart both esp and sim800, this, applied to an arduino nano or an arduino one, works well for me but, B U T this same using an esp32, becomes unstable when I move in a vehicle, is disconnected every time. This does not happen with a nano, what exactly brought me to esp32 is to be able to use their nucleuses separately, and the blessed OTA

What is promised is debt, here goes

#include <Update.h>
#define TINY_GSM_MODEM_SIM800
// Increase RX buffer
#define TINY_GSM_RX_BUFFER 1030

const char apn[] = "";
const char user[] = "";
const char pass[] = "";

HardwareSerial SerialAT(1);

#include <TinyGsmClient.h>
#include <CRC32.h>

#include "FS.h"
#include "SPIFFS.h"

#ifdef DUMP_AT_COMMANDS
#include <StreamDebugger.h>
StreamDebugger debugger(SerialAT, Serial);
TinyGsm modem(debugger);
#else
TinyGsm modem(SerialAT);
#endif
TinyGsmClient client(modem);

const char server[] = "here the server like sample.com.ar";
const int port = 80;

const char resource[] = "/updates/v1_0.bin"; //here de bin file

uint32_t knownCRC32 = 0x6f50d767;
uint32_t knownFileSize = 1024; // In case server does not send it

void setup()
{
    // Set console baud rate
    Serial.begin(115200);
    delay(10);

    if (!SPIFFS.begin(true))
    {
        Serial.println("SPIFFS Mount Failed");
        return;
    }
    SPIFFS.format();
    listDir(SPIFFS, "/", 0);

    // Set GSM module baud rate
    SerialAT.begin(57600);
    delay(3000);

    // Restart takes quite some time
    // To skip it, call init() instead of restart()
    Serial.println("Initializing modem...");
    modem.restart();

    String modemInfo = modem.getModemInfo();
    Serial.print("Modem: ");
    Serial.println(modemInfo);

    // Unlock your SIM card with a PIN
    //modem.simUnlock("1234");
}

void printPercent(uint32_t readLength, uint32_t contentLength)
{
    // If we know the total length
    if (contentLength != -1)
    {
        Serial.print("\r ");
        Serial.print((100.0 * readLength) / contentLength);
        Serial.print('%');
    }
    else
    {
        Serial.println(readLength);
    }
}

void loop()
{
    Serial.print("Waiting for network...");
    if (!modem.waitForNetwork())
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");

    Serial.print("Connecting to ");
    Serial.print(apn);
    if (!modem.gprsConnect(apn, user, pass))
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");

    Serial.print("Connecting to ");
    Serial.print(server);

    // if you get a connection, report back via serial:
    if (!client.connect(server, port))
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");
    // Make a HTTP request:
    client.print(String("GET ") + resource + " HTTP/1.0\r\n");
    client.print(String("Host: ") + server + "\r\n");
    client.print("Connection: close\r\n\r\n");

    long timeout = millis();
    while (client.available() == 0)
    {
        if (millis() - timeout > 5000L)
        {
            Serial.println(">>> Client Timeout !");
            client.stop();
            delay(10000L);
            return;
        }
    }

    Serial.println("Recibiendo Header");
    uint32_t contentLength = knownFileSize;

    File file = SPIFFS.open("/update.bin", FILE_APPEND);

    while (client.available())
    {
        String line = client.readStringUntil('\n');
        line.trim();
        //Serial.println(line);    // Uncomment this to show response header
        line.toLowerCase();
        if (line.startsWith("content-length:"))
        {
            contentLength = line.substring(line.lastIndexOf(':') + 1).toInt();
        }
        else if (line.length() == 0)
        {
            break;
        }
    }

    Serial.println("Recibiendo respuesta");
    timeout = millis();
    uint32_t readLength = 0;
    CRC32 crc;

    unsigned long timeElapsed = millis();
    printPercent(readLength, contentLength);

    while (readLength < contentLength && client.connected() && millis() - timeout < 10000L)
    {
        int i = 0;
        while (client.available())
        {

            if (!file.print(char(client.read())))
            {
                Serial.println("Fallo Append");
            }
            //Serial.print((char)c);       // Uncomment this to show data
            //crc.update(c);
            readLength++;

            if (readLength % (contentLength / 13) == 0)
            {
                printPercent(readLength, contentLength);
            }
            timeout = millis();
        }
    }

    file.close();

    printPercent(readLength, contentLength);
    timeElapsed = millis() - timeElapsed;
    Serial.println();

    client.stop();
    Serial.println("Desconectado del Servidor");

    modem.gprsDisconnect();
    Serial.println("GPRS Desconectado");
    Serial.println();

    float duration = float(timeElapsed) / 1000;

    Serial.print("Tamaño de Archivo: ");
    Serial.println(contentLength);
    Serial.print("Leido:  ");
    Serial.println(readLength);
    Serial.print("Calculado. CRC32:    0x");
    Serial.println(crc.finalize(), HEX);
    Serial.print("Conocido CRC32:    0x");
    Serial.println(knownCRC32, HEX);
    Serial.print("Bajado en:       ");
    Serial.print(duration);
    Serial.println("s");

    Serial.println("Se genera una espera de 3 segundos");
    for (int i = 0; i < 3; i++)
    {
        Serial.print(String(i) + "...");
        delay(1000);
    }

    //readFile(SPIFFS, "/update.bin");

    updateFromFS();

    // Do nothing forevermore
    while (true)
    {
        delay(1000);
    }
}

void appendFile(fs::FS &fs, const char *path, const char *message)
{
    Serial.printf("Appending to file: %s\n", path);

    File file = fs.open(path, FILE_APPEND);
    if (!file)
    {
        Serial.println("Failed to open file for appending");
        return;
    }
    if (file.print(message))
    {
        Serial.println("APOK");
    }
    else
    {
        Serial.println("APX");
    }
}

void readFile(fs::FS &fs, const char *path)
{
    Serial.printf("Reading file: %s\n", path);

    File file = fs.open(path);
    if (!file || file.isDirectory())
    {
        Serial.println("Failed to open file for reading");
        return;
    }

    Serial.print("Read from file: ");
    while (file.available())
    {
        Serial.write(file.read());
        delayMicroseconds(100);
    }
}

void writeFile(fs::FS &fs, const char *path, const char *message)
{
    Serial.printf("Writing file: %s\n", path);

    File file = fs.open(path, FILE_WRITE);
    if (!file)
    {
        Serial.println("Failed to open file for writing");
        return;
    }
    if (file.print(message))
    {
        Serial.println("File written");
    }
    else
    {
        Serial.println("Write failed");
    }
}

void listDir(fs::FS &fs, const char *dirname, uint8_t levels)
{
    Serial.printf("Listing directory: %s\n", dirname);

    File root = fs.open(dirname);
    if (!root)
    {
        Serial.println("Failed to open directory");
        return;
    }
    if (!root.isDirectory())
    {
        Serial.println("Not a directory");
        return;
    }

    File file = root.openNextFile();
    while (file)
    {
        if (file.isDirectory())
        {
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if (levels)
            {
                listDir(fs, file.name(), levels - 1);
            }
        }
        else
        {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("  SIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}

void deleteFile(fs::FS &fs, const char *path)
{
    Serial.printf("Deleting file: %s\n", path);
    if (fs.remove(path))
    {
        Serial.println("File deleted");
    }
    else
    {
        Serial.println("Delete failed");
    }
}

void updateFromFS()
{
    File updateBin = SPIFFS.open("/update.bin");
    if (updateBin)
    {
        if (updateBin.isDirectory())
        {
            Serial.println("Error, en el directorio");
            updateBin.close();
            return;
        }

        size_t updateSize = updateBin.size();

        if (updateSize > 0)
        {
            Serial.println("Intentando comenzar Actualización");
            performUpdate(updateBin, updateSize);
        }
        else
        {
            Serial.println("Error, archivo vacío");
        }

        updateBin.close();

        // whe finished remove the binary from sd card to indicate end of the process
        //fs.remove("/update.bin");
    }
    else
    {
        Serial.println("No se puede cargar el archivo");
    }
}

void performUpdate(Stream &updateSource, size_t updateSize)
{
    if (Update.begin(updateSize))
    {
        size_t written = Update.writeStream(updateSource);
        if (written == updateSize)
        {
            Serial.println("Escritos : " + String(written) + " successfully");
        }
        else
        {
            Serial.println("Solamente escritos : " + String(written) + "/" + String(updateSize) + ". Retry?");
        }
        if (Update.end())
        {
            Serial.println("OTA realizado!");
            if (Update.isFinished())
            {
                Serial.println("Ota exitoso, reiniciando!");
                ESP.restart();
            }
            else
            {
                Serial.println("Ota no terminó? Algo salió mal!");
            }
        }
        else
        {
            Serial.println("Ocurrió Error #: " + String(Update.getError()));
        }
    }
    else
    {
        Serial.println("Sin espacio suficiente para hacer OTA");
    }
}

hello, Which controller is suitable for this program. I am using ESP8266

[Black3rror]

@enerbits thanx for code sharing. I have a code for those who wants to have @enerbits code on ESP8266. and then a question. here is the code:

/**************************************************************
 *
 * For this example, you need to install CRC32 library:
 *   https://github.com/bakercp/CRC32
 *   or from http://librarymanager/all#CRC32+checksum
 *
 * TinyGSM Getting Started guide:
 *   http://tiny.cc/tiny-gsm-readme
 *
 * ATTENTION! Downloading big files requires of knowledge of
 * the TinyGSM internals and some modem specifics,
 * so this is for more experienced developers.
 *
 **************************************************************/

// Select your modem:
#define TINY_GSM_MODEM_SIM800
// #define TINY_GSM_MODEM_SIM808
// #define TINY_GSM_MODEM_SIM900
// #define TINY_GSM_MODEM_A6
// #define TINY_GSM_MODEM_A7
// #define TINY_GSM_MODEM_M590
// #define TINY_GSM_MODEM_ESP8266
// #define TINY_GSM_MODEM_XBEE

// Increase RX buffer if needed
#define TINY_GSM_RX_BUFFER 1024

#include <FS.h>
#include <TinyGsmClient.h>
#include <CRC32.h>

// Uncomment this if you want to see all AT commands
//#define DUMP_AT_COMMANDS

// Set serial for debug console (to the Serial Monitor, default speed 115200)
#define SerialMon Serial

// Use Hardware Serial on Mega, Leonardo, Micro
//#define SerialAT Serial1

// or Software Serial on Uno, Nano
#include <SoftwareSerial.h>
SoftwareSerial SerialAT(D2, D1); // RX, TX

// Your GPRS credentials
// Leave empty, if missing user or pass
const char apn[]  = "mtnirancell";  // replace with your apn
const char user[] = "";
const char pass[] = "";

// Server details
const char server[] = "uupload.ir"; // replace with your server name
const int  port = 80;

const char resource[]  = "/filelink/w93wgDpMMMKg/p6nv_gateway.bin"; // my program is a simple blink on LED_BUILTIN pin, so u can use it as well
uint32_t knownCRC32    = 0x6f50d767;
uint32_t knownFileSize = 1024;   // In case server does not send it

#ifdef DUMP_AT_COMMANDS
  #include <StreamDebugger.h>
  StreamDebugger debugger(SerialAT, SerialMon);
  TinyGsm modem(debugger);
#else
  TinyGsm modem(SerialAT);
#endif

TinyGsmClient client(modem);

void setup() {
  // Set console baud rate
  SerialMon.begin(115200);
  delay(10);
  Serial.println("Start of sketch ...");

  if (!SPIFFS.begin())
  {
      Serial.println("SPIFFS Mount Failed");
      return;
  }
  SPIFFS.format();
//  listDir(SPIFFS, "/", 0);

  // Set GSM module baud rate
  SerialAT.begin(9600);
  delay(3000);

  // Restart takes quite some time
  // To skip it, call init() instead of restart()
  SerialMon.println(F("Initializing modem..."));
  modem.restart();

  String modemInfo = modem.getModemInfo();
  SerialMon.print(F("Modem: "));
  SerialMon.println(modemInfo);

  // Unlock your SIM card with a PIN
  //modem.simUnlock("1234");
}

void printPercent(uint32_t readLength, uint32_t contentLength) {
  // If we know the total length
  if (contentLength != -1) {
    SerialMon.print("\r ");
    SerialMon.print((100.0 * readLength) / contentLength);
    SerialMon.print('%');
  } else {
    SerialMon.println(readLength);
  }
}

void loop() {
  SerialMon.print(F("Waiting for network..."));
  if (!modem.waitForNetwork()) {
    SerialMon.println(" fail");
    delay(10000);
    return;
  }
  SerialMon.println(" OK");

  SerialMon.print(F("Connecting to "));
  SerialMon.print(apn);
  if (!modem.gprsConnect(apn, user, pass)) {
    SerialMon.println(" fail");
    delay(10000);
    return;
  }
  SerialMon.println(" OK");

  SerialMon.print(F("Connecting to "));
  SerialMon.print(server);
  if (!client.connect(server, port)) {
    SerialMon.println(" fail");
    delay(10000);
    return;
  }
  SerialMon.println(" OK");

  // Make a HTTP GET request:
  client.print(String("GET ") + resource + " HTTP/1.0\r\n");
  client.print(String("Host: ") + server + "\r\n");
  client.print("Connection: close\r\n\r\n");

  long timeout = millis();
  while (client.available() == 0) {
    if (millis() - timeout > 5000L) {
      SerialMon.println(F(">>> Client Timeout !"));
      client.stop();
      delay(10000L);
      return;
    }
  }

  SerialMon.println(F("Reading response header"));
  uint32_t contentLength = knownFileSize;

  File file = SPIFFS.open("/update.bin", "a+");

  while (client.available()) {
    String line = client.readStringUntil('\n');
    line.trim();
    SerialMon.println(line);    // Uncomment this to show response header
    line.toLowerCase();
    if (line.startsWith("content-length:")) {
      contentLength = line.substring(line.lastIndexOf(':') + 1).toInt();
    } else if (line.length() == 0) {
      break;
    }
  }

  SerialMon.println(F("Reading response data"));
  timeout = millis();
  uint32_t readLength = 0;
  CRC32 crc;

  unsigned long timeElapsed = millis();
  printPercent(readLength, contentLength);
  while (readLength < contentLength && client.connected() /*&& millis() - timeout < 10000L*/) {
    int i = 0;
    while (client.available()) {
//      uint8_t c = client.read();
//      SerialMon.print((char)c);       // Uncomment this to show data
      if (!file.print(char(client.read())))
      {
          Serial.println("error writing character to SPIFFS");
      }
//      crc.update(c);
      readLength++;
      if (readLength % (contentLength / 13) == 0) {
        printPercent(readLength, contentLength);
      }
      timeout = millis();
    }
  }
  printPercent(readLength, contentLength);
  timeElapsed = millis() - timeElapsed;
  SerialMon.println();

  // Shutdown

  client.stop();
  SerialMon.println(F("Server disconnected"));

  modem.gprsDisconnect();
  SerialMon.println(F("GPRS disconnected"));

  float duration = float(timeElapsed) / 1000;

  SerialMon.println();
  SerialMon.print("Content-Length: ");   SerialMon.println(contentLength);
  SerialMon.print("Actually read:  ");   SerialMon.println(readLength);
  SerialMon.print("Calc. CRC32:    0x"); SerialMon.println(crc.finalize(), HEX);
  SerialMon.print("Known CRC32:    0x"); SerialMon.println(knownCRC32, HEX);
  SerialMon.print("Duration:       ");   SerialMon.print(duration); SerialMon.println("s");

  Serial.println("starting Update after 3 seconds  ");
  for (int i = 0; i < 3; i++)
  {
      Serial.print(String(i) + "...");
      delay(1000);
  }

  //readFile(SPIFFS, "/update.bin");

  updateFromFS();

//  if(md5.length()) {
//      if(!Update.setMD5(md5.c_str())) {
//          _lastError = HTTP_UE_SERVER_FAULTY_MD5;
//          DEBUG_HTTP_UPDATE("[httpUpdate] Update.setMD5 failed! (%s)\n", md5.c_str());
//          return false;
//      }
//  }

  // Do nothing forevermore
  while (true) {
    delay(1000);
  }
}

void updateFromFS()
{
    File updateBin = SPIFFS.open("/update.bin", "r");
    if (updateBin)
    {
//        if (updateBin.isDirectory())
//        {
//            Serial.println("Error, en el directorio");
//            updateBin.close();
//            return;
//        }

        size_t updateSize = updateBin.size();

        if (updateSize > 0)
        {
            Serial.println("start of updating");
            performUpdate(updateBin, updateSize);
        }
        else
        {
            Serial.println("Error, file is empty");
        }

        updateBin.close();

        // whe finished remove the binary from sd card to indicate end of the process
        //fs.remove("/update.bin");
    }
    else
    {
        Serial.println("can't open update.bin");
    }
}

void performUpdate(Stream &updateSource, size_t updateSize)
{
    if (Update.begin(updateSize))
    {
        size_t written = Update.writeStream(updateSource);
        if (written == updateSize)
        {
            Serial.println("writings : " + String(written) + " successfully");
        }
        else
        {
            Serial.println("only writing : " + String(written) + "/" + String(updateSize) + ". Retry?");
        }
        if (Update.end())
        {
            Serial.println("OTA accomplished!");
            if (Update.isFinished())
            {
                Serial.println("OTA ended. restarting!");
                ESP.restart();
            }
            else
            {
                Serial.println("OTA didn't finish? something went wrong!");
            }
        }
        else
        {
            Serial.println("Error occured #: " + String(Update.getError()));
        }
    }
    else
    {
        Serial.println("without enough space to do OTA");
    }
}

I had problem downloading the file. for me it takes about 3 to 4 minutes to download it so after about 80% I think it exceeds the time limit of connection and disconnects from server. so I can't download it anyway. but I've tested it by uploading update.bin file to SPIFFS directly from my computer and then trying updateFromFS(). result: it worked. so probably complete code also works.

now my question is: does the connection really have a time limit? how can I change it?

and also if u used the code above, I'll be glad to know the result.

[its-me-vicky]

Thank you for your response. Yes the HTTP has connection time. Please refer this article - https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Keep-Alive.

And in my case I used wemos D1 R1 development board and SIM800 GSM Module, the TinyGSM library is not working properly. AT Commands are not passing to the module. Any idea about it?

On Mon, Nov 26, 2018 at 4:56 AM Black3rror notifications@github.com wrote:

@enerbits https://github.com/enerbits thanx for code sharing. I have a code for those who wants to have @enerbits https://github.com/enerbits code on ESP8266. and then a question. here is the code:

/** For this example, you need to install CRC32 library: https://github.com/bakercp/CRC32 or from http://librarymanager/all#CRC32+checksum TinyGSM Getting Started guide: http://tiny.cc/tiny-gsm-readme ATTENTION! Downloading big files requires of knowledge of the TinyGSM internals and some modem specifics, so this is for more experienced developers. **/ // Select your modem:

define TINY_GSM_MODEM_SIM800// #define TINY_GSM_MODEM_SIM808// #define TINY_GSM_MODEM_SIM900// #define TINY_GSM_MODEM_A6// #define TINY_GSM_MODEM_A7// #define TINY_GSM_MODEM_M590// #define TINY_GSM_MODEM_ESP8266// #define TINY_GSM_MODEM_XBEE

// Increase RX buffer if needed

define TINY_GSM_RX_BUFFER 1024

include

include

include

// Uncomment this if you want to see all AT commands//#define DUMP_AT_COMMANDS // Set serial for debug console (to the Serial Monitor, default speed 115200)

define SerialMon Serial

// Use Hardware Serial on Mega, Leonardo, Micro//#define SerialAT Serial1 // or Software Serial on Uno, Nano

include

SoftwareSerial SerialAT(D2, D1); // RX, TX

// Your GPRS credentials// Leave empty, if missing user or passconst char apn[] = "mtnirancell"; // replace with your apnconst char user[] = "";const char pass[] = ""; // Server detailsconst char server[] = "uupload.ir"; // replace with your server nameconst int port = 80; const char resource[] = "/filelink/w93wgDpMMMKg/p6nv_gateway.bin"; // my program is a simple blink on LED_BUILTIN pin, so u can use it as welluint32_t knownCRC32 = 0x6f50d767;uint32_t knownFileSize = 1024; // In case server does not send it

ifdef DUMP_AT_COMMANDS

include

StreamDebugger debugger(SerialAT, SerialMon); TinyGsm modem(debugger);

else

TinyGsm modem(SerialAT);

endif

TinyGsmClient client(modem); void setup() { // Set console baud rate SerialMon.begin(115200); delay(10); Serial.println("Start of sketch ...");

if (!SPIFFS.begin()) { Serial.println("SPIFFS Mount Failed"); return; } SPIFFS.format();// listDir(SPIFFS, "/", 0);

// Set GSM module baud rate SerialAT.begin(9600); delay(3000);

// Restart takes quite some time // To skip it, call init() instead of restart() SerialMon.println(F("Initializing modem...")); modem.restart();

String modemInfo = modem.getModemInfo(); SerialMon.print(F("Modem: ")); SerialMon.println(modemInfo);

// Unlock your SIM card with a PIN //modem.simUnlock("1234"); } void printPercent(uint32_t readLength, uint32_t contentLength) { // If we know the total length if (contentLength != -1) { SerialMon.print("\r "); SerialMon.print((100.0 * readLength) / contentLength); SerialMon.print('%'); } else { SerialMon.println(readLength); } } void loop() { SerialMon.print(F("Waiting for network...")); if (!modem.waitForNetwork()) { SerialMon.println(" fail"); delay(10000); return; } SerialMon.println(" OK");

SerialMon.print(F("Connecting to ")); SerialMon.print(apn); if (!modem.gprsConnect(apn, user, pass)) { SerialMon.println(" fail"); delay(10000); return; } SerialMon.println(" OK");

SerialMon.print(F("Connecting to ")); SerialMon.print(server); if (!client.connect(server, port)) { SerialMon.println(" fail"); delay(10000); return; } SerialMon.println(" OK");

// Make a HTTP GET request: client.print(String("GET ") + resource + " HTTP/1.0\r\n"); client.print(String("Host: ") + server + "\r\n"); client.print("Connection: close\r\n\r\n");

long timeout = millis(); while (client.available() == 0) { if (millis() - timeout > 5000L) { SerialMon.println(F(">>> Client Timeout !")); client.stop(); delay(10000L); return; } }

SerialMon.println(F("Reading response header")); uint32_t contentLength = knownFileSize;

File file = SPIFFS.open("/update.bin", "a+");

while (client.available()) { String line = client.readStringUntil('\n'); line.trim(); SerialMon.println(line); // Uncomment this to show response header line.toLowerCase(); if (line.startsWith("content-length:")) { contentLength = line.substring(line.lastIndexOf(':') + 1).toInt(); } else if (line.length() == 0) { break; } }

SerialMon.println(F("Reading response data")); timeout = millis(); uint32_t readLength = 0; CRC32 crc;

unsigned long timeElapsed = millis(); printPercent(readLength, contentLength); while (readLength < contentLength && client.connected() /&& millis() - timeout < 10000L/) { int i = 0; while (client.available()) {// uint8_t c = client.read();// SerialMon.print((char)c); // Uncomment this to show data if (!file.print(char(client.read()))) { Serial.println("error writing character to SPIFFS"); }// crc.update(c); readLength++; if (readLength % (contentLength / 13) == 0) { printPercent(readLength, contentLength); } timeout = millis(); } } printPercent(readLength, contentLength); timeElapsed = millis() - timeElapsed; SerialMon.println();

// Shutdown

client.stop(); SerialMon.println(F("Server disconnected"));

modem.gprsDisconnect(); SerialMon.println(F("GPRS disconnected"));

float duration = float(timeElapsed) / 1000;

SerialMon.println(); SerialMon.print("Content-Length: "); SerialMon.println(contentLength); SerialMon.print("Actually read: "); SerialMon.println(readLength); SerialMon.print("Calc. CRC32: 0x"); SerialMon.println(crc.finalize(), HEX); SerialMon.print("Known CRC32: 0x"); SerialMon.println(knownCRC32, HEX); SerialMon.print("Duration: "); SerialMon.print(duration); SerialMon.println("s");

Serial.println("starting Update after 3 seconds "); for (int i = 0; i < 3; i++) { Serial.print(String(i) + "..."); delay(1000); }

//readFile(SPIFFS, "/update.bin");

updateFromFS(); // if(md5.length()) {// if(!Update.setMD5(md5.c_str())) {// _lastError = HTTP_UE_SERVER_FAULTY_MD5;// DEBUG_HTTP_UPDATE("[httpUpdate] Update.setMD5 failed! (%s)\n", md5.c_str());// return false;// }// }

// Do nothing forevermore while (true) { delay(1000); } }

void updateFromFS() { File updateBin = SPIFFS.open("/update.bin", "r"); if (updateBin) {// if (updateBin.isDirectory())// {// Serial.println("Error, en el directorio");// updateBin.close();// return;// }

    size_t updateSize = updateBin.size();

    if (updateSize > 0)
    {
        Serial.println("start of updating");
        performUpdate(updateBin, updateSize);
    }
    else
    {
        Serial.println("Error, file is empty");
    }

    updateBin.close();

    // whe finished remove the binary from sd card to indicate end of the process
    //fs.remove("/update.bin");
}
else
{
    Serial.println("can't open update.bin");
}

}

void performUpdate(Stream &updateSource, size_t updateSize) { if (Update.begin(updateSize)) { size_t written = Update.writeStream(updateSource); if (written == updateSize) { Serial.println("writings : " + String(written) + " successfully"); } else { Serial.println("only writing : " + String(written) + "/" + String(updateSize) + ". Retry?"); } if (Update.end()) { Serial.println("OTA accomplished!"); if (Update.isFinished()) { Serial.println("OTA ended. restarting!"); ESP.restart(); } else { Serial.println("OTA didn't finish? something went wrong!"); } } else { Serial.println("Error occured #: " + String(Update.getError())); } } else { Serial.println("without enough space to do OTA"); } }

I had problem downloading the file. for me it takes about 3 to 4 minutes to download it so after about 80% I think it exceeds the time limit of connection and disconnects from server. so I can't download it anyway. but I've tested it by uploading update.bin file to SPIFFS directly from my computer and then trying updateFromFS(). result: it worked. so probably complete code also works.

now my question is: does the connection really have a time limit? how can I change it?

and also if u used the code above, I'll be glad to know the result.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/vshymanskyy/TinyGSM/issues/133#issuecomment-441483570, or mute the thread https://github.com/notifications/unsubscribe-auth/AY6SLRjzXd5VqbIbVfaknlEX8DlgotTqks5uyyc3gaJpZM4SRxRD .

[Black3rror]

Thank you for your response. Yes the HTTP has connection time. Please refer this article - https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Keep-Alive. And in my case I used wemos D1 R1 development board and SIM800 GSM Module, the TinyGSM library is not working properly. AT Commands are not passing to the module. Any idea about it? … On Mon, Nov 26, 2018 at 4:56 AM Black3rror @.*> wrote: @enerbits https://github.com/enerbits thanx for code sharing. I have a code for those who wants to have @enerbits https://github.com/enerbits code on ESP8266. and then a question. here is the code: /**** For this example, you need to install CRC32 library: https://github.com/bakercp/CRC32 or from http://librarymanager/all#CRC32+checksum TinyGSM Getting Started guide: http://tiny.cc/tiny-gsm-readme ATTENTION! Downloading big files requires of knowledge of the TinyGSM internals and some modem specifics, so this is for more experienced developers. **/ // Select your modem: #define TINY_GSM_MODEM_SIM800// #define TINY_GSM_MODEM_SIM808// #define TINY_GSM_MODEM_SIM900// #define TINY_GSM_MODEM_A6// #define TINY_GSM_MODEM_A7// #define TINY_GSM_MODEM_M590// #define TINY_GSM_MODEM_ESP8266// #define TINY_GSM_MODEM_XBEE // Increase RX buffer if needed #define TINY_GSM_RX_BUFFER 1024 #include #include #include // Uncomment this if you want to see all AT commands//#define DUMP_AT_COMMANDS // Set serial for debug console (to the Serial Monitor, default speed 115200) #define SerialMon Serial // Use Hardware Serial on Mega, Leonardo, Micro//#define SerialAT Serial1 // or Software Serial on Uno, Nano #include SoftwareSerial SerialAT(D2, D1); // RX, TX // Your GPRS credentials// Leave empty, if missing user or passconst char apn[] = "mtnirancell"; // replace with your apnconst char user[] = "";const char pass[] = ""; // Server detailsconst char server[] = "uupload.ir"; // replace with your server nameconst int port = 80; const char resource[] = "/filelink/w93wgDpMMMKg/p6nv_gateway.bin"; // my program is a simple blink on LED_BUILTIN pin, so u can use it as welluint32_t knownCRC32 = 0x6f50d767;uint32_t knownFileSize = 1024; // In case server does not send it #ifdef DUMP_AT_COMMANDS #include StreamDebugger debugger(SerialAT, SerialMon); TinyGsm modem(debugger); #else TinyGsm modem(SerialAT); #endif TinyGsmClient client(modem); void setup() { // Set console baud rate SerialMon.begin(115200); delay(10); Serial.println("Start of sketch ..."); if (!SPIFFS.begin()) { Serial.println("SPIFFS Mount Failed"); return; } SPIFFS.format();// listDir(SPIFFS, "/", 0); // Set GSM module baud rate SerialAT.begin(9600); delay(3000); // Restart takes quite some time // To skip it, call init() instead of restart() SerialMon.println(F("Initializing modem...")); modem.restart(); String modemInfo = modem.getModemInfo(); SerialMon.print(F("Modem: ")); SerialMon.println(modemInfo); // Unlock your SIM card with a PIN //modem.simUnlock("1234"); } void printPercent(uint32_t readLength, uint32_t contentLength) { // If we know the total length if (contentLength != -1) { SerialMon.print("\r "); SerialMon.print((100.0 readLength) / contentLength); SerialMon.print('%'); } else { SerialMon.println(readLength); } } void loop() { SerialMon.print(F("Waiting for network...")); if (!modem.waitForNetwork()) { SerialMon.println(" fail"); delay(10000); return; } SerialMon.println(" OK"); SerialMon.print(F("Connecting to ")); SerialMon.print(apn); if (!modem.gprsConnect(apn, user, pass)) { SerialMon.println(" fail"); delay(10000); return; } SerialMon.println(" OK"); SerialMon.print(F("Connecting to ")); SerialMon.print(server); if (!client.connect(server, port)) { SerialMon.println(" fail"); delay(10000); return; } SerialMon.println(" OK"); // Make a HTTP GET request: client.print(String("GET ") + resource + " HTTP/1.0\r\n"); client.print(String("Host: ") + server + "\r\n"); client.print("Connection: close\r\n\r\n"); long timeout = millis(); while (client.available() == 0) { if (millis() - timeout > 5000L) { SerialMon.println(F(">>> Client Timeout !")); client.stop(); delay(10000L); return; } } SerialMon.println(F("Reading response header")); uint32_t contentLength = knownFileSize; File file = SPIFFS.open("/update.bin", "a+"); while (client.available()) { String line = client.readStringUntil('\n'); line.trim(); SerialMon.println(line); // Uncomment this to show response header line.toLowerCase(); if (line.startsWith("content-length:")) { contentLength = line.substring(line.lastIndexOf(':') + 1).toInt(); } else if (line.length() == 0) { break; } } SerialMon.println(F("Reading response data")); timeout = millis(); uint32_t readLength = 0; CRC32 crc; unsigned long timeElapsed = millis(); printPercent(readLength, contentLength); while (readLength < contentLength && client.connected() /&& millis() - timeout < 10000L*/) { int i = 0; while (client.available()) {// uint8_t c = client.read();// SerialMon.print((char)c); // Uncomment this to show data if (!file.print(char(client.read()))) { Serial.println("error writing character to SPIFFS"); }// crc.update(c); readLength++; if (readLength % (contentLength / 13) == 0) { printPercent(readLength, contentLength); } timeout = millis(); } } printPercent(readLength, contentLength); timeElapsed = millis() - timeElapsed; SerialMon.println(); // Shutdown client.stop(); SerialMon.println(F("Server disconnected")); modem.gprsDisconnect(); SerialMon.println(F("GPRS disconnected")); float duration = float(timeElapsed) / 1000; SerialMon.println(); SerialMon.print("Content-Length: "); SerialMon.println(contentLength); SerialMon.print("Actually read: "); SerialMon.println(readLength); SerialMon.print("Calc. CRC32: 0x"); SerialMon.println(crc.finalize(), HEX); SerialMon.print("Known CRC32: 0x"); SerialMon.println(knownCRC32, HEX); SerialMon.print("Duration: "); SerialMon.print(duration); SerialMon.println("s"); Serial.println("starting Update after 3 seconds "); for (int i = 0; i < 3; i++) { Serial.print(String(i) + "..."); delay(1000); } //readFile(SPIFFS, "/update.bin"); updateFromFS(); // if(md5.length()) {// if(!Update.setMD5(md5.c_str())) {// _lastError = HTTP_UE_SERVER_FAULTY_MD5;// DEBUG_HTTP_UPDATE("[httpUpdate] Update.setMD5 failed! (%s)\n", md5.c_str());// return false;// }// } // Do nothing forevermore while (true) { delay(1000); } } void updateFromFS() { File updateBin = SPIFFS.open("/update.bin", "r"); if (updateBin) {// if (updateBin.isDirectory())// {// Serial.println("Error, en el directorio");// updateBin.close();// return;// } size_t updateSize = updateBin.size(); if (updateSize > 0) { Serial.println("start of updating"); performUpdate(updateBin, updateSize); } else { Serial.println("Error, file is empty"); } updateBin.close(); // whe finished remove the binary from sd card to indicate end of the process //fs.remove("/update.bin"); } else { Serial.println("can't open update.bin"); } } void performUpdate(Stream &updateSource, size_t updateSize) { if (Update.begin(updateSize)) { size_t written = Update.writeStream(updateSource); if (written == updateSize) { Serial.println("writings : " + String(written) + " successfully"); } else { Serial.println("only writing : " + String(written) + "/" + String(updateSize) + ". Retry?"); } if (Update.end()) { Serial.println("OTA accomplished!"); if (Update.isFinished()) { Serial.println("OTA ended. restarting!"); ESP.restart(); } else { Serial.println("OTA didn't finish? something went wrong!"); } } else { Serial.println("Error occured #: " + String(Update.getError())); } } else { Serial.println("without enough space to do OTA"); } } I had problem downloading the file. for me it takes about 3 to 4 minutes to download it so after about 80% I think it exceeds the time limit of connection and disconnects from server. so I can't download it anyway. but I've tested it by uploading update.bin file to SPIFFS directly from my computer and then trying updateFromFS(). result: it worked. so probably complete code also works. now my question is: does the connection really have a time limit? how can I change it? and also if u used the code above, I'll be glad to know the result. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#133 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AY6SLRjzXd5VqbIbVfaknlEX8DlgotTqks5uyyc3gaJpZM4SRxRD .

Thanx for response. at first I had the same problem. I mean sometimes my SIM800 module got "AT" correctly but sometimes got "?T" instead of that. I've decreased the baud rate from 115200 to 9600 and used a PCB instead of wiring and the problem gone. hope decreasing the baud rate and soldering instead of wiring on bread board solve your problem.

[Warlib1975]

Here is some description, how I used ESP32 with SIM800L http://www.bizkit.ru/en/2019/04/03/13540/ with HardwareSerial. Maybe will be useful. It's MT, I haven't corrected yet the quality of translation, sorry.

[tarcisocallegario]

@vshymanskyy Hello Yes, you are absolutely right, feed the updater, it would be great. Due to my little knowledge about the free updater, my direct feed attempts did not work. And what you saw is the only way I found to make it work. Totally inefficient by the way.

Well ... going back to the topic that brought us. At last I was able to make the client not disconnect after several hours of operation (sending GPS data) using sim800 ...

in my case the solution was simple. short cables and well welded. It's more ... now to make sure I've soldier what could be said ... pin with pin

now ... no more problems ... my suspicion started when I inspected the talk between the esp and the sim800 ...

the famous "?" they appeared from time to time, and that's when the disaster happened ...

then it appeared, my amateur radio spirit ... that has to be noise.

I quickly removed the cables and welded pin with pin

ready. in my case. the problem was solved

I hope it serves someone.

Thank you!!!!

Pablo

Hi, Pablo Have you done any upgrade in your OTAcode ? Best Regards !

[elyasssmart]

@enerbits thanx for code sharing. I have a code for those who wants to have @enerbits code on ESP8266. and then a question. here is the code:

/**************************************************************
 *
 * For this example, you need to install CRC32 library:
 *   https://github.com/bakercp/CRC32
 *   or from http://librarymanager/all#CRC32+checksum
 *
 * TinyGSM Getting Started guide:
 *   http://tiny.cc/tiny-gsm-readme
 *
 * ATTENTION! Downloading big files requires of knowledge of
 * the TinyGSM internals and some modem specifics,
 * so this is for more experienced developers.
 *
 **************************************************************/

// Select your modem:
#define TINY_GSM_MODEM_SIM800
// #define TINY_GSM_MODEM_SIM808
// #define TINY_GSM_MODEM_SIM900
// #define TINY_GSM_MODEM_A6
// #define TINY_GSM_MODEM_A7
// #define TINY_GSM_MODEM_M590
// #define TINY_GSM_MODEM_ESP8266
// #define TINY_GSM_MODEM_XBEE

// Increase RX buffer if needed
#define TINY_GSM_RX_BUFFER 1024

#include <FS.h>
#include <TinyGsmClient.h>
#include <CRC32.h>

// Uncomment this if you want to see all AT commands
//#define DUMP_AT_COMMANDS

// Set serial for debug console (to the Serial Monitor, default speed 115200)
#define SerialMon Serial

// Use Hardware Serial on Mega, Leonardo, Micro
//#define SerialAT Serial1

// or Software Serial on Uno, Nano
#include <SoftwareSerial.h>
SoftwareSerial SerialAT(D2, D1); // RX, TX

// Your GPRS credentials
// Leave empty, if missing user or pass
const char apn[]  = "mtnirancell";  // replace with your apn
const char user[] = "";
const char pass[] = "";

// Server details
const char server[] = "uupload.ir"; // replace with your server name
const int  port = 80;

const char resource[]  = "/filelink/w93wgDpMMMKg/p6nv_gateway.bin"; // my program is a simple blink on LED_BUILTIN pin, so u can use it as well
uint32_t knownCRC32    = 0x6f50d767;
uint32_t knownFileSize = 1024;   // In case server does not send it

#ifdef DUMP_AT_COMMANDS
  #include <StreamDebugger.h>
  StreamDebugger debugger(SerialAT, SerialMon);
  TinyGsm modem(debugger);
#else
  TinyGsm modem(SerialAT);
#endif

TinyGsmClient client(modem);

void setup() {
  // Set console baud rate
  SerialMon.begin(115200);
  delay(10);
  Serial.println("Start of sketch ...");

  if (!SPIFFS.begin())
  {
      Serial.println("SPIFFS Mount Failed");
      return;
  }
  SPIFFS.format();
//  listDir(SPIFFS, "/", 0);

  // Set GSM module baud rate
  SerialAT.begin(9600);
  delay(3000);

  // Restart takes quite some time
  // To skip it, call init() instead of restart()
  SerialMon.println(F("Initializing modem..."));
  modem.restart();

  String modemInfo = modem.getModemInfo();
  SerialMon.print(F("Modem: "));
  SerialMon.println(modemInfo);

  // Unlock your SIM card with a PIN
  //modem.simUnlock("1234");
}

void printPercent(uint32_t readLength, uint32_t contentLength) {
  // If we know the total length
  if (contentLength != -1) {
    SerialMon.print("\r ");
    SerialMon.print((100.0 * readLength) / contentLength);
    SerialMon.print('%');
  } else {
    SerialMon.println(readLength);
  }
}

void loop() {
  SerialMon.print(F("Waiting for network..."));
  if (!modem.waitForNetwork()) {
    SerialMon.println(" fail");
    delay(10000);
    return;
  }
  SerialMon.println(" OK");

  SerialMon.print(F("Connecting to "));
  SerialMon.print(apn);
  if (!modem.gprsConnect(apn, user, pass)) {
    SerialMon.println(" fail");
    delay(10000);
    return;
  }
  SerialMon.println(" OK");

  SerialMon.print(F("Connecting to "));
  SerialMon.print(server);
  if (!client.connect(server, port)) {
    SerialMon.println(" fail");
    delay(10000);
    return;
  }
  SerialMon.println(" OK");

  // Make a HTTP GET request:
  client.print(String("GET ") + resource + " HTTP/1.0\r\n");
  client.print(String("Host: ") + server + "\r\n");
  client.print("Connection: close\r\n\r\n");

  long timeout = millis();
  while (client.available() == 0) {
    if (millis() - timeout > 5000L) {
      SerialMon.println(F(">>> Client Timeout !"));
      client.stop();
      delay(10000L);
      return;
    }
  }

  SerialMon.println(F("Reading response header"));
  uint32_t contentLength = knownFileSize;

  File file = SPIFFS.open("/update.bin", "a+");

  while (client.available()) {
    String line = client.readStringUntil('\n');
    line.trim();
    SerialMon.println(line);    // Uncomment this to show response header
    line.toLowerCase();
    if (line.startsWith("content-length:")) {
      contentLength = line.substring(line.lastIndexOf(':') + 1).toInt();
    } else if (line.length() == 0) {
      break;
    }
  }

  SerialMon.println(F("Reading response data"));
  timeout = millis();
  uint32_t readLength = 0;
  CRC32 crc;

  unsigned long timeElapsed = millis();
  printPercent(readLength, contentLength);
  while (readLength < contentLength && client.connected() /*&& millis() - timeout < 10000L*/) {
    int i = 0;
    while (client.available()) {
//      uint8_t c = client.read();
//      SerialMon.print((char)c);       // Uncomment this to show data
      if (!file.print(char(client.read())))
      {
          Serial.println("error writing character to SPIFFS");
      }
//      crc.update(c);
      readLength++;
      if (readLength % (contentLength / 13) == 0) {
        printPercent(readLength, contentLength);
      }
      timeout = millis();
    }
  }
  printPercent(readLength, contentLength);
  timeElapsed = millis() - timeElapsed;
  SerialMon.println();

  // Shutdown

  client.stop();
  SerialMon.println(F("Server disconnected"));

  modem.gprsDisconnect();
  SerialMon.println(F("GPRS disconnected"));

  float duration = float(timeElapsed) / 1000;

  SerialMon.println();
  SerialMon.print("Content-Length: ");   SerialMon.println(contentLength);
  SerialMon.print("Actually read:  ");   SerialMon.println(readLength);
  SerialMon.print("Calc. CRC32:    0x"); SerialMon.println(crc.finalize(), HEX);
  SerialMon.print("Known CRC32:    0x"); SerialMon.println(knownCRC32, HEX);
  SerialMon.print("Duration:       ");   SerialMon.print(duration); SerialMon.println("s");

  Serial.println("starting Update after 3 seconds  ");
  for (int i = 0; i < 3; i++)
  {
      Serial.print(String(i) + "...");
      delay(1000);
  }

  //readFile(SPIFFS, "/update.bin");

  updateFromFS();

//  if(md5.length()) {
//      if(!Update.setMD5(md5.c_str())) {
//          _lastError = HTTP_UE_SERVER_FAULTY_MD5;
//          DEBUG_HTTP_UPDATE("[httpUpdate] Update.setMD5 failed! (%s)\n", md5.c_str());
//          return false;
//      }
//  }

  // Do nothing forevermore
  while (true) {
    delay(1000);
  }
}

void updateFromFS()
{
    File updateBin = SPIFFS.open("/update.bin", "r");
    if (updateBin)
    {
//        if (updateBin.isDirectory())
//        {
//            Serial.println("Error, en el directorio");
//            updateBin.close();
//            return;
//        }

        size_t updateSize = updateBin.size();

        if (updateSize > 0)
        {
            Serial.println("start of updating");
            performUpdate(updateBin, updateSize);
        }
        else
        {
            Serial.println("Error, file is empty");
        }

        updateBin.close();

        // whe finished remove the binary from sd card to indicate end of the process
        //fs.remove("/update.bin");
    }
    else
    {
        Serial.println("can't open update.bin");
    }
}

void performUpdate(Stream &updateSource, size_t updateSize)
{
    if (Update.begin(updateSize))
    {
        size_t written = Update.writeStream(updateSource);
        if (written == updateSize)
        {
            Serial.println("writings : " + String(written) + " successfully");
        }
        else
        {
            Serial.println("only writing : " + String(written) + "/" + String(updateSize) + ". Retry?");
        }
        if (Update.end())
        {
            Serial.println("OTA accomplished!");
            if (Update.isFinished())
            {
                Serial.println("OTA ended. restarting!");
                ESP.restart();
            }
            else
            {
                Serial.println("OTA didn't finish? something went wrong!");
            }
        }
        else
        {
            Serial.println("Error occured #: " + String(Update.getError()));
        }
    }
    else
    {
        Serial.println("without enough space to do OTA");
    }
}

I had problem downloading the file. for me it takes about 3 to 4 minutes to download it so after about 80% I think it exceeds the time limit of connection and disconnects from server. so I can't download it anyway. but I've tested it by uploading update.bin file to SPIFFS directly from my computer and then trying updateFromFS(). result: it worked. so probably complete code also works.

now my question is: does the connection really have a time limit? how can I change it?

and also if u used the code above, I'll be glad to know the result.

Hey Dear, tnx for code i use for My Wemos D1 and work..

[rjjimenezl]

Hi, can someone help me with this problem I'm having? I'm using ESP32-WROOM-32U, SIM 808 EVB-V3.2 UART HardwareSerial communication I'm trying to send GPS data to an account in ubidots but I get this error, I've read it Almost everything and nothing has solved this problem for me, despite the fact that in the past I was able to run the same project without any problem, but now that I want to do it again, I can't because of this error. I would appreciate the help in advance. that line that says: getDateTimeStampGSM_modem() It is the last function that it enters before throwing the error.

I attach the complete function:

long getDateTimeStampGSM_modem() { //Obtém string com os dados de localização String loc = modem.getGsmLocation(); Serial.println("getGsmLocation()"); String dateTime; int timestamp; Serial.println("timestamp"); //Obtém substring com a data e hora getDateTime_GSMBuffer(loc, &dateTime);
//Calcula o timestamp baseado na data/hora timestamp = timeStamp_GSMBuffer(dateTime);

if(timestamp == -1) Serial.println("Failed to obtain timestamp (GSM): "+dateTime);

return timestamp; }