wrong result of testConnection() (sometimes)

[bosoft-ESP]

Hello After a long time using CTBot, I am finding that I am having quite a few problems lately. I use it in the following way:

Every 2 minutes:

`if (myBot.testConnection()){
    while(CTBotMessageText == myBot.getNewMessage(msg)){ 
    .
    .
    .
    }`

Lately, testconnection() returns 0 many times, so I think I don't have internet (4G router) and I don't try to read the new messages.

when sending a message:

`if (myBot.testConnection()){
    //String MENSA; (global)
    int32_t res=myBot.sendMessage(msg.sender.id,MENSA);

    if(res==0){ // error
        .
        .
        .
        }
    }`

If sending returns 0, I save the message to resend it when another message is sent correctly or testconnection() returns other than 0, but it turns out that sometimes sendMessage returns 0 when the message has been received correctly by the Telegram app on my cell phone.

Is there a "more perfect" way to make use of CTbot? I only use plain text, both sending and receiving.

Regards and thanks for your great work.

esp8266 version 3.1.1 arduinojson version 6.19 (6.20 continuously resets my ESP) ctbot 2.1.9 compiled with IDE 1.8.19

[shurillu]

Hello @bosoft-ESP, thank you for using the library. As you said, your environment/scenario is: an ESP8266 board connected via WiFi to a 4G router. So, there could be a WiFi active link (ESP8266 <--> 4G router) but no internet connection (4G router <--X--> internet). Some detail:

• the testConnection() member function call the getMe private member function that get the user details from telegram. It's intended to check the connection for the first time (is the token correct? for example), but if you have already other tasks to do (checking for new incoming messages or sending a new message) you could check the return parameter without using the testConnection().
• the getNewMessage() member function check new messages only every 3500 ms (3 seconds and half). This is needed because the Telegram server handle only one request every 3 seconds (more or less). If you call an update between these 3 seconds, Telehram "hang the connection" until 3 seconds are passed (since the last update), blocking the ESP8266 execution until the answer are provided. So, to avoid it, the getNewMessage() member function query the telegram server ONLY if 3500 ms are passed by from the last update; if not it returns CTBotMessageNoData (zero) . So, if you put a getNewMessage() call inside a loop, you will retrieve a bunch of CTBotMessageNoData until 3,5 seconds are passed. There is a way to wait a telegram response by setting the blocking getNewMessage() parameter to true. In that case, the execution is stopped until the Telegram server response is provided.

My suggestion is to write a function that check the WiFi connectivity and the internet connectivity (for example, pinging the google server). If the WiFi go down, try to bring it up, if the internet connectivity go down, simply wait until it comes up.

Use it in the loop/every time the sendMessage() fail (the return value is different from the recipient ID).

Cheers,

Stefano

P.S.: there is a new version of the CTBot library, not already published but downloadable form the v3.0.0 branch. There are several improvements and new member functions. If you have time, chack it out ;-)

[bosoft-ESP]

Thank you Stefano From what I understand, I shouldn't call testConnection() anymore? I mean, the first time, after connecting to the wifi, I use testConnection(). Once I receive the result other than 0, use sendMessage() and getNewMessage() at will, always exceeding 3500 ms between calls. I make an interval of 2 minutes between calls to getNewMessage().

As for the internet connection failure, I call several websites and have no problems. Also, to test, I installed a wifi camera and so far it has not given me any problems. I emphasize that the failure is recent (less than 1 year) and where I found it strange is when I get error in sending (=0) and the message arrives correctly to the app.

I have not tried 3.0. I will give it a try. I will change testConnection() previous to sendMessage() and getNewMessage() by ping call to google. I will report the result.

Greetings and thanks

Translated with www.DeepL.com/Translator (free version)

[bosoft-ESP]

Hello Stefano. Today I have seen the version 6.20.1 of ArduinoJson in the library manager of the IDE 1.8.19. I have installed it and it keeps doing continuous resets. Then I thought if, returning 0 when sending the message when it has been sent correctly could be from the ArduinoJson. I have installed version 6.19.3 for testing purposes. I will go down version if the error continues.

On the other hand, I have been investigating with the ping for 8266, and none of the ones I have tried works perfectly. Many of them always give error when pinging google even though on screen they give the correct data.

I will continue to report Regards and thanks

Translated with www.DeepL.com/Translator (free version)

[shurillu]

Thank you Stefano From what I understand, I shouldn't call testConnection() anymore? I mean, the first time, after connecting to the wifi, I use testConnection(). Once I receive the result other than 0, use sendMessage() and getNewMessage() at will, always exceeding 3500 ms between calls. I make an interval of 2 minutes between calls to getNewMessage().

In summary,

• testConnection() only if you need to know if the bot configuration is ok (tipically in the setup() block)
• getNewMessage() query the Telegram server only every 3500ms (or more) between two calls. You can call it faster, but if the timeout of 3500 ms is not lasted, the member function return CTBotMessageNoData
• sendMessage() is executed every time you call it (there is no timeout). Check the return value, the member function returns the recipient ID if no error occurred.

As for the internet connection failure, I call several websites and have no problems. Also, to test, I installed a wifi camera and so far it has not given me any problems. I emphasize that the failure is recent (less than 1 year) and where I found it strange is when I get error in sending (=0) and the message arrives correctly to the app.

Sadly it's difficult to point out what is the problem. Keep in mind that since three years ago, the only changes in the library are related to the fingerprint/Telegram certificate. Meanwhile Espressif has updated the ESP8266 toolchain/library, Benoit Blanchon has updated his library (ArduinoJson). Moreover, other libraries could interact with unpredictable behavior.. so the community help is invaluable (as you are doing!) Thanks,

Stefano

[shurillu]

Hello Stefano. Today I have seen the version 6.20.1 of ArduinoJson in the library manager of the IDE 1.8.19. I have installed it and it keeps doing continuous resets. Then I thought if, returning 0 when sending the message when it has been sent correctly could be from the ArduinoJson. I have installed version 6.19.3 for testing purposes. I will go down version if the error continues.

Regarding the 6.20.x version of the ArduinoJson, I have to check what changes are been done in the library in order to find what is the culprit of the board reset.

On the other hand, I have been investigating with the ping for 8266, and none of the ones I have tried works perfectly. Many of them always give error when pinging google even though on screen they give the correct data.

I will continue to report Regards and thanks

Thanks for your effort, regards

Stefano

[bosoft-ESP]

Hello Stefano. I keep testing and so far it seems that the ArduinoJson library that gives me fewer errors is the 6.19.2. I will try the 6.19.1. To check the errors, I have changed the return in the CTBot::sendMessage function to -1.

.
.
.
#elif ARDUINOJSON_VERSION_MAJOR == 6
    DynamicJsonDocument root(CTBOT_JSON6_BUFFER_SIZE);
    DeserializationError error = deserializeJson(root, sendCommand(FSTR("sendMessage"), parameters));
    if (error) {
        #if CTBOT_DEBUG_DISABLED==1
        serialLog(FSTR("getNewMessage error: ArduinoJson deserialization error code: "), CTBOT_DEBUG_JSON);
        serialLog(error.c_str(), CTBOT_DEBUG_JSON);
        serialLog("\n", CTBOT_DEBUG_JSON);
        #endif
        return -1; //<<--------
    }
#endif
.
.
.

I understand that this -1 is returned by the Json when it has an error. Well, since then, sendMessage has only returned 0 once. In the remaining 4 (-1) if I have received the message to the app.

I continue investigating

Greetings

Translated with www.DeepL.com/Translator (free version)

[shurillu]

Hello bosoft-ESP, thank you for all your effort!. Could I give and advice? If you open the file CTBotDefines.h, there is a define that enable the serial console debug messages. In detail, change this define

#define CTBOT_DEBUG_MODE CTBOT_DEBUG_DISABLED

in

#define CTBOT_DEBUG_MODE CTBOT_DEBUG_JSON

In that way you can see all JSON parsing error and we can identify what's going on.

Regards,

Stefano

[bosoft-ESP]

Hi Stefano I will try, but that wemos Mini D1 is installed elsewhere, and the errors are random. I guess because of the 4G coverage. I will try to make it save me the Json errors and send them via Telegram. In 3 days I have had 5 errors. As I told you, one was correct, the other 4 sent me duplicate messages (the original and the one that resends if the response when sending is <=0). I will be trying things, although I can not test every day or leave a screen connected continuously.

Regards

Translated with www.DeepL.com/Translator (free version)

[bosoft-ESP]

Hi Stefano. I'm going to try the following with ArduinoJson 6.19.2. in CTBot.h

class CTBot
{

public:
    // default constructor
    CTBot();
    // default destructor
    ~CTBot();
    String SendError;   //<------------- new String
    // set a static ip. If not set, use the DHCP. 
    // params
    //   ip        : the ip address

In sendMessage of CTBot.cpp (approx. line 358)

#elif ARDUINOJSON_VERSION_MAJOR == 6
    DynamicJsonDocument root(CTBOT_JSON6_BUFFER_SIZE);
    DeserializationError error = deserializeJson(root, sendCommand(FSTR("sendMessage"), parameters));
    if (error) {
        #if CTBOT_DEBUG_DISABLED==1
        serialLog(FSTR("getNewMessage error: ArduinoJson deserialization error code: "), CTBOT_DEBUG_JSON);
        serialLog(error.c_str(), CTBOT_DEBUG_JSON);
        serialLog("\n", CTBOT_DEBUG_JSON);
        #endif
        SendError=FSTR("ArduinoJson deserialization error code: ")+String(error.c_str())+"\r\n";
        return -1;
    }
#endif

If it gives error (return<=0) to send a new message, I will save SendError in my error system, which will be sent to me to the app as soon as an error-free send is performed. If you need any extra data, let me know.

Regards

[bosoft-ESP]

Hello I just received an error with the new configuration. I am attaching copy of the app screen. The first one is the correct answer. 1 minute later I receive the second one, which is set up as follows: [-1] return from senmessage(). Disapte, 18-02-23 a les 11:33:09> Day of the week, date and time of sending the original message. Llums entraeta apagades: Original message. ---> ArduinoJson deserialization error code: EmptyInput : SendError message

I hope it helps you. But if you need any more test, don't hesitate to let me know. In a few days, I will downgrade the ArduinoJson version.

Regards

Translated with www.DeepL.com/Translator (free version)

[shurillu]

Hello bosoft-ESP,

that error occurs when there is no data in the Telegram Server JSON response (EmptyInput error code). You can check it by storing the JSON data in another String variable (as you do for the SendError variable) calling this function just below the SendError assignment:

serializeJsonPretty(root, yourStringVariable)

In that case, you should have an empty string.

So, save the JSON response (as I explained before) and bypass the JSON check by disabing this define

#define CTBOT_CHECK_JSON 0

in the CTBotDefines.h. Once you got an error sending a message, send back the SendError variable and the JSON data. If you got errors in correctly dispached messages, something is going wrong in the JSON response.

Thanks!

Stefano

[bosoft-ESP]

Ok Stefano. I have done this:

#define CTBOT_CHECK_JSON 0  // in CTBotDefines.h

SendError=FSTR("ArduinoJson deserialization error code: ")+String(error.c_str())+"\r\n";
serializeJsonPretty(root, jsonStr);
SendError+="\r\n/"+jsonStr+"\\";
return -1;

Is it correct? I will pass you the answers as soon as I have them Thanks and best regards

[bosoft-ESP]

Hello Stefano. I think it is already solved. I have been testing for 2 days the modifications I made in SendMessage when I downgraded the ArduinoJson version to 6.18.4. I have not had any error. None. So, confirming my intuition, it was ArdunioJson error. I advise, for the moment, not to upgrade from version 6.18.4.

Greetings, and thank you very much for your patience and work. PS: the modifications in CTBot.cpp are installed in the D1 mini in case one day a Json error occurs.

Translated with www.DeepL.com/Translator (free version)

[bosoft-ESP]

Hello Stefano. After a week of testing the D1 Mini with ArdunoJson version 6.18.4, I have not received any sending error from SendMessage (-1 / 0), which leads me to affirm that there is a bug in version 6.18.5, and it is carried over to higher versions. The expected messages have arrived correctly to my Telegram app without duplicates due to the error. I close this issue.

Thank you very much for your hard work and effort. Regards

Translated with www.DeepL.com/Translator (free version)

[shurillu]

Hello bosoft-ESP, thank you for your precious feedback! Based on it, I'll check if there are some documented changes on ArduinoJson after the version 6.18.4 that may cause that behavior. If not, I'll contact the developer.

Again thank you a lot.

Cheers, Stefano

[bosoft-ESP]

Hello. I just saw the ArduinioJson version 6.21.0.

Tested on an esp01s, still no fix for the constant resets.

Does anyone know if they have fixed any other bugs? Regards

[bosoft-ESP]

Hello In case anyone doubts, ArduinoJson version 6.21.2 also crashes (continuous resets) with CTBot when testing it with an ESP01s. I go back to 6.18.4, which, so far, is the only one that has not given me errors. Regards