cotestatnt / AsyncTelegram2

Powerful, flexible and secure Arduino Telegram BOT library. Hardware independent, it can be used with any MCU capable of handling an SSL connection.
MIT License
83 stars 25 forks source link

Frequent reset after update to 2.1.7 #94

Closed MudiStar closed 1 year ago

MudiStar commented 1 year ago

Hello

I'm using your library in several esp 8266 and I'm pretty happy with it. So my code was running fine but since I updated to 2.1.7, I get the following reset sometimes 3 times per hour: Sometimes it takes longer.

ets Jan 8 2013,rst cause:4, boot mode:(3,6)

wdt reset load 0x4010f000, len 3460, room 16 tail 4 chksum 0xcc load 0x3fff20b8, len 40, room 4 tail 4 chksum 0xc9 csum 0xc9 v0007b6e0 ~ld

I also recognized, that if I don't have internet through the connected wifi (router restart after power outage) the esp keeps resetting with the same error.

Do you need more infomation to debug the problem?

Thanks

MudiStar commented 1 year ago
Exception Stack

``` I was just able to record another reset: --------------- CUT HERE FOR EXCEPTION DECODER --------------- Soft WDT reset >>>stack>>> ctx: sys sp: 3ffff4b0 end: 3fffffb0 offset: 01a0 3ffff650: 3fff0694 3fff6a24 401003ad 00000000 3ffff660: 0016fa85 3fff4168 00004181 000000fd 3ffff670: 59ef9205 4bc6a7f0 00000000 00003a98 3ffff680: 59ef9205 00000000 4bc6a7f0 00000000 3ffff690: 000000fd 000000fd 4010037e 00000000 3ffff6a0: 59ef9205 3ffff6d0 3fff425c 40213b4a 3ffff6b0: 00000000 3fff5504 3fff0fe4 00003a98 3ffff6c0: 00000000 0005a69a 3fff0fe4 40208a00 3ffff6d0: 000000fd 59f8b3b1 58d5a2a5 3c4bdcd3 3ffff6e0: 00000008 00000001 0016fa84 00000000 3ffff6f0: 40239504 40239554 3fff425c 40238b42 3ffff700: 3fff475c 3ffe98ca 3fff425c 3ffe98ca 3ffff710: 00000000 00000001 3fff0fe4 40208b1f 3ffff720: 00000000 00000001 3fff0fe4 40208d0f 3ffff730: 00000019 3fff03d0 00000001 40213a78 3ffff740: 00000000 00000000 000001bb 4021418a 3ffff750: 00000000 3fff5c2c 3fff0fe4 40207bd2 3ffff760: 3fff4164 4bc6a7f0 589374bc 3ffe99c0 3ffff770: 000001bb 3ffe98ca 3fff0fe4 3ffe99c0 3ffff780: 000001bb 3ffe98ca 3fff0fe4 40208db1 3ffff790: 4021d268 dca79a95 4021d268 dca79a95 3ffff7a0: 3ffff810 00000000 3ffefe4c 4021b9ae 3ffff7b0: 3ffff810 00000000 3ffefe4c 4020b4e8 3ffff7c0: 00000000 00000000 3ffefe4c 4020b553 3ffff7d0: 0000000c 00000000 401003ad 00000000 3ffff7e0: 00000000 3fff0fe4 3ffefe4c 00000000 3ffff7f0: 00000000 00000010 3ffefe4c 3fff0368 3ffff800: 00000000 00000000 3ffefe4c 4020b6c4 3ffff810: 696c227b 2274696d 222c313a 656d6974 3ffff820: 2274756f 222c303a 7366666f 3a227465 3ffff830: 50007d30 401043b8 00000005 4010222c 3ffff840: 00000005 00000000 00000020 40100298 3ffff850: 40102db7 3ffee130 00000005 4010222c 3ffff860: 3ffea745 401054c3 3ffee130 3ffee8c0 3ffff870: 00000005 00000000 00000020 40100298 3ffff880: 00000012 59931c86 00000005 4010222c 3ffff890: 3ffea745 401054c3 3ffee108 40100298 3ffff8a0: 40102db7 3ffee108 3ffee8c0 4010222c 3ffff8b0: 00000000 00000000 0000001f 40100298 3ffff8c0: 3ffeafd8 00000000 3fffc228 40105d5d 3ffff8d0: 4000050c 59b4d9f1 4010344e 00000100 3ffff8e0: 4000dd1a 00000030 00000010 ffffffff 3ffff8f0: 4000066d 59bcc1c7 4bc6a7f0 000059bc 3ffff900: 00004bc6 7438f12a 00000010 fffffffe 3ffff910: 00007438 3fffc6fc 7f1e7b90 59bcc1c7 3ffff920: 4bc6a7f0 3ffefe4c 3fff0368 00000030 3ffff930: 401003b8 00000030 3fffc228 40105d5d 3ffff940: 4000050c 0016f700 00000000 00000000 3ffff950: 4020ebf3 00000030 00000010 ffffffff 3ffff960: 4020ebf0 0016f72c 00418937 3ffefe4c 3ffff970: 00004bc6 00000000 00000010 fffffffe 3ffff980: 00000000 3fffc6fc 00000000 00000000 3ffff990: 3ffffd60 3ffefe4c 3fff0368 00000030 3ffff9a0: 3ffeafe4 2c9f0300 4000050c 3ffefe54 3ffff9b0: 00000000 3fff33dc 3ffffb78 4020da9a 3ffff9c0: 3ffffb78 3fff424f 00000006 000007ed 3ffff9d0: 000006d9 000006d9 3ffe86d4 40100973 3ffff9e0: 3ffffb78 32373631 34333036 00000006 3ffff9f0: 00000000 4bc6a7f0 08b43958 1a91350a 3ffffa00: 00000000 00000000 4bc6a7f0 00000000 3ffffa10: fffff82c 00000000 401003ad 00000000 3ffffa20: 0016fa20 00000020 3fff424c 40100b52 3ffffa30: 3ffe9aab 3ffffd60 00000000 3fff0368 3ffffa40: 00000000 3ffffd60 00000000 4020ec1b 3ffffa50: 00000005 00000000 00000020 40100298 3ffffa60: 00000005 00000000 00000020 40100298 3ffffa70: 3ffea745 401054c3 00000005 4010222c 3ffffa80: 3ffea745 401054c3 3ffee108 40100298 3ffffa90: 40102db7 3ffee108 3ffeea64 40102f94 3ffffaa0: 00000012 59c05a8f 3ffeea64 40102f94 3ffffab0: 3ffeafcc 00000000 00000000 00000100 3ffffac0: 00000012 59c05a8f 4010344e 00000100 3ffffad0: 3ffeafcc 7fffffff 00000000 00000001 3ffffae0: 00000001 00006208 0000001f 40100298 3ffffaf0: 3ffeafcc 2c9f0300 3fffc228 59c05a8f 3ffffb00: 3ffeafd8 2c9f0300 4000050c 3fffc278 3ffffb10: 40103170 3fffc200 00000022 ffffffff 3ffffb20: 402126b5 00000030 00000010 ffffffff 3ffffb30: 402016aa 00000000 00000000 00000000 3ffffb40: 00004bc6 00000000 00000010 fffffffe 3ffffb50: 000001f0 3fffc6fc 00000000 3ffffe38 3ffffb60: 3ffffdf8 3ffefe4c 3fff0368 00000030 3ffffb70: 00000360 3fffc6fc 00000000 3fff0354 3ffffb80: 00000000 3ffefe4c 3fff0368 00000030 3ffffb90: 3fff4268 3fff4258 00000020 00000000 3ffffba0: 00000000 00000000 00000000 00000000 3ffffbb0: 00000000 00000000 00000000 00000000 3ffffbc0: 00000000 00000000 00000000 00000000 3ffffbd0: 00000000 00000000 00000000 00000000 3ffffbe0: 00000000 00000000 00000000 0000005a 3ffffbf0: 00000004 00000080 00000020 40100b7c 3ffffc00: 00000000 3ffefe4c 000000dc 401004d8 3ffffc10: 3ffffc51 00000000 3ffffd80 4020166a 3ffffc20: 00000000 00000000 3ffffd60 3fff0368 3ffffc30: 3ffefe4c 00000000 3fff0354 40202466 3ffffc40: ffffffff 00000000 3ffea891 00000000 3ffffc50: 402635ba 3ffede88 3fff2aec 00000001 3ffffc60: 00000000 00000000 0000001f 40100298 3ffffc70: 00000005 00000005 3fffc228 40105d5d 3ffffc80: 4000050c 40262677 3ffede88 3fff5efc 3ffffc90: 401019b5 00000030 0000001f fffffffe 3ffffca0: 00000000 00000000 0000001f 40100298 3ffffcb0: 00000005 00000000 00000020 40100298 3ffffcc0: 000005e0 00000000 401042c9 3ffee8c0 3ffffcd0: 0000003c 00000000 3ffee108 000000fd 3ffffce0: 00000005 00000000 00000020 40100298 3ffffcf0: 00000005 00000000 00000005 4010222c 3ffffd00: 3ffea745 401054c3 3ffee130 4010222c 3ffffd10: 00000005 00000000 00000020 40100298 3ffffd20: 00000012 59a5ac2b 00000005 4010222c 3ffffd30: 3ffea745 401054c3 3ffee108 40102f94 3ffffd40: 40102db7 3ffee108 00000020 40100298 3ffffd50: 00000012 59bb8c1f 3ffeea64 40102f94 3ffffd60: 00000000 00000001 00000020 40100298 3ffffd70: 00000012 59bb8c1f 00000005 4010222c 3ffffd80: 00000005 00000000 00000000 00000000 3ffffd90: 00000000 00000000 00000000 00000000 3ffffda0: 3ffea745 00000000 00000000 4010222c 3ffffdb0: 00000000 3ffee108 3ffee8c0 4010222c 3ffffdc0: 00000000 00000000 00000000 40102f94 3ffffdd0: 00000000 00000000 00000000 00000000 3ffffde0: 00000000 00000000 00000000 40100298 3ffffdf0: 3ffeafe4 7fffffff 3fffc228 40105d5d 3ffffe00: 00000000 00006208 00000000 00000000 3ffffe10: 4000dd01 00000000 00000000 ffffffff 3ffffe20: 00000000 00000000 4bc6a7f0 00000000 3ffffe30: 00004bc6 7a2b59d3 00000000 fffffffe 3ffffe40: 00000000 00000000 4e6659d3 00000000 3ffffe50: 00000000 94bc6a7f 00000000 00000030 3ffffe60: 00000000 00000000 00000000 fffffffe 3ffffe70: 00000000 00000000 00000001 00000000 3ffffe80: 00000000 00000000 0000001f 40100298 3ffffe90: 00000000 3fffc6fc 3fffc228 40105d5d 3ffffea0: 4000050c 3ffe8630 3fff0368 00000030 3ffffeb0: 40213ba4 00000030 00000010 ffffffff 3ffffec0: 00000000 4bc6a7f0 01cac083 1a913502 3ffffed0: 00000000 00000000 4bc6a7f0 00000000 3ffffee0: 00000360 3fffc6fc 401003ad 00000000 3ffffef0: 0016fa20 3fffdad0 3fff0368 00000030 3fffff00: 00000000 3fffdad0 3fff0368 00000001 3fffff10: 3fff403c 3fff0b1c 3fff4070 40216c16 3fffff20: 00000000 4bc6a7f0 049ba5e3 1a913505 3fffff30: 00000000 4bc6a7f0 072b020c 1a913508 3fffff40: 00000000 00000000 4bc6a7f0 00000000 3fffff50: 0016fa20 3fffff70 401003ad 00000000 3fffff60: 0016fa20 3fff0b1c 3fff0080 3fff0368 3fffff70: 3ffe8630 3ffefdd8 00000001 3fff0368 3fffff80: 3fffdad0 00000000 3fff0354 3fff0368 3fffff90: 3fffdad0 00000000 3fff0354 40205b00 3fffffa0: feefeffe feefeffe 3fff0354 40213b98 <<::expiredOneShot() const at C:\Users\----\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2\cores\esp8266/PolledTimeout.h line 234 0x401003ad: millis() at C:\Users\----\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2\cores\esp8266\core_esp8266_wiring.cpp line 193 0x40213b98: loop_wrapper() at C:\Users\----\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2\cores\esp8266\core_esp8266_main.cpp line 201 ```

MudiStar commented 1 year ago

So it got a lot better since the esp core update to 3.1.0 Still one problem remains: reset of the esp if no internet is available.

0x4020ac08: AsyncTelegram2::checkConnection() at C:\Users\Arduino\libraries\AsyncTelegram2\src\AsyncTelegram2.cpp line 39

What I understand is, that the connection is checked in the frequency of the main loop and then the esp crashes this is what I also see in the serial monitor

1) all startet with begin():

bool AsyncTelegram2::begin()
{
    checkConnection();
    return getMe();
}

2) Connection is checked:

bool AsyncTelegram2::checkConnection()
{
    // Start connection with Telegramn server (if necessary)
    if (!telegramClient->connected())
    {
        m_lastmsg_timestamp = millis();
        log_info("Start handshaking...");

        if (!telegramClient->connect(TELEGRAM_HOST, TELEGRAM_PORT))
        {
            Serial.println("\n\nUnable to connect to Telegram server");
            reset();
        }

3) reset is called

bool AsyncTelegram2::reset(void)
{
    log_info("Restart Telegram connection\n");
    telegramClient->stop();
    m_lastmsg_timestamp = millis();
    m_waitingReply = false;
    return checkConnection();
}

4) calls checkConnection: endless loop

MudiStar commented 1 year ago

Interims Solution: comment out reset()

bool AsyncTelegram2::checkConnection()
{
    // Start connection with Telegramn server (if necessary)
    if (!telegramClient->connected())
    {
        m_lastmsg_timestamp = millis();
        log_info("Start handshaking...");

        if (!telegramClient->connect(TELEGRAM_HOST, TELEGRAM_PORT))
        {
            Serial.println("\n\nUnable to connect to Telegram server");
            //reset();
        }
#if DEBUG_ENABLE
        else
        {
        static uint32_t lastCTime;
            log_debug("Connected using Telegram hostname\n"
                      "Last connection was %d seconds ago\n",
                      (int)(millis() - lastCTime) / 1000);
            lastCTime = millis();
        }
#endif
    }
    return telegramClient->connected();
}
MudiStar commented 1 year ago

I'm running into resets on the ESP8266 again, like 15 times in 24h. The ESP placed next to the router Everything works fine on the ESP32.

@cotestatnt Is there a way you can help me out? I recorded more Exception Stacks for debugging.

cotestatnt commented 1 year ago

Hi @MudiStar First of all, I'm sorry for late reply but in this last months I was really busy at work.

I suspect this bug is actually "double". The reset due to the reset() statement (sorry for the pun) should be caused by a "death loop" that this statement introduces.

As for the reset which occurs once every 15h, I think it is due to the "delay" introduced by the failed connection with the Telegram server which can happen from time to time for the various reasons that trigger the ESP8266 software watchdog.

I'm trying out some recently introduced changes. As soon as I have positive results I also update the repository.

cotestatnt commented 1 year ago

Hi @MudiStar I just uploaded the edited file. I've been trying a sketch since this morning and it hasn't reset until now, looks promising.

To make it easier for me to manage this problem, I make sure that on reboot the esp sends me a message containing the cause of the last reset

extern "C" {
#include <user_interface.h> // https://github.com/esp8266/Arduino actually tools/sdk/include
}

void setup {
  .....
  ......
  rst_info *myResetInfo;
  myResetInfo = ESP.getResetInfoPtr();

  char welcome_msg[256];
  snprintf(welcome_msg, sizeof(welcome_msg),
    "myResetInfo->reason %x \nBOT @%s online\n/help all commands avalaible.",
    myResetInfo->reason,
    myBot.getBotName()
  );

  // Send a message to specific user who has started your bot
  myBot.sendTo(userid, welcome_msg);
}
MudiStar commented 1 year ago

Hello @cotestatnt

I'm happy that you "are back" ;-)

I compiled my two programs I'm using your library with. Fingers crossed. In the meantime, I've noticed that it seems to be stable just running your OTA example. If I add a webserver and some other non-blocking functions, everything starts getting unstable. Same code on the ESP32 works fine.

My WakeOnLan code is pretty straight forward. So no pin readings or something See attached picture. I couldn't find a pattern yet.

Log

MudiStar commented 1 year ago

It is not working for me. Two restarts so far I try to get a stack dump

MudiStar commented 1 year ago

This doesn't happen if I'm just running the Webserver. My second Bot is running so far. Maybe you know what is going on

Exception 5: Alloca: MOVSP instruction, if caller's registers are not in the register file PC: 0x40100217 EXCVADDR: 0x00000000

Decoding stack results 0x402166b0: esp8266::MDNSImplementation::MDNSResponder::_parseMessage() at C:\Users\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\ESP8266mDNS\src\LEAmDNS_Control.cpp line 124 0x40204c1d: esp8266webserver::ESP8266WebServerTemplate ::handleClient() at C:\Users\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\ESP8266WebServer\src/ESP8266WebServer-impl.h line 282 0x40213b0c: abort() at C:\Users\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_postmortem.cpp line 289 0x40100217: cont_check(cont_t*) at C:\Users\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\cont_util.cpp line 56

MudiStar commented 1 year ago

Reset is triggered by Telegram connection because something seems to be blocking

12:49:55.996 -> 12:49:56 17:1:2023 12:49:56.999 -> 12:49:57 17:1:2023 12:50:12.091 -> 12:50:12.091 -> 12:50:12.091 -> Unable to connect to Telegram server 12:50:20.768 -> 12:50:20.768 -> ets Jan 8 2013,rst cause:4, boot mode:(3,7) 12:50:20.768 -> 12:50:20.768 -> wdt reset

MudiStar commented 1 year ago

Each time a little different. I stop spamming now

Exception 5: Alloca: MOVSP instruction, if caller's registers are not in the register file PC: 0x401003df EXCVADDR: 0x00000000

Decoding stack results 0x4021135c: Stream::SendGenericPeekBuffer(Print*, int, int, unsigned int) at C:\Users\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\StreamSend.cpp line 66 0x402195dc: esp8266::MDNSImplementation::MDNSResponder::_writeMDNSAnswer_PTR_NAME(esp8266::MDNSImplementation::MDNSResponder::stcMDNSService&, esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter&) at C:\Users\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\libraries\ESP8266mDNS\src\LEAmDNS_Transfer.cpp line 1665 0x401003df: millis() at C:\Users\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.1\cores\esp8266\core_esp8266_wiring.cpp line 176

cotestatnt commented 1 year ago

Good morning @MudiStar I've just noticed you are running the latest esp8266 core version while mine was a little older. I'm triyng now with the latest too.

Unfortunately with the esp8266 there has always been this problem and I have never been able to really figure it out because I mainly use the esp32. I'll update you as soon as I have some more information about the issue.

MudiStar commented 1 year ago

I have an ESP8266 running without any problems since October with your version 2.1.6 after fixing a bug in just one line with an additional "if", it keeps crashing randomly The same line is ok running on the ESP32.

Maybe a library was updated simultaneously or I was just lucky that it was working Made no sense for me

If you need my code or something, feel free to ask