me-no-dev / AsyncTCP

Async TCP Library for ESP32
GNU Lesser General Public License v3.0
756 stars 439 forks source link

ESP32 crashes while connecting to server ... sometimes #88

Closed joaolrc closed 4 years ago

joaolrc commented 4 years ago

Hi All. So i wrote a library for Async HTTP calls on top of AsyncTCP and i am testing , making 2 concurrent requests and sometimes i get this error followed by a crash:

assertion "new_rcv_ann_wnd <= 0xffff" failed: file "/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp.c", line 779, function: tcp_update_rcv_ann_wnd
abort() was called at PC 0x40118acf on core 0

Backtrace: 0x4008c454:0x3ffb3f80 0x4008c685:0x3ffb3fa0 0x40118acf:0x3ffb3fc0 0x40132b32:0x3ffb3ff0 0x40132bc1:0x3ffb4010 0x4016a4ba:0x3ffb4030 0x4012f07c:0x3ffb4050 0x40088b9d:0x3ffb4080

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:8896
load:0x40080400,len:5828
entry 0x400806ac

Header File (AsyncHttpClient.h)

#ifndef ASYNC_HTTP_CLIENT_H
#define ASYNC_HTTP_CLIENT_H

#include <Arduino.h>
#include <functional>
#include <AsyncTCP.h>

// #define AHC_VERBOSE_LVL 3 //0 to 3 --> adjust to modify verbose
#define AHC_MAX_RESPONSE_SIZE 5000
#define AHC_MAX_REQBODY_SIZE 1000
#define AHC_EMPTY_RESPONSE -1

// typedef std::function<void(int, String)> ahc_responseCallback_t;
typedef void (*ahc_responseCallbackPtr_t)(int,String);      //pointer to request callback type

//to store response from server parsed
typedef struct {
    int httpCode;
    String responseBody;
}ahc_requestResponse_t;

typedef struct {
    String host;
    String query;
    String body;
    ahc_responseCallbackPtr_t respCallbackPtr;
    uint16_t port;
    bool islastMethodGet;
}ahc_lastCalledParameters_t;

class AsyncHttpClient{
public:

    bool get( String host, String query="", ahc_responseCallbackPtr_t callback = NULL, uint16_t port=80 );
    bool post( String host, String query="", String body="", ahc_responseCallbackPtr_t callback = NULL, uint16_t port=80 );
    bool cancelRequest();

private:

    AsyncClient * _aClient = NULL;
    String _dataToSend = "";
    String _lastServerResponse = "";
    ahc_lastCalledParameters_t _myLastParameters= {"", "", "", NULL, 80, true};
    uint8_t _connectionAttemptCounter = 0;
    // ahc_responseCallback_t * _myResponseCallback = NULL;

    //debug Function
    void debugPrint(String stringToPrint, uint8_t verboseLvl);
    //Parse full raw response from server and return ahc_requestResponse struct (see above definition)
    ahc_requestResponse_t parseResponse(String response);
    //count number if occurences of substring in string
    uint16_t countSubStr(String fullStr, String toFind="\r\n");
    //define client onConnect, onError, onDisconnect ... events. And send request
    void defineEvents();
    void fallbackDnsError();

protected:

};

#endif /* ASYNC_HTTP_CLIENT_H */

Source file (AsyncHttpClient.cpp)

void AsyncHttpClient::fallbackDnsError(){
    debugPrint(F("[AsyncHttpClient] DNS fallback"), 1);
    if (_connectionAttemptCounter>=5){
        cancelRequest();
    }
    if (_myLastParameters.islastMethodGet){
        get(_myLastParameters.host, _myLastParameters.query, _myLastParameters.respCallbackPtr, _myLastParameters.port);
    }else{
        post(_myLastParameters.host, _myLastParameters.query, _myLastParameters.body, _myLastParameters.respCallbackPtr, _myLastParameters.port);
    }

}

bool AsyncHttpClient::get( String host, String query, ahc_responseCallbackPtr_t callback, uint16_t port){
    //client already exists - prevent concurrent requests
    if (_aClient) 
        return false;

    _aClient = new AsyncClient();

    _myLastParameters.host = host;
    _myLastParameters.query = query;
    _myLastParameters.respCallbackPtr = callback;
    _myLastParameters.port = port;
    _myLastParameters.islastMethodGet = true;

    //format msg according HTTP standard
    _dataToSend = "GET "+query+" HTTP/1.1\r\nHost: "+host+"\r\nConnection: close\r\n\r\n";

    if (!_aClient) return false; //could not allocate client

    defineEvents();

    if (!_aClient->connect(_myLastParameters.host.c_str(), _myLastParameters.port)){
        debugPrint(F("[AsyncHttpClient] Connection Failed"), 1);
        AsyncClient *client = _aClient;
        _aClient = NULL;
        delete client;
        return false;
    }

    return true;
}

bool AsyncHttpClient::post( String host, String query, String body, ahc_responseCallbackPtr_t callback, uint16_t port ){
    //client already exists - prevent concurrent requests
    if (_aClient) 
        return false;

    _aClient = new AsyncClient();

    _myLastParameters.host = host;
    _myLastParameters.query = query;
    _myLastParameters.body = body;
    _myLastParameters.respCallbackPtr = callback;
    _myLastParameters.port = port;
    _myLastParameters.islastMethodGet = false;

    //format msg according HTTP standard
    _dataToSend = "POST "+query+" HTTP/1.1\r\nHost: "+host+"\r\nConnection: close\r\nContent-Length: "+String(body.length())+"\r\nContent-Type: application/json\r\n\r\n"+body;

    if (!_aClient) return false; //could not allocate client

    defineEvents();

    if (!_aClient->connect(_myLastParameters.host.c_str(), _myLastParameters.port)){
        debugPrint(F("[AsyncHttpClient] Connection Failed\n"), 1);
        AsyncClient *client = _aClient;
        _aClient = NULL;
        delete client;
        return false;
    }

    return true;
}

void AsyncHttpClient::defineEvents(){
    _aClient->onError([this](void *arg, AsyncClient *client, int error){
        char buf[50];
        snprintf(buf, 50, "[AsyncHttpClient] Connect Error: %d\n", error);
        debugPrint(buf, 1);
        _aClient = NULL;
        delete client;
        //Dirty Fix to error -55 --> DNS failed
        if ((error == -55) && (_connectionAttemptCounter<10) ){
            debugPrint(F("[AsyncHttpClient] Recursive Calling\n"), 1);
            _connectionAttemptCounter++;  
            fallbackDnsError();
        }else{
            if (_myLastParameters.respCallbackPtr!=NULL){
                (*_myLastParameters.respCallbackPtr)(error, "");
            }
        }
    }, NULL);

    _aClient->onConnect([this](void *arg, AsyncClient *client){
        //define events first
        client->onError(NULL, NULL);

        client->onDisconnect([this](void *arg, AsyncClient *c){
            char buff[AHC_MAX_RESPONSE_SIZE];
            snprintf(buff, AHC_MAX_RESPONSE_SIZE, "[AsyncHttpClient] Disconnected. Response from server: %s\n", _lastServerResponse.c_str());
            debugPrint(buff, 2);

            ahc_requestResponse_t reqResp = parseResponse(_lastServerResponse);
            if (_myLastParameters.respCallbackPtr!=NULL){
                (*_myLastParameters.respCallbackPtr)(reqResp.httpCode, reqResp.responseBody);
            }

            _lastServerResponse = "";
            _aClient = NULL;
            delete c;
        }, NULL);

        //parse response from server
        client->onData([this](void *arg, AsyncClient *c, void *data, size_t len) {
            char buf[50];
            snprintf(buf, 50, "[AsyncHttpClient] DataReceived. Len: %u\n", len);
            debugPrint(buf,2);
            if (len>AHC_MAX_RESPONSE_SIZE){
                debugPrint(F("[AsyncHttpClient] Response len > AHC_MAX_RESPONSE_SIZE. Ignoring response\n"), 1);
                _lastServerResponse = "";
                return;
            }
            uint8_t *d = (uint8_t *)data;
            char aux_response[len];
            for (size_t i = 0; i < len; i++){
                aux_response[i] = d[i];
            }
            _lastServerResponse += String(aux_response);

        }, NULL);

        //send the request
        char buf[AHC_MAX_REQBODY_SIZE];
        snprintf(buf, AHC_MAX_REQBODY_SIZE, "[AsyncHttpClient] Connected. Request:\n%s", _dataToSend.c_str() );
        debugPrint(buf, 2);
        _connectionAttemptCounter = 0;
        client->write( _dataToSend.c_str() );
    }, NULL);
}

ahc_requestResponse_t AsyncHttpClient::parseResponse(String response){
    ahc_requestResponse_t myRequestResponse;
    if (response == ""){
        debugPrint(F("[AsyncHttpClient] Empty Response\n"), 1);
        return {AHC_EMPTY_RESPONSE, response};
    }
    //retrieve HTTP code from response header
    uint8_t aux_codeBegining = response.indexOf("HTTP/1.1 ")+9;
    int httpCode = response.substring(aux_codeBegining, aux_codeBegining+3).toInt();
    char * buff = new char[AHC_MAX_RESPONSE_SIZE];
    snprintf(buff,AHC_MAX_RESPONSE_SIZE, "[AsyncHttpClient] HTTP CODE --> %d\n", httpCode);
    debugPrint(buff, 2);
    myRequestResponse.httpCode = httpCode;
    //Split response into header and body
    String respHeader = response.substring(0, response.indexOf("\r\n\r\n"));
    String respBody = response.substring( response.indexOf("\r\n\r\n")+4 );
    //check if chunked response
    if (respHeader.indexOf("Transfer-Encoding: chunked")>=0){
        //pass respBody to auxiliar char* and clean to receive the real body without chunk numbers
        char aux_respBody[AHC_MAX_RESPONSE_SIZE];
        respBody.toCharArray(aux_respBody, AHC_MAX_RESPONSE_SIZE);
        respBody="";
        char *line;
        /* get the first line */
         line = strtok((char *)aux_respBody, (char*) "\r\n");
         /* walk through other lines */
         while( line != NULL ) {
            snprintf(buff,AHC_MAX_RESPONSE_SIZE, "[AsyncHttpClient] line: %s\n", line);
            debugPrint(buff, 3);
            char* endptr;
            unsigned long numericValue = strtoul(line, &endptr, 16);
            if (!*endptr){ //if string is numeric --> chunked response number
                snprintf(buff,AHC_MAX_RESPONSE_SIZE, "[AsyncHttpClient] Number: %lu\n", numericValue);
                debugPrint(buff, 3);
                if (numericValue == 0){
                    break;
                }
            }else{ // is part of the body --> append to body
                snprintf(buff, AHC_MAX_RESPONSE_SIZE,"[AsyncHttpClient] Response part: %s\n", endptr );
                debugPrint(buff, 3);
                respBody += endptr;
            }
            //point to next line
            line = strtok(NULL, (char*) "\r\n");
        }
        respBody.trim();
        snprintf(buff, AHC_MAX_RESPONSE_SIZE, "[AsyncHttpClient] Finished parsing chunked body:\n%s\n", respBody.c_str());
        debugPrint(buff, 3);
        myRequestResponse.responseBody = respBody;
        delete[] buff;
        return myRequestResponse;

    }else{
        snprintf(buff, AHC_MAX_RESPONSE_SIZE, "[AsyncHttpClient] FULL Body : \n%s", respBody.c_str());
        debugPrint(buff, 2);
        myRequestResponse.responseBody = respBody;
        delete[] buff;
        return myRequestResponse;
    }    
}

//i thought this was needed but its not. Anyway i wil keep it here until i make a string library
uint16_t AsyncHttpClient::countSubStr(String fullStr, String toFind){
    uint16_t countOccurrences = 0;
    uint16_t toFindLength = toFind.length();
    int indexOf = fullStr.indexOf(toFind);
    while (indexOf >= 0){
        countOccurrences++;
        //remove chunk where substring was found
        fullStr = fullStr.substring( fullStr.indexOf(toFind)+toFindLength );
        //update index of substring
        indexOf = fullStr.indexOf(toFind);   
    }

    return countOccurrences;
}

bool AsyncHttpClient::cancelRequest(){
    if (_aClient==NULL){
        return false;
    }
    if (_aClient->connected() && _lastServerResponse=="" ){
        debugPrint(F("[AsyncHttpClient] Canceling Request. Closing open connenction\n"), 1);
        _aClient->close();
    }
    if (_aClient->connecting() ){
        debugPrint(F("[AsyncHttpClient] Canceling Request. Aborting connection request\n"), 1);
        _aClient->close();
    }
    return true;
}

void AsyncHttpClient::debugPrint(String stringToPrint, uint8_t verboseLvl){
#ifdef AHC_VERBOSE_LVL
    if (AHC_VERBOSE_LVL >= verboseLvl)
        Serial.print(stringToPrint);
#endif
}

Testing Notes

I am instantiating the class twice and making concurrent requests in a for loop.The returned response is JSON. I think my problem is how i handle deletion and heap allocation of the AsybcTCP client instance maybe in the get and post methods but im fighting this battle for some days now and starting to get a bit frustrated. Can someone help me on this?

stale[bot] commented 4 years ago

[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 4 years ago

[STALE_DEL] This stale issue has been automatically closed. Thank you for your contributions.

ivancmz commented 4 years ago

I'm having exactly the same problem, did you find anything @joaolrc

joaolrc commented 4 years ago

@ivancmz not really. I just started using this library https://github.com/boblemaire/asyncHTTPrequest Works well and for now it's ok byut still want to finish my own. Another cool thing i wish to have is https connections. Do you know how can i achieve this?

BlueAndi commented 4 years ago

Decode the backtrace, otherwise its difficult.

Anyway I did the same ;-) and its working: https://github.com/BlueAndi/esp-rgb-led-matrix/blob/master/src/Web/AsyncHttpClient.h https://github.com/BlueAndi/esp-rgb-led-matrix/blob/master/src/Web/AsyncHttpClient.cpp

DaeMonSxy commented 1 year ago

Hi, jumping in also.

assert failed: tcp_update_rcv_ann_wnd IDF/components/lwip/lwip/src/core/tcp.c:951 (new_rcv_ann_wnd <= 0xffff)

my solution seems working within 1-2-3 hours, then suddenly crashing. how did anyone solved the issue?

asynctcp code changes? platformio build_flags ?

any solution is welcome.

drmpf commented 1 year ago

I have the same error calling line is tcp_recved(msg->pcb, msg->received); from AsyncTCP.cpp Back trace is

Decoding stack results
0x40083dc5: panic_abort at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/esp_system/panic.c line 408
0x4008bfa5: esp_system_abort at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/esp_system/esp_system.c line 137
0x40091571: __assert_func at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/newlib/assert.c line 85
0x400fd18e: tcp_update_rcv_ann_wnd at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/lwip/lwip/src/core/tcp.c line 951
0x400fd23c: tcp_recved at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/lwip/lwip/src/core/tcp.c line 991
0x400e0dea: _tcp_recved_api(tcpip_api_call_data*) at C:\Users\matthew\Documents\Arduino\libraries\AsyncTCP\src\AsyncTCP.cpp line 422
0x400f9f24: tcpip_thread at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/lwip/lwip/src/api/tcpip.c line 172
Kamdzik commented 10 months ago

hello I have exactly the same error , has anyone solved it yet?

DaeMonSxy commented 10 months ago

Asynctcp has its own issues. Im also waiting for someone who would rewrite the lib, and share a proper/bug less version.

Meanwhile the only thing I have found is to change in the lib the line: _async_queue = xQueueCreate(256, sizeof(lwip_event_packet_t *)); // dms/ prevents "async_tcp" Watchdog error

original contains 32,

Iam using that way, seems less prone to crash, hope it help, reply if so.