Fix url parser - Githubissues

joaolrc commented 4 years ago

Hi I noticed that you have a bug in the _parseUrl function which is not able to parse url's that redirect to root and dont have trailing frontslash. Example: http://192.168.1.74 The parser was returning false when calling with this string. For it to parse you would have to call it with: http://192.168.1.74/ First i was just adding a frontslash if it was missing but other problems came up. This fixes it. Maybe i left some unnecessary logs over there but works now. Anyway, great job with this library :)

boblemaire commented 3 years ago

I think I agree that this was a problem, but decided to fix it with a rewrite of parseURL for two reasons:

Looking at the way it worked, the order of the components wasn't enforced entirely. For instance if a URL had no path but included a slash in the query
The PR was half-baked with the logs still there. Didn't feel like cleaning that up for you. This rewrite is cleaner and should solve your problem as well.

joaolrc commented 3 years ago

Testing your new code, it crashesd: [D][FlexaAgent.cpp:142] _fa_doHttpRequest(): POST http://flexa-base.azurewebsites.net/api/DeviceGet { "regId": "$m6LJeoELGFL" }

CORRUPT HEAP: Bad tail at 0x3ffce284. Expected 0xbaad5678 got 0xbaad5600 assertion "head != NULL" failed: file "/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/heap/multi_heap_poisoning.c", line 214, function: multi_heap_free abort() was called at PC 0x40117eb7 on core 1

Backtrace: 0x4008d028:0x3ffb19a0 0x4008d259:0x3ffb19c0 0x40117eb7:0x3ffb19e0 0x4008cc9d:0x3ffb1a10 0x40084d12:0x3ffb1a30 0x4008526d:0x3ffb1a50 0x4000bec7:0x3ffb1a70 0x40147f49:0x3ffb1a90 0x401482d1:0x3ffb1ab0 0x400ddc13:0x3ffb1ad0 0x400dea40:0x3ffb1af0 0x400e11d5:0x3ffb1b10 0x400e12c2:0x3ffb1c20 0x400e6c03:0x3ffb1d40 0x400d1ddd:0x3ffb1f20 0x400ea567:0x3ffb1fb0 0x400896e1:0x3ffb1fd0

Rebooting... ets Jun 8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0018,len:4 load:0x3fff001c,len:1044 load:0x40078000,len:8896 load:0x40080400,len:5828 entry 0x400806ac

HttpResponseStruct_t _fa_doHttpRequest( char* httpEndpoint, size_t jsonReturnedDocCapacity , char** requestKeys, char** requestValues, bool isPostRequest, size_t jsonSendDocCapacity, uint8_t maxRetries){
    DynamicJsonDocument returnedDoc(jsonReturnedDocCapacity);
    HttpResponseStruct_t objectToReturn = {SUCCESSCODE_DEFAULTERROR, 0, "", returnedDoc};
    //check Wifi connectivity
    if (!payninoStd_hasNetworkConnectivity) {
        objectToReturn.successCode = SUCCESSCODE_NOWIFICONNECTIVITY;
        return objectToReturn;
    }

    uint8_t requestKeysLength=sizeof(requestKeys)/sizeof(char*);
    uint8_t requestValuesLength=sizeof(requestValues)/sizeof(char*);
    if (requestValuesLength != requestKeysLength){
        log_e("Req Build ERR");
        objectToReturn.successCode = SUCCESSCODE_DIFFERENTPARAMETERSLENGTH;
        return objectToReturn;
    }

    String reqBody;
    String reqEndpoint;
    if (isPostRequest){  //append parameters to Body as JSON
        DynamicJsonDocument docToSend(jsonSendDocCapacity);
        for ( uint8_t i=0 ; i<=requestKeysLength ; i++ ){
            docToSend[requestKeys[i]] = requestValues[i];
        }

        serializeJsonPretty(docToSend,reqBody);
        log_d("POST %s\n%s\n", httpEndpoint, reqBody.c_str());

    }else{ //GET request - append to query
        reqEndpoint = httpEndpoint;
        if (requestKeys!=NULL){
            reqEndpoint+='?';
            for ( uint8_t i=0 ; i<=requestKeysLength ; i++ ){
                reqEndpoint+=requestKeys[i]+'=';
                if (i!=requestKeysLength) reqEndpoint+=requestValues[i]+'&';
            }
        }
        log_d("GET: %s\n",reqEndpoint.c_str());
    }

    bool blockExec = true;

    _fa_flexaReq.onReadyStateChange([&](void* optParm, asyncHTTPrequest* request, int readyState){
        if(readyState == 4){
            objectToReturn.rawResponse =  request->responseText();
            objectToReturn.httpStatusCode = request->responseHTTPcode();
            if (objectToReturn.httpStatusCode==200){
                log_v("Req OK: %s", objectToReturn.rawResponse.c_str());
                DeserializationError err = deserializeJson(returnedDoc, objectToReturn.rawResponse);
                if (err){
                    log_e("Deserialize ERR: %s\n", err.c_str());
                    objectToReturn.successCode = SUCCESSCODE_JSONPARSEFAILED;
                }else{
                    objectToReturn.jsonResponseDoc = returnedDoc;
                    objectToReturn.successCode = SUCCESSCODE_SUCCESSFULLREQUEST;
                }
            }else{
                objectToReturn.successCode = SUCCESSCODE_SERVERDOWN;
            }

            blockExec = false;
        }
    });

    if (isPostRequest){
        if(_fa_flexaReq.readyState() == 0 || _fa_flexaReq.readyState() == 4){
            if (_fa_flexaReq.open("POST", reqEndpoint.c_str())){
                _fa_flexaReq.setReqHeader("Connection", "close");
                _fa_flexaReq.setReqHeader("Content-Type", "application/json");
                _fa_flexaReq.setReqHeader("Content-Length", String(reqBody.length()).c_str());
                _fa_flexaReq.send(reqBody);
            }else{
                log_e("Open req ERR");
                objectToReturn.successCode = SUCCESSCODE_DNSFAILED;
                return objectToReturn;
            }
        }

    }else{
        if(_fa_flexaReq.readyState() == 0 || _fa_flexaReq.readyState() == 4){
            if (_fa_flexaReq.open("GET", reqEndpoint.c_str())){
                _fa_flexaReq.send();
            }else{
                log_e("Open req ERR");
                objectToReturn.successCode = SUCCESSCODE_DNSFAILED;
                return objectToReturn;
            }
        }

    }

    while(blockExec) yield(); 
    return objectToReturn;
}

joaolrc commented 3 years ago

I had a typo in the code above. But still, it should not crash. With older version it failed without crash

boblemaire commented 3 years ago

I have no idea where this might be happening. Maybe you could use the stacktrace analyzer to find where asyncHTTPrequest is involved or otherwise investigate. You have the .elf file. What was the typo? What is the URL? How is this different from the crash originally reported in URL without trailing '/' after address causes crash on ESP32 #30?

boblemaire / asyncHTTPrequest

Fix url parser #29