Closed ENG-WiPhilia closed 2 months ago
Hi Juan Carlos,
Thank you very much for sharing this.
I'm assuming you're referring to HTTP's Chunked Transfer Encoding.
This encoding was added in version 1.1 of the protocol, so your server must not use it if the client uses HTTP/1.0
.
I promised I would add an adapter to handle chunked transfer encoding in my StreamUtils library, but I never got the time to do it. I'll try to allocate some time after the release of 7.1.
Best regards, Benoit
In my project, I needed filtering a long answer comming from HTTP TLSv1.2 encrypted server connection. HTTP 1.0 protocol chunk the text answer when the data is long. I developed the TLSCustomReader class for cleaning the HTTP headers and the chunked characters for leaving a continuous stream of acceptable chars to the JSON Document to allow filtering correctly the incomming data. How is not a WEB Server, but a very specific application server, I have to deal with the HTTP 1.0 protocol.
I am very impressed with you work in doing the ArduinoJSON library, really, I think this is the most organized and documented library I have seen doing my project. I do not know if You have worked yet with solving HTTP chunked data comming into the CustomReader Class. If you need some idea of how doing that, i bring you my solution as a contribution to your project or to your library users.
The Solution in C++ implemented in arduino IDE: // the JSON Document with the filter options to be applied to the incomming stream JsonDocument hueFilter; // arduinoJSONv7 // the JSON Document with the incomming stream data JsonDocument doc; // ArduinoJSONv7 // // // initialize finalPack // // txData = 0; //Create the JSON CustomReader for Steam read() function using the TLS connection TLSCustomReader CustomReader(connectTLS, txData); // Deserialize the JSON directly from the custom reader DeserializationError error; error = deserializeJson(doc, CustomReader, DeserializationOption::Filter(hueFilter));
Note that connectTLS was created before [connectTLS = createTLSConnection(connectNum); ] ;however, i think this is not required for the implementation of the customer reader objet of this contribution, but you can demand for it if you required the connectTLS is used to send the data [ret = esp_tls_conn_write(connectTLS, httpRequest + written_bytes, strlen(httpRequest) - written_bytes)] and to receive the data in the custom reader class:
struct TLSCustomReader { // ArduinoJSON Custom Reader Class // convert Steam read() to readable JSON Document data in eliminating // http headers and chunk information from Http1.1 at the refill buffer // ** // this class is used to fill the ArduinoJSON Document // ArduinoJSON Document to replace String Manipulation of version 7 // More memory is required comparing with plain String if no filtering // is applied, filtering required to reduce memory usage then // this struct is required as Custom Reader for put filtered // Stream serial information into the ArduinoJSON Document // ***** esp_tls_t connect; static const size_t bufferSize = 1024; const char chunkInd = "\r\n"; char buffer[bufferSize]; char jsonBuf[bufferSize]; char hexBuffer[16]; size_t hexlength; size_t bytesRead, bytesReady, charsRead; size_t pos, posTemp; bool chunked, cutChunk, headers, finalPack; unsigned long remainData; bool lookHTMLerr;
I implememted this solution and after many months and hundreds of streams received using it, i have not errors in the received data. I know is a lot of information, but you can feel free to contact me if you need details or clarify any of the precedent content.
Juan Carlos Gomez Casal jcgcasal@gmail.com
Note: I can write/speak in french if you prefer.