request->send(200, textPlainStr, jsonChartDataCharStr); - Without using String Class - to save heap

[salasidis]

Is your feature request related to a problem? Please describe.

I have a relatively web page - about 100kBytes. In addition, I need to send large amounts of Json data - about 1Mbyte sensor data via the async web interface.

I save all the data in the SDRAM (the web page as well as the Json data string (created from SD Card logged data).

When responding to the request via request->send(200, textPlainStr, jsonChartDataCharStr); // jsonChartDataCharStr is a C type char * string

The library converts the (char *) array (JSon or the web page) into a String class variable before sending. This results in large amounts of heap use, and basically I cannot send more than 4 months of telemetry data (whereas the design calls for at least 2 years).

Describe the solution you'd like

I would like to have a request->send call that looks at the variable passed, and if it is a char * leaves it as such, without changing it to a String. Not only will this be faster, but it will also use a LOT less memory

Describe alternatives you've considered

My alternatives of making many calls to retrieve the data in small batches would add needless complexity, and also require multiple accesses to the SD card, which I want to avoid

Additional context

This would not be an issue if heap space was not so critical, but the portenta only has a small finite amount, and this library consumes about 80+% of it in my application in a single function call.

[salasidis]

Maybe the request belongs in Portenta_H7_AsyncWebServer, but the idea is the same

[salasidis]

It would be nice if any mallocs could be limited, or at least made to optionally use the SDRAM, and not the main RAM.

Also any string manipulations (adding a header etc) could be done in place - just having to ensure that the passed string is at least a minimum size larger than the contents it holds.

[khoih-prog]

Try on ESP32 / ESP8266 first, using ESPAsyncWebServer to see your code is OK and possible to send such a large data.

If everything is OK, then go back here with all the terminal output to prove, report the status before I spend time to look at it. Certainly with MRE, etc.

I'm closing the issue now, and will reopen after you provide all the requested info.

Good Luck,

[salasidis]

Will post here - not sure if it reopens (otherwise will create another).

I do not have access to an ESP32 board, but I used the AdvancedAsyncWebServer example to demonstrate the issue.

On the default program, I added some code to print the heap size, and I increased the graph array from 50 to 500 points.

On the modified program, I created a C string instead of an Arduino String, and sent the same 500 line graph svg file

There is a word file that shows the outputs, and the outputs demonstrate that if sending an Arduino String, the software requires 2x extra heap space on top of the the actual string size.

If sending a C string, it takes 3x.

My proposal is to have an option (separate call, or modify the current ones), that can use the c string as is, and simply pre and post append to it any headers and footers required, without creating new Arduino Strings in the library

(having trouble posting ZXIP file - will try after submit this)

[salasidis]

Will open another Issue - ? can't upload file since this one is staying closed?

[khoih-prog]

HI @salasidis

Thanks for your zip file (I finally got it), and at least now I know the issue you're talking about.

Normally, for short sketches, you can copy-and-paste directly onto the issue (with code tags), don't use complex and unsafe to-be-downloaded zip. Check how I did in the new issue.

I'm opening the new issue in the correct place Portenta_H7_AsyncWebServer as request->send(200, textPlainStr, jsonChartDataCharStr); - Without using String Class - to save heap #6

See you there,

Regards,