lbernstone
commented
3 months ago What does your html look like? That error happens when the file size is not provided.
Open kapyaar opened 3 months ago
lbernstone
commented
3 months ago What does your html look like? That error happens when the file size is not provided.
kapyaar
commented
3 months ago @lbernstone It is a bunch of html files (each with html, js, css, etc), image files,json files, etc that went through the mkspiffs tool, resulting in a .bin file. This is the same .bin file that was generated and successfully uploaded to the board using the ESP32 Spiffs update tool. Not sure where the file size need to be provided?
kapyaar
commented
3 months ago @lbernstone Just checking in. Could you kindly clarify where the file size need to be provided?
lbernstone
commented
3 months ago The OTAWebUpdater example shows how to get the browser to add the file size to your request, and then have the Update library use the file size.
kapyaar
commented
2 months ago @lbernstone Thank you. I am a bit confused. Some thoughts/ findings:
a. Firmware update already works fine in the Examples>WebUpdater.ino in question, without submitting the filesize. b. The html code on /update page on that example is just a form input, no JS, no file size calculation
<!DOCTYPE html>
<html lang='en'>
<head>
<meta charset='utf-8'>
<meta name='viewport' content='width=device-width,initial-scale=1'/>
</head>
<body>
<form method='POST' action='' enctype='multipart/form-data'>
Firmware:<br>
<input type='file' accept='.bin,.bin.gz' name='firmware'>
<input type='submit' value='Update Firmware'>
</form>
<form method='POST' action='' enctype='multipart/form-data'>
FileSystem:<br>
<input type='file' accept='.bin,.bin.gz,.image' name='filesystem'>
<input type='submit' value='Update FileSystem'>
</form>
</body>
</html>Update SPIFFS...
[ 8838][V][HTTPClient.cpp:242] beginInternal(): url: http://testServer.com/spiffs.bin
[ 8848][D][HTTPClient.cpp:293] beginInternal(): protocol: http, host: testServer.com port: 80 url: /spiffs.bin
[ 9518][D][HTTPClient.cpp:574] sendRequest(): request type: 'GET' redirCount: 0
[ 9526][D][NetworkManager.cpp:83] hostByName(): Clearing DNS cache
[ 9775][D][HTTPClient.cpp:1112] connect(): connected to watchdog15.com:80
[ 9864][V][HTTPClient.cpp:1201] handleHeaderResponse(): RX: 'HTTP/1.1 200 OK'
[ 9872][V][HTTPClient.cpp:1201] handleHeaderResponse(): RX: 'Date: Thu, 11 Jul 2024 16:53:35 GMT'
[ 9881][V][HTTPClient.cpp:1201] handleHeaderResponse(): RX: 'Server: Apache'
[ 9888][V][HTTPClient.cpp:1201] handleHeaderResponse(): RX: 'Last-Modified: Thu, 11 Jul 2024 16:49:02 GMT'
[ 9898][V][HTTPClient.cpp:1201] handleHeaderResponse(): RX: 'Accept-Ranges: bytes'
[ 9905][V][HTTPClient.cpp:1201] handleHeaderResponse(): RX: 'Content-Length: 1572864'
[ 9913][V][HTTPClient.cpp:1201] handleHeaderResponse(): RX: 'Connection: close'
[ 9921][V][HTTPClient.cpp:1201] handleHeaderResponse(): RX: 'Content-Type: application/octet-stream'
[ 9930][V][HTTPClient.cpp:1201] handleHeaderResponse(): RX: ''
[ 9936][D][HTTPClient.cpp:1257] handleHeaderResponse(): code: 200
[ 9942][D][HTTPClient.cpp:1260] handleHeaderResponse(): size: 1572864
[ 9948][D][HTTPClient.cpp:618] sendRequest(): sendRequest code=200
[ 9954][D][HTTPUpdate.cpp:227] handleUpdate(): Header read fin.
[ 9960][D][HTTPUpdate.cpp:228] handleUpdate(): Server header:
[ 9966][D][HTTPUpdate.cpp:229] handleUpdate(): - code: 200
[ 9972][D][HTTPUpdate.cpp:230] handleUpdate(): - len: 1572864
[ 9978][D][HTTPUpdate.cpp:242] handleUpdate(): ESP32 info:
[ 9983][D][HTTPUpdate.cpp:243] handleUpdate(): - free Space: 3342336
[ 9990][D][HTTPUpdate.cpp:244] handleUpdate(): - current Sketch Size: 1012448
[ 10278][D][HTTPUpdate.cpp:298] handleUpdate(): runUpdate spiffs...
[ 46235][D][HTTPUpdate.cpp:337] handleUpdate(): Update ok
[ 46240][D][NetworkClient.cpp:562] connected(): Disconnected: RES: 0, ERR: 128
[ 46247][D][HTTPClient.cpp:393] disconnect(): tcp is closedand it goes on to
Update sketch...
So, why does Examples>WebUpdater.ino fail when updating spiffs? To me it looks like,
Does this make sense? Or am I missing something here? Thank you again, for your time.
lbernstone
commented
2 months ago I'm not going to update that library. It probably just needs to be retired. Here's a patch for OTAWebUpdater.ino . I'll submit it as a PR once I have some time to stress test it (it seems to be a bit slower than it should be).
if (upload.status == UPLOAD_FILE_START) {
int otaType = upload.filename.indexOf(".spiffs.") || upload.filename.indexOf(".ffat.") ? U_SPIFFS : U_FLASH;
Serial.printf("Receiving Update: %s, Size: %d, Type: %d\n", upload.filename.c_str(), fsize, otaType);
if (!Update.begin(fsize, otaType)) {@lbernstone Just thought I will check back on if you had the chance to perform the stress tests on this? I would like to make a suggestion. In the Examples>WebUpdater.ino it offered two different file select options, one for firmware, one for spiffs. The benefit is that the process does not make it necessary that the filename has "spiffs" or "ffat" in it. In my opinion, this would be a nice feature to keep, when you get to work on the OTAWebUpdater example.
lbernstone
commented
1 month ago These are intended to be examples of esp32 code, not html. Those filenames are the format that the file system extension uses, so will likely be the filenames anyhow.
kapyaar
commented
1 month ago Sure, I agree. It is just that, even when the spiffs spits out a file that has .spiffs in the file name, I (and possibly others) may rename the file to some naming convention that does not have the spiffs/ ffat in the name.
In my opinion, keeping them independent of file name does not look hard, we can simply update the html file to have two inputs and have the form field hold the name.
Regardless, I appreciate your/ team in all your efforts, however you find it fitting in this case :) b/r
kapyaar
commented
1 month ago @lbernstone It seems to me, SPIFFS.totalBytes() is not returning the correct value. In a use case where 1.5MB spiffs is selected, SPIFFS.totalBytes() is returning 1438481 where as the actual value should be 1572864.
lbernstone
commented
1 month ago You shouldn't use SPIFFS any more, it is highly inefficient. LittleFS has replaced it.
kapyaar
commented
1 month ago Unfortunately, I have to support an existing project, which uses spiffs. Not for a new one. Is it reasonable to expect that the issue at hand will have a resolution?
lbernstone
commented
1 month ago What issue? That SPIFFS doesn't provide the full amount of space in a partition or the original issue on this ticket? AFAIK, SPIFFS is no longer under development. Please note, I don't work for Espressif. If you have production code which is impacted by an issue here, you need to speak to a sales rep/engineer.
kapyaar
commented
1 month ago OK, Because you are the only one who responded to the issue, and you suggested that you were not going to fix WebUpdater.ino, rather, do a PR on OTAWebUpdater example, I took it that you were part of the team. Did nt know otherwise, and did nt mean to bother you with later messages had I known that :)
As to "what issue", It looks to me the same. The reason why WebUpdater.ino is failing for bad size given is because in the Updater library, SPIFFS.totalBytes() is returning incorrect value AFAIK
@me-no-dev or @SuGlider Tagging you for your kind attention. Is the issue in this thread fixable?
lbernstone
commented
1 month ago SPIFFS.totalBytes does not return the partition size, but the maximum amount of data that could be stored (this comes from upstream). The code really should be changed to get the partition information. If you just want to get it working without writing any code, then change https://github.com/espressif/arduino-esp32/blob/master/libraries/HTTPUpdateServer/src/HTTPUpdateServer.h#L107 to if (!Update.begin(UPDATE_SIZE_UNKNOWN, U_SPIFFS)) {
Note, my objection to that library is that it is redundant, and extremely limiting, since it sets up a WebServer which can then not be extended by the user, and hard codes the html.
kapyaar
commented
1 month ago Thanks, I do understand the workaround. Even If I move to littleFS, the filesystem update method would still be the same, and need this workaround, correct? In that case, If you are recommending that I just make this change locally, everytime there is a new release, I have to keep making this custom modification. Not a good situation to have. If the general agreement is that SPIFFS.totalbytes() is not going to work, and will not be updated, would it make sense to update to the source to reflect the change you mentioned in L107?
lbernstone
commented
1 month ago LittleFS gives the partition size for LittleFS.totalBytes(), so it should just work. If you want to submit a PR to fix this library, then please take into consideration my comments about the limitations and match it more closely to OTAWebUpdater.ino so that it will be maintainable in the future (maybe even to the point of using the same html.h file).
Board
ESP32 Dev Module, 8MB Flash
Device Description
ESP32 Dev Module, 8MB flash
Hardware Configuration
8MB ESP32 Dev Module. No attachments
Version
latest master (checkout manually)
IDE Name
Arduino IDE 2.3.2
Operating System
Windows 11
Flash frequency
40MHz
PSRAM enabled
no
Upload speed
921600
Description
I am using 8MB ESP32 Dev Module, with the following IDE settings.
I use arduino IDE 2.3.2 for firmware but go back to 1.8.9 to use the ESP32 Sketch Data Upload Tool using spiffs. This works fine when using the IDE. But if I take the same spiffs file that is generated using the mkspiffs tool, use the webupdater.ino example, and use the Filesystem update option, It throws
Update error: Bad Size GivenSketch
Debug Message
When trying to access /update
When selecting SPIFFS FILE and submitting
On the webpage,
Other Steps to Reproduce
Tried on few different boards, similar result.
I have checked existing issues, online documentation and the Troubleshooting Guide