search

luc-github / ESP3D

FW for ESP8266/ESP8285/ESP32 used with 3D printer
GNU General Public License v3.0
1.73k stars 464 forks source link

[BenchMark]Web upload / Web Download on different FS and platform #576

Open luc-github opened 3 years ago

luc-github commented 3 years ago

I think it would be very informative to bench the speed of upload / download a file using FTP / WebDav / Webserver to LocalFS / SD on ESP32 and ESP8266 using MCVE to be sure the comparaison is correct

Target Speed KB/S Note
ESP32 + WEBSERVER + FAT 56.01
ESP32 + WEBSERVER + LITTLEFS 59.60
ESP8266 + WEBSERVER + LITTLEFS
ESP32 + WEBSERVER + SD native 75.06
ESP8266 + WEBSERVER + SD native
ESP32 + WEBSERVER + SD Fat 101.9 ~122.4 SDFAT 2.1.2
ESP8266 + WEBSERVER + SD Fat
ESP32 + WEBSERVER + SDIO
ESP8266 + WEBSERVER + SDIO
ESP32 + WEBDAV+ LITTLEFS
ESP8266 + WEBDAV+ LITTLEFS
ESP32 + WEBDAV+ SD native
ESP8266 + WEBDAV+ SD native
ESP32 + WEBDAV+ SD Fat
ESP8266 + WEBDAV+ SD Fat
ESP32 + WEBDAV+ SDIO
ESP8266 + WEBDAV+ SDIO
ESP32 + FTP+ LITTLEFS
ESP8266 + FTP+ LITTLEFS
ESP32 + FTP+ SD native
ESP8266 + FTP+ SD native
ESP32 + FTP+ SD Fat
ESP8266 + FTP+ SD Fat
ESP32 + FTP+ SDIO
ESP8266 + FTP+ SDIO

Others FW:

FW Target Speed KB/S Note
grblHAL ESP32 + FatFS + SDCard + WebDAV 300~350
grblHAL iMXRT1062 (Teensy4) + FatFS + SDCard + WebDAV 1127 Ethernet
grblHAL STM32F7xx + FatFS + SDCard + WebDAV 350 Ethernet

Adding with PSRAM could also be a thing to check on ESP32

Once done need to compare with speed in ESP3D / ESP3DLib and GRBL_ESP32 to see what must be improved and how This is based on feedback ESP32 WebServer upload is very slow compare to ESP8266

Using any asyncwebserver / external webserver is not the current target so it is out of scope - the goal is to do a status of current situation to plan improvements / changes

luc-github commented 3 years ago

this PR may solve the speed issue https://github.com/espressif/arduino-esp32/pull/4787 (not tested)

luc-github commented 2 years ago

Just listing files with Native ESP32 SD is slow in core 1.X, but it 10 times slower or more with core 2.X when using SPI, so it is a must to use the SDFAT2 on ESP32 with SPI SD Native Core 1.x SDListing-1 x

SD Native Core 2.x SDListing-2 x

SDFat2 FAT2

luc-github commented 2 years ago

I have added a simple bench check in FS file upload abd SD File Upload in ESP3DLib 3.0 - and tested ESP32

image

same test should be done on ESP3D to see if using core 0 instead of core 1 affect the perfomance

I will test other configurations when I have time

liqijian101 commented 2 years ago

hi luc,I test esp32s2 with sdfat2.0 in spi mode .I find that the sector size of tf card can influence the speed of upload files.

luc-github commented 2 years ago

what size did you set? what speed did you get?

liqijian101 commented 2 years ago

A low speed card, 8g capacity, 2048ksec, Max speed 270ksec a high speed card, 16G capacity, 64ksec, Max speed 190KSEC, and I found that SDFAT2 does not support 32G cards

liqijian101 commented 2 years ago

I also found that using esp32s2sdfat2.0 is faster than using 1.0

luc-github commented 2 years ago

I did configuration with compatibility with 3Dprinter FW in mind, so 4G/8G is the target, 16G may be also supported, be sur you use https://www.sdcard.org/downloads/formatter/

and use Fat32 format I did tested 32G/64G/128G in past but long time I did not recheck the support, and definitly not with SDFat 2.x

terjeio commented 1 year ago

Some grblHAL speeds with FatFS + SDCard + WebDAV, using WinSCP for testing:

RP2040 (Pi Pico W) support for WebUI is on my todo list, will be interesting to see how this fares...

luc-github commented 1 year ago

@terjeio thank you I have added your figures on top

luc-github commented 1 year ago

For information @pipi61 reported https://github.com/luc-github/ESP3D/issues/491#issuecomment-766157740 that esp32 + webdav + SDIO can reach 500KB/s

Additionnaly Reprap porting team, that added esp32 support to duetwifi, claim that they reach 1.2MB/s, but in case of Duet wifi, esp32 is only used as data bridge, the webserver is hosted by the main MCU, and I think the SD card is SDIO one, but communitation between ESP32 and MCU are SPI only - TBC as I never took time to dig in

But it shows that there is area of improvement 🐱