Make it possible to replace ESPFS

[UncleRus]

It would be great if users will be able to replace espfs with the custom file system, e.g., FatFS on MicroSD, spiffs, etc.

[chmorgan]

Have you had a chance to take a look at the vfs cgi function?I haven’t used it yet but feedback and patches welcome. Oh and is there any mention of that function in the readme? If not there should be.

[phatpaul]

I'm going to try to use ROUTE_CGI("*", cgiEspVfsHook)

But shouldn't I disable the ESPFS since it will not be needed? If I delete the /html dir from my build machine, the build fails. Should I just empty the dir instead?

[chmorgan]

@phatpaul I've had a chance to use those vfs functions on Linux at least and they are working as expected.

Haven't considered that approach on the esp side. I'd imagine removing the files in that directory is a good idea. It would be helpful maybe to have a Kconfig option to disable ESPFS building.

[phatpaul]

I've got it working with FAT fs. Mounted using esp_vfs_fat_spiflash_mount(). But the transfer speed is quite slow. I suspect that the FILE_CHUNK_LEN 1024 is causing the slowness.
But increasing it to 2048 caused my program to crash.
First there was a stack-overflow in httpd task. Then I doubled the stack, then it just hangs.

Do you have any recommendations for speeding up the transfers? Thanks.

[chmorgan]

@phatpaul I'm using it here with a spiffs filesystem but not with a lot of data. Imo it would be helpful to profile where the filesystem interactions occur, measure the time around the filesystem reads and confirm that this is the issue. Is this FAT fs in internal flash memory or on an external device etc?

[phatpaul]

The FAT fs is in internal flash memory. Nevermind, I think the slow-down was because I was printing info to the console at each chunk. Removing that sped it up.
Anyway, would you please point me to some guidance on how to profile/optimize my application? I have finally gotten JTAG working with openOCD on the WROVER dev-kit.

My next question (let me know if I should start a new issue): I'm getting these errors after refreshing the web client browser:

I (30042) cgiEspVfsHook: GET: /spiflash/html/index.html
I (30082) cgiEspVfsHook: GET: /spiflash/html/css/themes/default/jquery.mobile-1.4.5.mi
n.css
I (30092) cgiEspVfsHook: GET: /spiflash/html/css/index.css
I (30102) cgiEspVfsHook: GET: /spiflash/html/libs/jquery/jquery-1.12.3.min.js
I (30112) cgiEspVfsHook: GET: /spiflash/html/libs/jquery/jquery.mobile-1.4.5.min.js
I (30122) cgiEspVfsHook: GET: /spiflash/html/index.js
I (31272) cgiEspVfsHook: GET: /spiflash/html/img/hue-slider-23.png
I (31272) cgiEspVfsHook: GET: /spiflash/html/img/sat-slider-23.png
I (31282) cgiEspVfsHook: GET: /spiflash/html/img/bright-slider-23.png
I (31302) cgiEspVfsHook: GET: /spiflash/html/css/themes/default/images/ajax-loader.gif

I (64252) cgiEspVfsHook: GET: /spiflash/html/index.html
E (64512) httpd-freertos: accept failed
accept: Too many open files in system
E (64512) httpd-freertos: accept failed
accept: Too many open files in system
E (64522) httpd-freertos: accept failed
accept: Too many open files in system

Does cgiEspVfsHook not close the files after it serves them?

[chmorgan]

Hi @phatpaul.

This looks like the client has established too many http/https simultaneous connections with the server. I've seen this before and I've always increased the number of connections to avoid the issue. I do suspect there is something that libesphttpd is doing wrong though but I haven't been able to figure it out.

I would try increasing the number of connections, that should resolve it. There is also a limit in lwip for total sockets that you'll want to increase via Kconfig / make menuconfig.