chmorgan / libesphttpd

Libesphttpd - web server for ESP8266 / ESP32
Mozilla Public License 2.0
128 stars 45 forks source link

Static files handling #92

Closed yzazik closed 3 years ago

yzazik commented 3 years ago

Hello, (sigh) well, this is not really an issue any more, or maybe not yet again since I tried to use this code to give my web server a bit of asynchronicity but using it turned out to be more problematic than I expected so I left this idea alone for now, but. I thought that the idea behind serving static files should have been in... well, not serving them as dynamic ones rather than letting them to be "cached forever" (like that was ever any good). By that I mean, rather send (initial) response with E-Tag or Last-Modified with max-age let's say 2-5min (I'd rather send "304 Not Modified" more often than let the file be cached for a year and keep pressing F5 on every page I switch to) just enough time till I re-upload a new image :-). And then there is no need in static file distinction since with this strategy dynamic content can be served the same way as long as dynamic stuff hasn't expired. E-Tag is some sort of CRC/MD5 and rather identifies the file content rather than the file properties (name, location, mtime, etc). Kinda heavy to evaluate on the fly and storing it with the file would mean that ESPFS would need either extra flag and field to save it after the header or trail after the file content similar to NTFS streams which I guess a bit cleaner way of having filename.md5 files around. Last-Modified requires just file MTIME (modified time) which is, guess what, not in ESPFS either. It's a shame really, reserving 256 bytes for a file name using only like a third of it and not sacrificing 4 bytes for time_t. One could use app image compile time instead but that only works if ESPFS is linked with the image. And again, adding more fields in the header without breaking backward compatibility would probably require as much code as rather using dynamic headers (like TLV encoding). And couldn't really send 304 Not Modified since everything is send with "OK" (304 OK in this case) and buried deep down, more copy-paste-modify code, making original one... well, left out. Someone really have to remove ESP8266 tag from the page, this code won't build on v3.3 8266 SDK. Mostly because... yep, guessed right, ESPFS, no similar mmap, no pyhton3, no heatshrink2 that currently works there.

phatpaul commented 3 years ago

That was a bit hard to follow, but I think you're wanting to optimize the browser's caching of static files. I have worked on that for some time and have a solution that works well for me.

  1. index.html is always served with the header: Cache-Control: no-store, must-revalidate, no-cache, max-age=0 so that it is never cached by a browser (a well-behaved browser). Maybe not ideal, but it's only a few kB.
  2. my static content (other than index.html) is served from a subdir /static/ and httpd is configured to serve files within with: Cache-Control: max-age=365000000, public, immutable. That tells the browser that it can cache those files forever. The only way I can ensure that the browser refreshes those files is to rename them (or add some cache-busting query with a random number which is ignored by httpd i.e. http://myserv/static/staticfile.js?cache-buster=23552143423 )
  3. my static content is generated with a build script (gulp, google-closure-compiler, npm, and many gulp plugins) which adds a hash to the filename of each file, generated from the file's content. i.e. static/app-e61e3967ff.min.css
  4. The build script automatically modifies the Githubissues.
  5. Githubissues is a development platform for aggregating issues.