search

Aircoookie / WLED

Control WS2812B and many more types of digital RGB LEDs with an ESP8266 or ESP32 over WiFi!
https://kno.wled.ge
MIT License
14.83k stars 3.2k forks source link

Idea: gzip compress webui code and serve it with "content-encoding: gzip" #72

Closed wiesendaniel closed 5 years ago

wiesendaniel commented 5 years ago

While digging into the code I was thinking of ways to save some space for better support of 512k chips.

As far as I understand unminified and uncompressed html/css/js code gets bundled into the binary. If files are preprocessed to gzip format and then bundled it might save a ton of space. In addition the http response needs to contain the content-encoding: gzip header. And that should be it.

To reduce size even further it might be helpful to put html, css and js into separate files. Those could be used to reduce code duplication. Multiple definitions of background colors and such. On top minifiers like UglifyJS can reduce code footprint by shortening variable names and removing whitespace chars e.g.

What are your thoughts on that?

Aircoookie commented 5 years ago

Thanks for making me aware of this! I didn't know you could apply compression to HTML and browsers being able to decompress it! Seems like browser support is not an issue either. A first test has shown that gzip would compress the 35k the index.html has to just 11k. This is amazing, since it would allow for supporting all the features again on 512k chips. I'll try to implement this today. It has another advantage, on modern phones and PCs we should be able to assume a low decompression time, which should bring the page loading time on ESP8266 down from 1.5sec to 750ms or even better.

Multiple files for js, css and html is something I'd do on a larger server. With the 8266 though, I've experienced it crashing or not responding for 10-30 seconds when sending too many requests at once. So, I'd rather want to keep everything together in index.html to only have a single request/response.

I'm already using this minifier. It reduces the size by 10-20%, but the 60-70% we'd save additionally using gzip are great!

Aircoookie commented 5 years ago

Alright, there is a small roadblock. The index.html is not static, the theme colors are injected during sending. The mobile UI doesn't have that, so I'll get it to work with it first before finding a solution to the classic UI. (the page already sends another request to get the current color, the response could also contain the color theme to apply). This could also be used to add theme capabilities to the mobile UI, which I'd like to do.

wiesendaniel commented 5 years ago

I'm glad that this seems to work out.

You are right. To load multiple files could be a problem with a single threaded webserver.

Gzip itself should do a good job on it's own. I do a lot of web stuff for work. The benefits of minifiers when already using gzip is at least questionable.

Maybe a single page application approach (just one html file and the side is put together by javascript) is a thing to consider but might not be worth it in terms of js overhead.

But the dynamic content I would handle with JS. A simple Json interface which presents the dynamic values as http interface and a bit of Ajax magic in js should do the job. https://www.w3schools.com/xml/ajax_intro.asp

Then your html is all static again and can be compressed. Which, depending on the server config, might as well be cached in the browser and mean a benefit in load time.

Aircoookie commented 5 years ago

Great! I just got gzip compression working with the mobile UI! It drops the binary size by 20k and reduces the page loading time from 1.5sec to 700ms.

Yes, ajax over XML is what I already use for receiving the current LED color/effect/... in the UI. I'll just extend it to also send the theme colors on the first load, then I can easily have a static HTML.

Once I'm done with that, I'll release v0.8.2 which will only contain these size/speed improvements before starting to work on v0.9.0.

Even though gzip will reduce the binary size by about 45k, that is still not enough to support all features on ESP-01, but at least the mobile UI will be working! You just need to opt-out of one other feature you don't need (either Blynk, Alexa, or hue sync).

Once gzip is ready, I'll also give local caching a try. It should make the page load instantly after the first time. To still support instant updates of the UI together with new software, I'd probably use the "ETag" method for cache invalidation.