With new code, frogfs is much better tooled for embedded storage of a website.
On the web, the best supported compression algorithm is brotli. All browsers support it (96.93% support in caniuse), and it provides a strong compression advantage (for example, gzip encoded JS file will take 660 bytes while brotli will only use 440 bytes)
So this PR mainly add brotli as a transformation filter, like gzip and deflate used to do it.
It doesn't add any code in the microcontroller for decompressing the brotli's data, it's only used as a preprocessing step to reduce the filesystem size. The microcontroller will simply send the data as-is when requested.
Fixes
While testing this PR, I've found few subtle bugs that I corrected.
In the current code, the later filter on CSS files will apply uglifycss on a gzipped input. That because, when the xforms dict is created in apply_rules, it first find the '*' filter, thus a gzip is transform is created. Then it finds the '*.css' filter, and create the uglifycss transform (it later find the gzip filter again, but since it already exists, it only replace it, not changing the order in the dictionary). You end up with a dict containing gzip, uglifycss in that order.
I've solved that simply by ranking the filters. A filter with a higher rank will overwrite the xforms dict thus preventing bad transforms order in the dictionary. A lower rank filter will be ignored if an higher rank filter was processed before it. I think it's the most logical solution, since you don't want to mix the actions of filters with higher specializations.
no transform crashed the processing
In the current code, if you have a no something filter, the ent["transform"] will contain a False argument (as a way to prevent processing this action). Yet, the actual code doing the processing was only expecting a dict, crashing the processing. Now, the processing handles the case carefully by bypassing the processing as expected.
Purpose
With new code, frogfs is much better tooled for embedded storage of a website. On the web, the best supported compression algorithm is brotli. All browsers support it (96.93% support in caniuse), and it provides a strong compression advantage (for example, gzip encoded JS file will take 660 bytes while brotli will only use 440 bytes)
So this PR mainly add brotli as a transformation filter, like gzip and deflate used to do it.
It doesn't add any code in the microcontroller for decompressing the brotli's data, it's only used as a preprocessing step to reduce the filesystem size. The microcontroller will simply send the data as-is when requested.
Fixes
While testing this PR, I've found few subtle bugs that I corrected.
Transform order wasn't respected
Let's say you had this in your YAML file:
In the current code, the later filter on CSS files will apply uglifycss on a gzipped input. That because, when the
xformsdict is created inapply_rules, it first find the'*'filter, thus agzipis transform is created. Then it finds the'*.css'filter, and create theuglifycsstransform (it later find thegzipfilter again, but since it already exists, it only replace it, not changing the order in the dictionary). You end up with a dict containinggzip,uglifycssin that order.I've solved that simply by ranking the filters. A filter with a higher rank will overwrite the
xformsdict thus preventing bad transforms order in the dictionary. A lower rank filter will be ignored if an higher rank filter was processed before it. I think it's the most logical solution, since you don't want to mix the actions of filters with higher specializations.no transform crashed the processing
In the current code, if you have a
no somethingfilter, theent["transform"]will contain aFalseargument (as a way to prevent processing this action). Yet, the actual code doing the processing was only expecting adict, crashing the processing. Now, the processing handles the case carefully by bypassing the processing as expected.Fixing spell error