[Feature Request] Optional Asset Compression

[quinnjr]

Feature Request - Optional Asset Compression

This feature request asks for the implementation of optional asset compression by trunk.

trunk would follow a similar pattern to webpack-compression-plugin by allowing users to configure --build asset compression for all files generated by trunk.

NOTE

In relation to https://github.com/thedodd/trunk/issues/81, I've checked and confirmed that actix-web and warp support serving compressed assets.

Example

Cargo.toml changes

...

[features]
compression = []
gzip-compression = ["compression", "flate2"]
...

[dependencies]
...
flate2 = { version = "1.0.18", default-features = false, features = ["tokio", "zlib"], optional = true }

Configuration in Trunk.toml

...
[[compression]]
# Specifies the compression algorithm.
algorithm = "gzip" 
# Options to pass to the compression algorithm. Optional.
# Ex: Passing the `level` option to gzip
options = { level = 6 }
# A RegExp test to pass to include assets that pass the RegExp test. Optional.
test: "/\.(js|css|png|wasm)(\?.*)?$/i"
# Allow for inclusion of certain assets via glob. Optional.
include = []
# Allow for exclusion of certain assets via glob. Optional.
exclude = ["favicon.ico", "*.gz"]
# Size of asset (in bytes) that should be compressed. Less than the threshold will not be compressed. Optional.
threshold = 8192
ratio = 0.8 # Minimum compressed asset size ratio for output. Optional.

Algorithm Targets

• [ ] zlib (Deflate, gzip) (flate2)
• [ ] brotli
• [ ] zopfli (Rust support??)
• [ ] zstd (zstd, gzip)

[thedodd]

@quinnjr excellent feature request. I'll get this triaged soon, and add it as part of a new milestone for moving off of Tide.

[quinnjr]

One thing that I think would be helpful for adding this feature would be a more virtual pipeline-like build system. Currently, from what I've dived into the core code of trunk, the system would require writing the non-compressed assets to the filesystem before working on compressing them, which would also require a separate step to regenerate index.html with the proper filenames or proxy configuration to serve the compressed assets before uncompressed assets, which I have no experience doing on something like a CDN service.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[9SMTM6]

FYI: I've been able to get brotli compression somewhat working with the server by adding .precompressed_br() here and 4 lines below that, and by manually running brotli dist/<assetname> after trunk serve --release.

I tried to get this working with a hook, but it seems the hook results were removed after it ran? I saw the br file during compression, but then when the hook finished up it was deleted.

This would not be the real implementation, of course. If we add compression in the form of precompression, then this should probably only be done in release, or be configurable at least, as at least brotli on linux takes longer than all other steps combined. And this feature request asks for a lot more configurability.

But it shows it should be possible for someone knowledgable in this codebase without TOO much hassle, now that this uses axum, and that axum added support for this.

And I've also got to add that the results of compression on wasm are pretty nice. Better than any other trick I tried. The test-project uses egui + wgpu with webgl to webgpu translation, this leads to an ~5MB wasm bundle with a lot of other tricks (~7.3MB without these). With brotli and default settings that wasm bundle gets compressed to ~1.5MB.

[ctron]

Sounds cool. I am wondering what role axum would play here. As axum is only used for the trunk serve part, which is for development only.

[9SMTM6]

I am wondering what role axum would play here. As axum is only used for the trunk serve part, which is for development only.

Thats correct, this was a result of the opening post. It refers to a proposed switch of trunk to other frameworks for trunk serve (or an equivalent). So serving the compressed files was something that they did want to do for the development server.

Combine that with me not having setup serving of compressed artifacts before.

In the meantime I found the reason my hook did not work, it was not compressing the file in the staging directory but the (old) distribution dir.

I've got compression working with github pages. Sadly they don't seem to support serving brotli precompressed files, but gzip works.

Anyways. As you point out, the question is whether the development server should serve compressed files at all. I think that it'd be nice to be closer to the proposed deployment, however as seen with the github pages situation, just serving some compressed files doesn't mean it's the same thing as done in deployment. Whatever server you use will have to be configured, and based on that configuration the deployment might do something radically different.

So I'm not sure what you guys envision the feature to look like precisely. I'd say that either the compression should only be done with --release or it should be configurable. Other than that, the proposal in the opener has a bunch of configuration for compression. I can't imagine all that configuration making development easier. I'd like to note that eg. wasm-opt only supports mininal configuration in trunk (the optimization level, um unable to set eg. The fast math flag). So I'm not sure that in the current state so much configuration for the proposed feature is required.

[ctron]

I think it makes sense to have that, and make it configurable (release or not).