response compression support

[manniL]

Describe the feature

Related: https://github.com/nuxt/nuxt/issues/19411

It'd be nice to have a way to enable compression not only for static assets but also dynamic responses via Nitro's node-server preset.

Additional information

• [X] Would you be willing to help implement this feature?

[pi0]

PR is welcome as an opt-in feature. This should be used by care btw because of performance overhead implications, compressing on CDN / Reverse proxy layer is always better idea.

[atinux]

Shall we enable compression by default for static assets when enabled via serveStatic?

Nevermind, just saw compressPublicAssets option

[manniL]

@pi0 I agree! Maybe not even as opt-in feature but as code snippet/example with a plugin and hooks 🤔

[VladislavYakonyuk]

When can we expect support in Nuxt 3?

[deshbirc]

PR is welcome as an opt-in feature. This should be used by care btw because of performance overhead implications, compressing on CDN / Reverse proxy layer is always better idea.

@pi0 A CDN is ideal for compressing binary files and assets like images, scripts, styles, and fonts. However, for text compression, specifically for HTML payloads from SSR, a CDN may not be suitable. Adding a reverse proxy layer just for text compression can be an overkill for some applications e.g. an ECS task/container behind an AWS load balancer.

[das-nirmal]

Hi guys, I have enabled text compression in 'render:reponse' nitro hook as shown below - any feedback / concern / red-flag would be welcome!

import zlib from 'node:zlib'; 
import { promisify } from 'node:util';

const gzip = promisify(zlib.gzip);

export default defineNitroPlugin((nitro) => {

  nitro.hooks.hook('render:response', async (response, { event }) => {

    if (!response.headers['content-type'].startsWith('text/html'))
      return;

    // Inspect or Modify the renderer response here
    const compressedBody = await gzip(Buffer.from(<string>response.body, 'utf-8'));
    setHeader(event, 'Content-Encoding', 'gzip');
    send(event, compressedBody);
  })
})

[pi0]

Thanks for the snippet dear @das-nirmal I haven't locally verified it but looks fine (you could also use gzip streaming see this h3 example).

Using the new render:response seems a good idea if we want to support built-in compression as an option 👍🏼

[CodeDredd]

Hey there, @das-nirmal I have a smiliar solution like you. Although yours is using better functions. My first approach was this:

import zlib from 'node:zlib';
import type { ZlibOptions, BrotliOptions } from 'node:zlib';
import { isArray } from '@vue/shared';

export default defineNitroPlugin((nitroApp) => {
    nitroApp.hooks.hook('render:response', async (response, { event }) => {
        const acceptedEncoding = event.node.req.headers['accept-encoding'];
        if (acceptedEncoding && !isArray(acceptedEncoding) && response.body) {
            if (/gzip/.test(acceptedEncoding)) {
                const content = await compress(Buffer.from(response.body), 'gzip');
                event.node.res.setHeader('content-encoding', 'gzip');
                event.node.res.setHeader('content-type', 'text/html;charset=utf-8');
                event.node.res.end(content);
            }
        }
    });
});

export type CompressionOptions = Partial<ZlibOptions> | Partial<BrotliOptions>;

/**
 * Compression core method
 * @param content
 * @param algorithm
 * @param options
 */
function compress(
    content: Buffer,
    algorithm: 'gzip' | 'brotliCompress' | 'deflate' | 'deflateRaw',
    options: CompressionOptions = {}
) {
    return new Promise<Buffer>((resolve, reject) => {
        // @ts-ignore
        zlib[algorithm](content, options, (err, result) =>
            err ? reject(err) : resolve(result)
        );
    });
}

I am going to create an complete working solution with all the suggestions from @pi0 and better syntax. This plugin is going defently complexer.

[Hebilicious]

@pi0 Once we migrate to a monorepo and extract the presets, could we also consider publishing some "official" middlewares/plugin that can easily be imported/used ? Compression is a good candidate.

[pi0]

@Hebilicious Yes i think we might introduce similar modules concept for nitro for such reusable solutions.

On compression, i am still not sure (sine global interceptors are new in h3) but we might support it via a h3 middleware for compression depending on platform agnostic CompressionStream since now we have built-in compression support.

[CodeDredd]

@pi0 I looked at the compressionStream and it is great. Although it only supports deflate and gzip rihgt now. So for brotli this won't work yet.

Do i get you right that you see this plugin more in h3 than in nitro ?

[pi0]

@CodeDredd Indeed and yes i aim for h3 at some point. But i think if you want to you can directly go ahead and introduce this in nitro (behind an experimental flag). I would personally go with CompressionStream even with limitations considering it is a WebStandard and feature will be cross platform by default (and is ready to use for stream as well...). gzip should be considerably good enough for simple json/html response compression

[CodeDredd]

@pi0 Ok i get you point. See it also in h3. I could make an package h3-compress or add it directly as utility function to h3. After that their can be an experimental flag for it in nitro.

Do you see it more in a seperate package or in h3 directly ?

[pi0]

A seperate package would be also nice start in meantime so you can freely experiment different versions. Feel free to ping me and make a PR to include in h3 docs 👍🏼

[atinux]

I like the idea of h3-compress package!

[CodeDredd]

Ok its done. I created a package for it. 🎉 https://github.com/CodeDredd/h3-compression

I noticed also that the stream compresssion can't be used in the nitro plugin. Maybe you have any toughts on it @pi0 ?

[pi0]

Amazing work @CodeDredd Do you mind to help making a nitro sandbox or playground somewhere? I would be happy to investigate possible issues.

(Small note since it is still early stage, h3-compression could be also a nicer name)

[CodeDredd]

@pio Ok i am on it....changing the name now fast. since your right. its still early stage.

[CodeDredd]

@pi0 Here is the reproduction: https://stackblitz.com/edit/nuxt-starter-ffgnpo?file=server%2Fplugins%2Fcompression.ts Unfortunately i don't know how to change the node version. It is running on 16 at that doesn't support CompressionStream

[pi0]

Let's move discussion about investigations to https://github.com/CodeDredd/h3-compression/issues/1 👍🏼

(I will also check with stackblitz team about CompressionStream support)

[CodeDredd]

@pi0 i think this ticket can be closed right?

[pi0]

Well yes but let's keep tracking it in nitro until we have a built-in feature. Even tough i would still strongly recommend to use reverse proxies to offload compression, i think this feature worth to be considered for nitro core.

BTW thanks so much for your work on h3-compression @CodeDredd it will be certainly be used as inspiration for next steps ❤️