🤗 [Question]: More information about compression

sebastianbuechler commented 1 year ago

Question Description

Is it possible to add more information about compression in the docs here?

Specifically, what are the minimum payload sizes for gzip, br, and deflate in order for the compression to take place? Here it says for gzip it is 1024KB, but it seems to be smaller for deflate

What happens if multiple encodings are supplied in the header. It seems that the priorities look like this:

br
gzip
deflate

What if the header specifies multiple compression algos, but the one with the highest prio does require a higher minimum payload? Wouldn't it make sense to try to compress with another algo?

Code Snippet (optional)

No response

Checklist:

[X] I agree to follow Fiber's Code of Conduct.
[X] I have checked for existing issues that describe my questions prior to opening this one.
[X] I understand that improperly formatted questions may be closed without explanation.

welcome[bot] commented 1 year ago

Thanks for opening your first issue here! 🎉 Be sure to follow the issue template! If you need help or want to chat with us, join us on Discord https://gofiber.io/discord

the-hotmann commented 1 year ago

I really want to join in on here since there are some unxecpected things I noticed:

to compress and servce static files you dont need "github.com/gofiber/fiber/v2/middleware/compress".
precompressing is not possible (AFAIK)
app.Use(compress.New(compress.Config{Level: 2,})) does not have an effect on compression for me

I think the middleware compression is broken honestly/very bad structured. To configure it I need basic fiber.Config, fiber.Static and a not working compress.Config. While all of this should be solely part of the compress middleware.

I ask myself, why precompressing is not working? compress (or static) is given the folder which it should serve files from, hence, it should precompress them all into .deflate, .gz and .br whenever it starts. If a file is requested that is not compressed yet, it should do it on the fly. Also these files should be deleted/recompressed, once the source has changed.

At least thats how I think it should work.

There also is a setting:

app := fiber.New(fiber.Config{
    CompressedFileSuffix: ".fiber.gz",
})

What does it do, when the accepted compression is deflate or br? Using the same .gz extension? I also miss the option to set the compression levels. Fastest and Best and just syntetically trimmed down options. There are:

br: 1-11 gz: 1-9

The question now is (for brotli) is BestCompression level 9 or level 11?

In general there are a lot of settings missing I think. I would structure it like this:

(defaults configured)

var ConfigDefault = Config{
    Next:  nil,
    Enabled: true,
    CompressedFileSuffix: fiber,   // suffix WITHOUT extension! If set to "" will be removed
    Debug: false,                  // sets some custom header for every file which was served compressed, which shows the compression technique and level
    Level_A_GZ: 9,                 // values from 1-9
    Level_S_GZ: 6,                 // values from 1-9
    GZ_Comp_Storage: "",           // folder to store the generated files from 'Level_A_GZ' (if not exist, will, will be created)
    Level_A_BR: 11,                // values from 1-11
    Level_S_BR: 5,                 // values from 1-11
    BR_Comp_Storage: "",           // folder to store the generated files from 'Level_A_BR' (if not exist, will be created)
    PreCompres: false,
}

Note:

LEVEL_S_* stands for the level of streamcompression. When a file needs to be compressed on-the-fly. Things need to be faster here. The files geenrated by this will NOT be written to the storage to not have differently compressed files. Also this is not for archival. This compressionlevel also should be used to serve dynamically generated html and JSON etc.
LEVEL_A_* stands for the level of archivalcompression. Basically what should be done to store it back to the storage, next to the original file or if provided to GZ_Comp_Storage, or BR_Comp_Storage.

Many questions and the docs/manual does not provide much info. Also much potential to improve in functionality. I also think ALL compression settings should vanish from:

fiber.Config
static.Config

May I tag @ReneWerner87? Thanks :)

gaby commented 3 months ago

@efectn Does #3006 fix any of the issues raised above?

I don't think pre-compression is feasible, specially if the directory has a lot of files and with the Cache that issue goes away.

efectn commented 3 months ago

@efectn Does #3006 fix any of the issues raised above?

I don't think pre-compression is feasible, specially if the directory has a lot of files and with the Cache that issue goes away.

No it doesn't. Compression part is completely related to fasthttp

gofiber / fiber