minetest / contentdb

A content database for Minetest mods, games, and more
https://content.minetest.net
GNU Affero General Public License v3.0
93 stars 45 forks source link

Add image optimiser/resizer for cover images #324

Closed rubenwardy closed 1 year ago

rubenwardy commented 3 years ago

Here's the best general idea, I think: you have 3 types of images: thumbnails, reencodes, and originals. Originals you present when they click a screenshot thumbnail to blow up the full sized screenshot. Reencodes you use in package detail page banners and Featured. Thumbnails you obviously use where you're already using them. When a user first uploads an image, use the original as both the original and reencode temporarily, and then generate the thumbnail the way you've been doing so for now. Then, you have a periodic job that runs with lower priority but continuously churns through a work queue of producing high-quality re-encodes and higher-quality thumbnails, and replacing the temporary versions. You'll of course have to keep track of the queue. I use guetzli, which spits out plain JPEGs which are compatible with all the usual decoders, but it does a heavier search to find more optimal quantizations and has I think a slightly more refined psychovisual model or something. The drawback is that it's a LOT of work, it took like 1 or 2 minutes each to reencode the JPEG images that I used for the NodeCore screenshots.

Warr1024 commented 3 years ago

Another suggestion: when I say "original image" in the above, I'm talking about original pixels, but not necessarily the original bytes. If the image can be run through a lossless re-encoder and end up as a smaller file size with the same pixels, we should be able to substitute that one, and not need to store the original.

For PNG, we could try some lossless recompressors, like pngcrush, optipng, advpng, or ect. Some of the deflate optimizers can run very slowly (e.g. brute-forcing) but we could at least try moderate settings, like ensuring at least zlib level 9 was used. We also don't need to keep metadata; I think CDB should guarantee ONLY the preservation of pixels here.

For JPEG, jpegtran can help, and can losslessly trim about 10% off with simple huffman optimization, and trying both progressive and baseline encodings (though progressive usually wins on large images like screenshots).