Mention that QOI is compressable format

[Dimava]

Unlike most of other image formats which can not be compressed further, QOI can be considerably compressed with general compression algorithms Unlike other image formats which will generate highly binary different files for partially same images, QOI generates alike files, alloving to compress variant sets with insane compression (I've got 11MB .qoi.7z out of 4x 12MB pngs)

Variant set: A CG set depicting only a single scenery from a static perspective. (they often have 50-90% of same pixels)

Please write the information about high compression rates for both single images and image sets in the readmes

[AngusJohnson]

Yes, I think this point could easily be missed.

In my testing, combining QOI with ZLib compression (QOZ) produces file sizes significantly smaller (roughly 10-25%) than when using PNG format. Of course this gives up most (but not all) of QOI's speed advantages, but QOZ still retains a significant (2-5 times) speed advantage when compressing.

[oscardssmith]

note that zlib is no longer a good choice for compression. QOI+ZSTD or QOI+LZ4 are going to give much better compression vs speed tradeoffs.

[AngusJohnson]

I chose ZLib because its libraries are so widely available and for simplicity of adoption.

[oscardssmith]

To me, that feels like a weird argument for an image format that's less than 2 months old.

[magnus-ISU]

I agree with @oscardssmith . Limiting recommendations for backwards compatibility when no existing support exists makes no sense

[apankrat]

Just a random thought - does re-compressing QOI with QOI yield anything interesting?

[oscardssmith]

no. QOI is very specifically designed for 3 or 4 channel images. The result after QOI compression isn't that.

[apankrat]

no. QOI is very specifically designed for 3 or 4 channel images. The result after QOI compression isn't that.

Yeah, sure, I know that. It's just sometimes compression with one algorithm creates certain patterns that can be further compressed with another. bzip2-compressed mysql dumps can be compressed further 10% by gzip, even though the latter is "less sophisticated". This sort of thing.

So for a new compression algorithm it's worth checking if applying it recursively may show some interesting results. It's exceedingly rare, but when it happens it's spectacular.

I did a quick test just now and QOI can't compress further its own output. 400 x 400 photo crop went from 468K to 204K to 269K. So it's a no.

[magnus-ISU]

I mean, that's completely unsurprising though. Why would variable-length data chunks of QOI be able to be compressed by opcodes which describe difference-in-RGB movements? lol

[chocolate42]

I chose ZLib because its libraries are so widely available and for simplicity of adoption.

LZ4 and ZSTD are widely available too, the only likely places that they aren't available is for micro controllers or other hardware that aren't physically capable of running the algorithms. On Debian/Ubuntu adding basic support is the same for all three: Installing liblz4-dev/libzstd-dev/zlib1g-dev from the standard repo, including lz4.h/zstd.h/zlib.h, and the half a dozen lines it takes to call the respective simple encoder/decoder.

Yeah, sure, I know that. It's just sometimes compression with one algorithm creates certain patterns that can be further compressed with another.

A smart QOI-recompressor would at least take advantage of QOI being a byte-aligned format. The tags might be ANS-coded, the RGBA elements may be grouped for further processing, etc. The problem is that QOI output being byte aligned still allows for entropy coders to get good results so a QOI-recompressor would have to be excessively smart to compete.

[pseregiet]

I chose ZLib because its libraries are so widely available and for simplicity of adoption.

LZ4 and ZSTD are extremely easy to get working in your program. Yesterday I tried using both of them for my own game files package system and it works, just few files .c and .h files from their repo, call one function to compress, one to decompress... I recommend trying it out.

[HughPH]

dotnet has had a Brotli implementation since core 2.1 (but no zstd) and qoi unsurprisingly works well with Brotli too.

With compression, qoi looks to be better than PNG, perhaps there could be a Compression byte in the header, 0 for none and other values assigned to presently-known algorithms. However, this would significantly complicate interpretation of interchanged .qoi files - i.e. files from diverse sources.

[rayrobdod]

Having gzip or brotili or whatever other compression format wrap a qoi file is good enough; there is no reason to make those external compressions a part of the qoi format.

[zzo38]

It can be seen that in many cases, QOI+DEFLATE compresses better than PNG, but sometimes it isn't a better compression. PNG alone is usually better than QOI alone, though. (The below is a list of the contents of the ZIP archive with the test files)

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2021-12-21 04:47:44 D....            0            0  qoi_test_images
2021-12-21 04:47:44 .....       349827       333282  qoi_test_images/dice.png
2021-12-21 04:47:40 .....       519653       350872  qoi_test_images/dice.qoi
2021-11-21 10:18:50 .....       593463       593449  qoi_test_images/kodim10.png
2021-12-21 04:40:01 .....       652383       522804  qoi_test_images/kodim10.qoi
2021-11-21 10:19:32 .....       557596       557596  qoi_test_images/kodim23.png
2021-12-21 04:41:07 .....       675251       544532  qoi_test_images/kodim23.qoi
2021-12-08 16:06:13 .....        16605        16526  qoi_test_images/qoi_logo.png
2021-12-21 04:44:14 .....        16488         8107  qoi_test_images/qoi_logo.qoi
2021-12-21 04:40:17 .....        14227        13619  qoi_test_images/testcard.png
2021-12-19 15:18:05 .....        21857         7402  qoi_test_images/testcard.qoi
2021-12-21 04:47:25 .....        18371        17692  qoi_test_images/testcard_rgba.png
2021-12-19 15:21:21 .....        24167         9453  qoi_test_images/testcard_rgba.qoi
2016-07-19 08:19:40 .....      1344960      1328424  qoi_test_images/wikipedia_008.png
2021-12-21 04:42:27 .....      1521134      1343930  qoi_test_images/wikipedia_008.qoi
------------------- ----- ------------ ------------  ------------------------
                               6325982      5647688  14 files, 1 folders

[tuohirv]

Yes, I think this point could easily be missed.

In my testing, combining QOI with ZLib compression (QOZ) produces file sizes significantly smaller (roughly 10-25%) than when using PNG format. Of course this gives up most (but not all) of QOI's speed advantages, but QOZ still retains a significant (2-5 times) speed advantage when compressing.

Yes I confirm. We actually made our internal format based on this: qoi + lzma. You can try it out at https://github.com/sumo-apps/voiconv .. The benefit of "qoi" before lzma is at least to decrease (significally) the size of data going to heavier compression.

[L3P3]

Oh my, this thread is full of spam, discussing irelevant off-topic stuff. Including semi-personal attacks, now including mine.

Getting back to the original topic of practical further compression: There are use cases for this, imagine (ha ha) having a game with thousands of images. You want to be able to quickly decompress random files but for downloading that game, you can keep it all relatively small. Or decompress from qoi-set.zstd into memory and decode the qoi lazily.

And then, every dev using qoi as a library can just wrap it in zstd or similar by self, no need to discuss extending the format. There might be even more microcontroller friendly decompressors that might be combined with qoi for tiny flash+memory footprints. Now, benchmarking that kind of stuff would be helpful and bringing this topic forward!

Somebody wants to benchmark a test image set with a selection of some common/uncommon compressors? (Test both single qoi and a set of qoi per compression run!) Then we could add this to the readme!

[tansy]

I did some test of 'entropy coding' for qoi (compressing with regular compressors). Used subset from benchmark images and default compression levels (except zstd, which was set to 12, to somewhat match gzip level 6. Scripts used to make this test, in case someone wanted to test it themself are also attached.

That's the result:

20737618 qoi.bmp
 7035249 qoi
 6630663 qoi.lz4
 5850003 qoi.gz-H
 5721098 png
 5476417 qoi.zst
 5471989 qoi.gz
 5113443 qoi.lzma
 5102454 qoi.bz2

Here are some more elaborate results (with times and calculated speeds):

# sorted by ratio
# method   ctime    cspeed  dtime   dspeed   compr.     ratio
qoi.bz2    2.762   7.5082   1.491  13.9085  5102454  0.246048
qoi.lbz2   2.124   9.7634   1.122  18.4827  5102454  0.246048
qoi.lzma   5.934   3.4947   1.534  13.5187  5113443  0.246578
qoi.gz     1.410  14.7075   0.543  38.1908  5471989  0.263868
qoi.zst    2.061  10.0619   0.491  42.2355  5476417  0.264081
png       10.308   2.0118   2.361   8.7834  5721098  0.275880
qoi.gz-H   0.859  24.1416   0.549  37.7734  5850003  0.282096
qoi.lz4    0.533  38.9074   0.409  50.7032  6630663  0.319741
qoi        0.437  47.4545   0.332  62.4627  7035249  0.339251

# sorted by ctime
# method   ctime   cspeed   dtime   dspeed   compr.     ratio
qoi        0.437  47.4545   0.332  62.4627  7035249  0.339251
qoi.lz4    0.533  38.9074   0.409  50.7032  6630663  0.319741
qoi.gz-H   0.859  24.1416   0.549  37.7734  5850003  0.282096
qoi.gz     1.410  14.7075   0.543  38.1908  5471989  0.263868
qoi.zst    2.061  10.0619   0.491  42.2355  5476417  0.264081
qoi.lbz2   2.124   9.7635   1.122  18.4827  5102454  0.246048
qoi.bz2    2.762   7.5082   1.491  13.9085  5102454  0.246048
qoi.lzma   5.934   3.4947   1.534  13.5187  5113443  0.246578
png       10.308   2.0118   2.361   8.7834  5721098  0.275880

# sorted by dtime
# method   ctime    cspeed  dtime   dspeed   compr.     ratio
qoi        0.437  47.4545   0.332  62.4627  7035249  0.339251
qoi.lz4    0.533  38.9074   0.409  50.7032  6630663  0.319741
qoi.zst    2.061  10.0619   0.491  42.2355  5476417  0.264081
qoi.gz     1.410  14.7075   0.543  38.1908  5471989  0.263868
qoi.gz-H   0.859  24.1416   0.549  37.7734  5850003  0.282096
qoi.lbz2   2.124   9.7635   1.122  18.4827  5102454  0.246048
qoi.bz2    2.762   7.5082   1.491  13.9085  5102454  0.246048
qoi.lzma   5.934   3.4947   1.534  13.5187  5113443  0.246578
png       10.308   2.0118   2.361   8.7834  5721098  0.275880

As can be seen lz4 is inefficient in this application, which becomes clear when one realises that lz4 does not use any entropy coding, and the entropy coding, even pure prefix code like huffman from deflate, is what makes a difference in this contest. Lzma is very tight but still loses to bzip2, both in terms of compression and speed; zstd turns out to be worse and slower than gzip (only decompression is slightly faster but it doesn't make up to it.

All in all, the best compressor (by ration) is bzip2, and by speed gzip/deflate, which beats png both, with compression (4%-5%) and with speed (~7 times).

---

qoiformat-bench-1.zip