search

phoboslab / qoi

The “Quite OK Image Format” for fast, lossless image compression
MIT License
6.92k stars 330 forks source link

Support gray8 #170

Closed rxrbln closed 2 years ago

rxrbln commented 2 years ago

Congratulations to the surprisingly simple and quite ok image format!

IMHO it would be nice to officially support 8-bit gray. It could be simply indicated by channel = 1 or 2 for w/ or w/o alpha and only using the 1st channel for the gray values.

jstavats commented 2 years ago

I also agree. The encoding will need to be different as a lot of the image space savings here are coming from not needing 3 bytes per pixel, and of course grey8 is only one byte per pixel in the first place so that wouldn't help. I've got some ideas on how to do some similar things to 8 bit grey images and may take a whack at that. If I do and come up with something half decent would it be considered as an addition to the format? (Either way it would still be useful to me)

rxrbln commented 2 years ago

I was more thinking of just using the quite ok encoding without any further modifications and keep stuff as simple as possible.

oscardssmith commented 2 years ago

The problem with this is that QOI will generally not provide compression for single channel images. The 1 channel variant would probably need tags that let you compress 2 pixels with small diffs into 1 byte.

rxrbln commented 2 years ago

It would still run length compress though. One could use delta encoding to compress 3 similar pixels at a time.

jstavats commented 2 years ago

So, I implemented a 1-byte per pixel greyscale version encoder/decoder, and I'm quite happy with both the speed and compression. It is "somewhat" faster than libpng using the intel performance primitives version of the zlib compression library with its fastest settings (which can be considerably faster than the stock zlib as it allows a special non-optimal compression type for speed). When in that mode this encoder also results in a smaller file than png. If I change the zlib compression level it can compress the data better, but then it takes considerably longer to save the image. I'll post what I'm using for the "chunks" when I clean it up a bit, they are "heavily inspired" by the colour chunks.

jstavats commented 2 years ago

I've changed the format on the 1 byte per pixel greyscale version i've been playing with. I did a benchmark of it here, which I'm quite happy with:

https://github.com/jstavats/qoi/blob/master/qoi-grey8-benchmark.txt

There are problems with decoded image not quite matching encoded periodically, and the comments in the file are old. But I'm getting happier with the structure and performance of it.

bitbank2 commented 2 years ago

I also implemented a 1-channel modification to QOI which is still simple and similar to QOI's 3/4-channel, but compresses much better than treating 1-channel as 4. It's also quite fast. Speaking of speed, I sped up (On my MacBook M1) the decoding by 1.8x and the encoding by 2.3x while still supporting the original "intent" of the C code. You can see it in my fork here: https://github.com/bitbank2/qoi

jstavats commented 2 years ago

Interesting 1-channel version, what is the purpose of the QOI_OP_BADRUN8? I think its the same as my QOI_OP1_RAW which is a raw series of bytes (up to 62-ish). I've also been doing further work and tried a AVX2 version of my greyscale version (adding to the colour version found here https://github.com/MKCG/qoi/tree/simd_avx2_implementation , thanks @MKCG ). Quite fun. I removed the index as that opcode was 1 byte and thought that the byte could just be stored in the same space. How often does the QOI_OP_INDEX8 get emitted? 8 indices doesn't sound like enough options for greyscale, though it might be for screenshots/pixel art, but I'd expect those in a palettized format as they'd largely be colour. One can think of the colour QOI index colours as a sort of dynamic palette already.

I found that I'm not really getting the expected speedups over my libpng with intel performance primitives' accelerated zlib version that I was hoping for. It seems for greyscale to be only getting about ~20% faster encode than that. I also found that the libpng encode settings in the qoibench.c are the default which are not speed optimal. When I add the following lines before the png_set_IHDR call: png_set_filter(png, PNG_FILTER_TYPE_BASE, PNG_FILTER_SUB); png_set_compression_level(png, 1); png_set_compression_strategy(png, 3);

I get a more time efficient encoding with result of (non-accelerated zlib and default 3/4 byte colour handling):

# Grand total for ../qoi_benchmark_suite/
        decode ms   encode ms   decode mpps   encode mpps   size kb    rate
libpng:       8.1        14.2         57.08         32.67       474   28.9%
stbi:         7.5        68.4         62.06          6.78       561   34.2%
qoi:          2.2         2.9        207.29        157.68       463   28.2%

same zlib and AVX2 default colour handling

# Grand total for ../qoi_benchmark_suite/
        decode ms   encode ms   decode mpps   encode mpps   size kb    rate
libpng:       7.8        13.6         59.53         34.07       474   28.9%
stbi:         7.2        59.8         64.14          7.76       561   34.2%
qoi:          2.1         1.6        223.18        292.51       463   28.2%
bitbank2 commented 2 years ago

Are you running grayscale images through the original QOI benchmark code? If so, the "rate" value is quite distorted. For example, a 1000x1000 grayscale image will be promoted to 4-bytes per pixel before compression and the results will be calculated as if it's compressing a 4MB file instead of 1MB file.

To answer your question, yes, my BADRUN8 op is for runs of "uncompressible" pixels. For gradient areas, the DIFF8 op allows 2:1 compression and for areas with few unique 'colors' used, the INDEX8 op allows 2:1 compression. The combination of runs of repeating pixels along with those 2 2:1 ops allows much better compression of grayscale images than the original QOI way of doing it.

jstavats commented 2 years ago

When testing grey, I'm forcing the png to be loaded to be grey, using the commend line switch --forcebpp 1 I added to the version of qoibench.c here: https://github.com/phoboslab/qoi/pull/173

My images are the qoi_benchmark_suite ones but all are loaded in as greyscale.

I can see the index thing helping with non-photo images with abrupt intensity changes. My use case is machine vision so "real" images. The question was more how often do you get 2 pixels back to back that both hit index values.

bitbank2 commented 2 years ago

My grayscale method seems to do better on some images and worse on others compared to the original QOI. I'll spend some time tweaking it and see if I can do better. It's a first pass effort after all :) My cleanup of the general C code speeds it up in every case. The removal of the RGBA union is one reason.

I haven't done a detailed statistical dump of the ops used, so I can't really say how often the pair of index values works. I think some of the bulk is from not fleshing out the BADRUN code to look for runs of bad pixels. That was unfinished business that I need to code still.

bitbank2 commented 2 years ago

Confirmed - the lack of "bad runs" > 1 was hurting my compressed size. New results for my screenshot image (now it beats PNG in size and speed):

/Users/laurencebank/Projects/qoi/images/gray_screenshot.png size: 1113x726

    decode ms   encode ms   decode mpps   encode mpps   size kb    rate

libpng: 3.3 32.9 241.69 24.59 78 10.0% stbi: 6.2 37.7 129.83 21.45 53 6.7% qoi: 5.4 2.2 148.34 371.29 71 9.0%

P.S. A few runs through the profiler show that the INDEX8 op is much more 'popular' than the DIFF8, but there's a pretty even distribution amongst all of the OPs I created.

bitbank2 commented 2 years ago

I just did a tweak to my fork of QOI to improve compression of images with lots of repeating pixels. It improves my test images by 10% in size without affecting speed nor being detrimental to the compression of other images. This makes it incompatible with the current QOI 'standard', but at this point I assume my QOI version will become a different 'standard'.

rxrbln commented 2 years ago

do you have more details about the changes? One major thing I would find missing in the current standard, is x and y resolution in dpi. One of the more important meta data commonly needed, ... :-/ do you have a patch for the latest gray work?

bitbank2 commented 2 years ago

@rxrbln I used a couple of "counts" within the RUN tag to add RUN256 and RUN1024. For images with lots of repeating pixels this saves some space. I don't have a "patch" for grayscale to work, but my fork has proper grayscale support by changing the bench tool and the QOI.h codec.

rxrbln commented 2 years ago

if you have a fork you can easily generate the git diff ;-) it would certainly help to move any qoi gray compression forward, ... or if you prefer write it down in a form like the standard spec sheet.

bitbank2 commented 2 years ago

I'm still experimenting with the changes, so I don't think it makes sense to call it 'done'. I just started working on QOI this week and would like to test it on more images and add palette color image support. If you want to try my code, just clone my fork and play with it.

bitbank2 commented 2 years ago

I've successfully added support for RGB565 (and grayscale). I deleted my fork of QOI and am starting a new format (Simple Lossless Image Codec), since my format is no longer QOI compatible. I'll share it publicly when I'm finished with it.

bitbank2 commented 2 years ago

My SLIC codec is now public. Here's how it's different from QOI:

https://github.com/bitbank2/SLIC