Conversion of Format8bppIndexed image to gray fails (rather subtly)

[davidei1955]

I am having an issue when converting 8bppIndexed images to grayscale. The attached image is a png image with Pixelformat.Format8bppIndexed https://user-images.githubusercontent.com/79710713/177016592-a2410453-908f-4914-9c25-c526d80c836d.png and exhibits the problem. If I use:

        imageCI.Image = original.ConvertPixelFormat(PixelFormat.Format8bppIndexed, 
            PredefinedColorsQuantizer.FromCustomFunction(c => c.ToGray()));

to convert it to a gray image, some pixels are not gray. The problem also occurs with 8bppIndexed gif and bmp images that start out with a color (not all-gray) Palette. FWIW, if I use:

        imageCI.Image = original.ConvertPixelFormat(PixelFormat.Format24bppRgb, 
            PredefinedColorsQuantizer.FromCustomFunction(c => c.ToGray()));

it works fine; a completely gray image is returned. The code I'm using to test that an image is gray is:

public static bool IsGray(this Bitmap image) {
    Bitmap img = null;
    try {
        if (((int) image.PixelFormat) == 8207) {  //8207 == PixelFormat.Cmyk32
            img = image;
            image = image.To24bppRgb();
        }
        BitmapData imageData = null;
        try {
            Rectangle rect = new Rectangle(0, 0, image.Width, image.Height);
            imageData = image.LockBits(rect, ImageLockMode.ReadOnly, image.PixelFormat);
            int imageBytesperPixel = Image.GetPixelFormatSize(imageData.PixelFormat) / 8;
            int imageRowPad = Math.Abs(imageData.Stride) - (imageData.Width * imageBytesperPixel);
            int imageSize = imageData.Height * imageData.Stride;
            int imageEnd = (int) IntPtr.Add(imageData.Scan0, imageSize);
            bool is8bppIndexed = image.PixelFormat == PixelFormat.Format8bppIndexed;
            Assert.True((imageBytesperPixel == 1) || (imageBytesperPixel >= 3), 
                    $"Unsupported PixelFormat = {imageData.PixelFormat}");
            unsafe {
                byte* imagePtr = (byte*) imageData.Scan0;
                for (int i = 0; i < imageData.Height; i++) {
                    for (int j = 0; j < imageData.Width; j++) {
                        if (is8bppIndexed) {
                            if ((int) imagePtr >= imageEnd) {
                                continue;
                            }
                            byte index = imagePtr[0];
                            var color = image.Palette.Entries[index];
                            if ((color.R != color.G) || (color.R != color.B)) {
                                return false;
                            }
                        } else {
                            if ((((int) imagePtr) + 2) >= imageEnd) {
                                continue;
                            }
                            byte blue = imagePtr[0];
                            byte green = imagePtr[1];
                            byte red = imagePtr[2];
                            if (red != blue || (blue != green)) {
                                return false;
                            }
                        }
                        imagePtr += imageBytesperPixel;
                    }
                    imagePtr += imageRowPad;
                }
            }
        } finally {
            image?.UnlockBits(imageData);
        }
        return true;
    } finally {
        if (img != null) {
            img.Dispose();
        }
    }
}

I'm using KGySoft.Drawing version 6.3.2

[koszeggy]

There is a good reason for this behavior, and fortunately the fix is also really simple.

First, the reason:

• The FromCustomFunction returns quantizer without a palette.
• If your target pixel format is an indexed one but the quantizer has no palette, then ConvertPixelFormat tries to preserve the original palette whenever possible; otherwise, will use the system default palette.
• In your case the custom quantizer function returns colors that cannot be represented in the original palette pefectly so the result will be "quantized further" to use the closest possible colors.

So in your case the original palette is preserved. If you use my debugger visualizers you can see that the result still has a colorful palette, and the conversion really tried its best while attempting to match the "grayest" colors of the palette with more or less success:

The solution: When converting to pixel format Format8bppIndexed it is highly recommended to use a quantizer that has a palette; otherwise, the colors might be off as you could see. And PredefinedColorsQuantizer happens to have a Grayscale quantizer that uses a palette. So the fix is really simple:

imageCI.Image = original.ConvertPixelFormat(PixelFormat.Format8bppIndexed, 
            PredefinedColorsQuantizer.Grayscale());

[koszeggy]

Btw, your IsGray can be really, really simplified like this, and it supports every pixel format - (ok, except CMYK but it's coming soon):

public static bool IsGray(this Bitmap image)
{
    using IReadableBitmapData bitmapData = image.GetReadableBitmapData();
    var row = bitmapData.FirstRow;
    do
    {
        for (int x = 0; x < row.Width; x++)
        {
            Color32 color = row[x];
            if (color != color.ToGray())
                return false;
        }
    } while (row.MoveNextRow());
    return true;
}

[davidei1955]

Thanks for the quick answer. FWIW, I got the gray quantizer function from here: https://docs.kgysoft.net/drawing/html/M_KGySoft_Drawing_Imaging_PredefinedColorsQuantizer_FromCustomFunction_1.htm

Do you recommend that I use

    imageCI.Image = original.ConvertPixelFormat(PixelFormat.Format8bppIndexed, PredefinedColorsQuantizer.Grayscale());

for all source image PixelFormats being converted to gray, or just 8bppIndexed? How about when the target PixelFormat is not indexed? PredefinedColorsQuantizer.Grayscale() seems simpler and more intuitive.

[koszeggy]

I apologize if my code examples are sometimes confusing. ToGray in that example was just a demonstration for a custom function. But please note that ToGray can preserve transparency (as demonstrated in this example), meaning, it can generally return 256² = 65536 different values, which obviously does not fit for an 8bpp pixel format.

Do you recommend that I use [...] for all source image PixelFormats being converted to gray, or just 8bppIndexed? How about when the target PixelFormat is not indexed?

The source pixel format can be anything. As for the target, every quantizer has a PixelFormatHint property that returns the smallest compatible pixel format. Graycale returns Format8bppIndexed, meaning, you should use it for at least 8bpp images but works well for any other pixel formats that are larger than that (well, except for color 16bpp formats, because they cannot display 256 different gray shades but you get the idea).