Support for palette access when reading indexed image formats

[Paul-JanPauptit]

Prerequisites

• [X] I have written a descriptive issue title
• [X] I have verified that I am running the latest version of ImageSharp
• [X] I have verified if the problem exist in both DEBUG and RELEASE mode
• [X] I have searched open and closed issues to ensure it has not already been reported

ImageSharp version

3.0.0

Other ImageSharp packages and versions

-

Environment (Operating system, version and so on)

Windows, any

.NET Framework version

6.

Description

I am processing 4-bpp indexed images (16 colors) for retro & fantasy platform development (TIC80 in particular). For that we often need access to the original palette of the image.

I understand this is not a primary use case for ImageSharp, but it would be great if ImageSharp offered just that... and like @JimBobSquarePants mentioned on twitter, "why not submit an issue". 😄

Steps to Reproduce

Load an indexed image. Try to access the palette.

Images

No response

[JimBobSquarePants]

@Paul-JanPauptit What format are your images in?

Completed

• [x] gif
• [x] png

For future

• [x] bmp
• [ ] tiff
• [x] ico
• [x] cur

[Paul-JanPauptit]

@JimBobSquarePants I am currently in a bit of crunch so my demoscene toolbuilding is on hold for a couple of weeks, but I wanted to say that I really appreciate that you have been adding this functionality. <3

I don't have a set workflow yet, I think centering it around PNG should work perfectly. Basically being able to read out the palette and pixels should solve all my problems.

[JimBobSquarePants]

Happy to help!

I'll eventually port bitmap and maybe tiff across to do it also but will leave that for future releases as it gets complicated.

[Beyley]

Would this also extend to storing the palette used by a quantize operation on an Image for later access?

Im trying to write a custom ImageSharp encoder for the LittleBigPlanet PSP custom texture format "MIP", which uses an 8bit indexed pixel format. currently to my knowledge, after i run a quantize with a "max colors" of 256, i need to loop over every pixel to fill out a palette myself, it would be nice if after doing this i could somehow retrieve the palette used by the quantization, so that i can just directly write that to the file rather than needing to iterate over every pixel and fill out a palette myself (which is quite slow).

heres how i currently get the palette after doing a quantize

Dictionary<Rgba32, byte> outputPalette = new(256);
{
    for (int x = 0; x < image.Width; x++)
    {
        for (int y = 0; y < image.Height; y++)
        {
            Rgba32 col = image[x, y];
            //If the color is not in the palette
            if (!outputPalette.TryGetValue(col, out _))
            {
                //Add the color to the palette
                outputPalette[col] = (byte)outputPalette.Count;
            }
        }
    }
}

[benbaker76]

Completed

• [x] gif
• [x] png

@JimBobSquarePants Where can I find this branch?

[JimBobSquarePants]

Support for those types has already been released.

[benbaker76]

Support for those types has already been released.

Great! Is there an example on reading the raw palette and indices of a 4-bit / 8-bit indexed png?

[JimBobSquarePants]

Sure..

You need to query the metata for each type.

using Image<TPixel> image = ... ;
PngMetadata metadata = image.Metadata.GetPngMetadata(); 

// You want to check for null here. 
ReadOnlySpan<Color> colorTable = metadata.ColorTable.Value.Span;

[benbaker76]

Sure..

You need to query the metata for each type.

using Image<TPixel> image = ... ;
PngMetadata metadata = image.Metadata.GetPngMetadata(); 

// You want to check for null here. 
ReadOnlySpan<Color> colorTable = metadata.ColorTable.Value.Span;

That's great but how do I read the indices? What pixel format do I need to read it?

Most of the image libraries I've looked at for .NET convert indexed pngs' to RGB. It would be fantastic if your library can read the raw indices.

Also is it possible to write indexed pngs' in both 4-bit and 8-bit indexed formats with a custom palettes?

[JimBobSquarePants]

That's great but how do I read the indices? What pixel format do I need to read it?

You don't read indices. An image buffer is a 2D representation of the decoded image pixels where the pixel format represents the requested layout in memory of each pixel. Indices are already read during decoding so there is none of that will-it-throw-an-exception nonsense you get in System.Drawing when trying to do graphics options that should be agnostic.

ImageSharp will automatically decode the indexed image data into whatever pixel format you explicitly choose to use or the most memory efficient and appropriate pixel format.

Also is it possible to write indexed pngs' in both 4-bit and 8-bit indexed formats with a custom palettes?

Can you please keep questions focused and in the discussion channels. You should also read and understand the API docs before asking questions. (Also, yes you can)

[benbaker76]

You don't read indices. An image buffer is a 2D representation of the decoded image pixels where the pixel format represents the requested layout in memory of each pixel. Indices are already read during decoding so there is none of that will-it-throw-an-exception nonsense you get in System.Drawing when trying to do graphics options that should be agnostic.

Okay it looks like I need to make a feature request. Thanks for your time.

[benbaker76]

I've moved to using the pngcs library.

[JimBobSquarePants]

Would this also extend to storing the palette used by a quantize operation on an Image for later access?

Im trying to write a custom ImageSharp encoder for the LittleBigPlanet PSP custom texture format "MIP", which uses an 8bit indexed pixel format. currently to my knowledge, after i run a quantize with a "max colors" of 256, i need to loop over every pixel to fill out a palette myself, it would be nice if after doing this i could somehow retrieve the palette used by the quantization, so that i can just directly write that to the file rather than needing to iterate over every pixel and fill out a palette myself (which is quite slow).

heres how i currently get the palette after doing a quantize

Dictionary<Rgba32, byte> outputPalette = new(256);
{
    for (int x = 0; x < image.Width; x++)
    {
        for (int y = 0; y < image.Height; y++)
        {
            Rgba32 col = image[x, y];
            //If the color is not in the palette
            if (!outputPalette.TryGetValue(col, out _))
            {
                //Add the color to the palette
                outputPalette[col] = (byte)outputPalette.Count;
            }
        }
    }
}

Sorry, I missed this. I think you should have a look at how other quantizing image encoders do this (GifEncoderCore for example). Individual frames are quantized during encoding which yields an IndexedImageFrame contaiing the palette. This is then directly encoded.