IDAT Headers Need More Documenation

[Triangle4]

IDAT headers do not appear to require chunk length/type in smaller files.

In Adobe Photoshop or Microsoft Paint:

• Create a 2x2 pixel image.
• Add 4 pixels, 1 red , 1 black, 1 green and 1 blue
• Save.
• Now, if you decode the data, you will see that IDAT is either missing or has the wrong chunk length & type In my case, and should be the case as with most, the Length is an obscenely large number in the billions. and the type does not adhere to the ascii alpha character standard.

I highly doubt that Microsoft or Adobe has failed to adhere to the standards, given their need for maximizing compatibility. Additionally the images work everywhere.

Can we please improve the documentation to clarify this? Is it always necessary to supply length and type? And if not, what circumstances are exceptions?

Kindly,

Andrew

[randy408]

Do you have any example files?

[Triangle4]

Giving an example file is moot, as the data may be converted during upload. However, here it is: <----- 2x2px

In case the image data is change by github, here is the actual contents in ascii: .PNG........IHDR.............ýÔ.s ....sRGB.®Î.é....gAMA..±..üa..... pHYs...Ã...Ã.Ço¨d....IDAT.Wc..ÿ.. Hþg...þ.þÜà.q....IEND®B`.

In Decimal: 137 80 78 71 13 10 26 10 0 0 0 13 73 72 68 82 0 0 0 2 0 0 0 2 8 2 0 0 0 253 212 154 115 0 0 0 1 115 82 71 66 0 174 206 0 17 73 68 65 84 24 87 99 0 130 255 32 2 72 254 103 0 0 15 254 2 254 220 224 143 113 0 0 0 0 73 69 78 68 174 66 96 130

And right now I dont have a hex copy.

What I have noticed, is that the IDAT header comes after the length, which is not clear in the documentation.

[DavidBuchanan314]

Here's a hexdump of your PNG

00000000  89 50 4e 47 0d 0a 1a 0a  00 00 00 0d 49 48 44 52  |.PNG........IHDR|
00000010  00 00 00 02 00 00 00 02  08 02 00 00 00 fd d4 9a  |................|
00000020  73 00 00 00 01 73 52 47  42 00 ae ce 1c e9 00 00  |s....sRGB.......|
00000030  00 04 67 41 4d 41 00 00  b1 8f 0b fc 61 05 00 00  |..gAMA......a...|
00000040  00 09 70 48 59 73 00 00  0e c3 00 00 0e c3 01 c7  |..pHYs..........|
00000050  6f a8 64 00 00 00 11 49  44 41 54 18 57 63 00 82  |o.d....IDAT.Wc..|
00000060  ff 20 02 48 fe 67 00 00  0f fe 02 fe dc e0 8f 71  |. .H.g.........q|
00000070  00 00 00 00 49 45 4e 44  ae 42 60 82              |....IEND.B`.|
0000007c

It appears to be correct, conforming to the spec as-written, and as commonly implemented.

You may be interested in the Chunk Layout section of the spec: https://www.w3.org/TR/png-3/#5Chunk-layout

[DavidBuchanan314]

Specifications do tend to be like that, heh. You might be interested in this article I wrote that offers a more informal introduction to the format https://www.da.vidbuchanan.co.uk/blog/hello-png.html

[DavidBuchanan314]

In that case I also have just the thing you might want - a dead simple/minimal deflate decompressor written in python https://gist.github.com/DavidBuchanan314/423fa0ce44253a02184a6d654325779f

It's not especially well commented, but it does have verbose logging.

Edit: Several replies from OP have been deleted for some reason. I wasn't talking to myself!

[Triangle4]

Hey David,

Thanks for your help on this. Sharing this with anyone who wants it. I will make it more complete later on. It works fully, thanks to you. https://github.com/Triangle4/PNGLIB/blob/main/Decompressor.cs

Additional: It breaks down with larger images. I have isolated it to:

int[] codelengthsAlphabetLengths = new int[19];
int[] foo = { 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15 };

for (int i = 0; i < hclen; i++)
{
    int x = foo[i];
    codelengthsAlphabetLengths[x] = ReadDataElement(3);
    Debug.Log(codelengthsAlphabetLengths[x]);
}

Cant quite figure out why yet, still debugging. Debug.Log(codelengthsAlphabetLengths[x]) equates to 0 in several locations resulting in the tree at some indices being null

Update: Now we are getting back data, but its not correct. The code has been updated and the image used attached.

[svgeesus]

chris@SuperNomad:/mnt/c/Users/chris/Documents/PNG$ pngcheck -vv Triangle4-supposed-no-IDAT.png
File: Triangle4-supposed-no-IDAT.png (124 bytes)
  chunk IHDR at offset 0x0000c, length 13
    2 x 2 image, 24-bit RGB, non-interlaced
  chunk sRGB at offset 0x00025, length 1
    rendering intent = perceptual
  chunk gAMA at offset 0x00032, length 4: 0.45455
  chunk pHYs at offset 0x00042, length 9: 3779x3779 pixels/meter (96 dpi)
  chunk IDAT at offset 0x00057, length 17
    zlib: deflated, 512-byte window, fast compression
  chunk IEND at offset 0x00074, length 0
No errors detected in Triangle4-supposed-no-IDAT.png (6 chunks, -933.3% compression).

This is a valid PNG and clearly has an IDAT chunk.

[Triangle4]

chris@SuperNomad:/mnt/c/Users/chris/Documents/PNG$ pngcheck -vv Triangle4-supposed-no-IDAT.png
File: Triangle4-supposed-no-IDAT.png (124 bytes)
  chunk IHDR at offset 0x0000c, length 13
    2 x 2 image, 24-bit RGB, non-interlaced
  chunk sRGB at offset 0x00025, length 1
    rendering intent = perceptual
  chunk gAMA at offset 0x00032, length 4: 0.45455
  chunk pHYs at offset 0x00042, length 9: 3779x3779 pixels/meter (96 dpi)
  chunk IDAT at offset 0x00057, length 17
    zlib: deflated, 512-byte window, fast compression
  chunk IEND at offset 0x00074, length 0
No errors detected in Triangle4-supposed-no-IDAT.png (6 chunks, -933.3% compression).

This is a valid PNG and clearly has an IDAT chunk.

I dont think you understood the question. We knew this was a valid PNG and IDat chunk. Our issue was with the lack of documentation for these circumstances which we would like to see rectified.