Make an open layered image format.

[HeroesGrave]

.PDN? Nope. Closed, and no specification.

.PSD? Nope. Closed. Has some specification, but the format itself is bloated due to all of photoshops functions and difficult to parse.

.XCF? Nope. GIMP-specific. Even the developers recommended against using it as a proper image format.

What the world needs is a proper open layered image format.

So, why don't we do it?

This will be a long term goal, as we don't want to just throw something together (which is the purpose of #42: to get something simple working).

If anyone has any ideas/wants to do a prototype, please leave some comments here.

It should also be cross-language if possible. ie: Not relying too heavily on Java-specific libraries.

[Longor1996]

This should be very easy to do actually!

What it has to have:

• PNG-Format like Compression for the single images. (hardest part)
• A binary Tree for the layers, that can save basic binary data (integer and string) in any matter and order.
• Backward compatibility.
• Has to be readable by ANY programming language that can 'understand' images.

Hard things to do:

• Cross-Platform Endianess AND Datatypes.
• VERY good Image Compression.
• Backward compatibility.

Easy things:

• Binary Data Tree (I already did that stuff, look at my 'BinaryDataTree' repository! Its easy to do!)
• Overall format-specification of the files.

It isn't that hard overall, and the hardest part of all is the compression.

A very basic Format specification I quickly thougt up:

(sint32 = Signed Integer 32 Bit)
(uint16 = Unsigned Integer 16 Bit)
(sint8 = Signed Integer 8 Bit/ A Byte)
(str = UTF-16 String (encoding?))
([<number>] denotes an array of the type standing on the left)

<file structure>
{
  // header
  sint32 magicNumber = The 32-Bit MD5 Hash of the formats name.
  sint32 imageWidth = The width of the image.
  sint32 imageHeight = The height of the image.
  uint16 imageCount = The amount of image-layers in this file.

  $image[imageCount] images = A list of ALL the images. Contains at least ONE image.
  $layerNode root = The root layer-node.
}

type -> $image
{
  // Need compressed PNG-like and easy to load in image-format!
  // How about multiple formats?
  // like:
  // 8Bit-Greyscale, 8Bit-ColorPallet, R5G5B5A1, R8G8B8A8, Binary-Indexed, ZBIN(INT32-RGBA)

  sint32 frameLength = The length of the information of this image.
  uint16 frameID = The ID of this frame. Ideally this would be the index in the frame-array.
  str frameEncoding = The image-encoding of this frame.
  <frameEncoding>[frameLength] frameData = The image-information itself as binary data.
}

type -> $layerNode
{
  str blendMode = The blending mode of this layer-node.
  str layerName = The name of this layer-node.
  uint16 frameID = The ID of the image linked to this node.
  uint16 childCount = The amount of child-layers this node has.
  $layerNode[childCount] = The child-nodes.
}

Ideas/Suggestions/Corrections/Mistakes?

[HeroesGrave]

Notes 1:

For the sake of simplicity when loading, let's keep the available formats simple:

• RGBA8 (obviously)
• 8-bit Greyscale
• RGBA4 (16-bit) for a bit of compatibility with older hardware.
• Colour Palette. (not sure if anyone uses this, but lots of formats support it)
• Can add new pixel formats if neccessary. (for forwards-compatibility and customisation)

[HeroesGrave]

Notes 2:

I'm not sure if blend mode data should be directly stored in the format, nor the layer name. That comes across as more Paint.JAVA-specific data.

What we need is the option to have metadata stored in certain parts of the image. So applications that need such data can read it, but those that don't can just skip over it.

Parts that need metadata:

• The main header.
• The header for each layer.
• A footer, for extra metadata that doesn't have importance, but should be included anyway?

[HeroesGrave]

Notes 3:

Compression. We're not aiming for .png level, but we want some level of compression available.

However, we need to make sure that the compression method is readily available for most programming languages, so that will require looking up available libraries and APIs.

Another option, which is kind-of lazy, is to have .pngs embed into the file as the image data. Of course, then we have a dependency on libpng and/or some other library which loads them (freeimage?), but that doesn't seem like a bad idea if we want decent compression.

[HeroesGrave]

Notes 4:

Store data in a tree format instead of mapping the tree elements to a list.

(Then we can use recursion, which is much more fun!)

[HeroesGrave]

Do we want to do this from the start instead of #42?

If so, I'll make a second draft of the image format based on what you've done above, and what you think of my above notes, and start looking at our options for cross-language compatibility.

[Longor1996]

Lot's of text you wrote there.

On note 1: Reducing the whole thing to just a couple of formats is a good idea. Support++

On note 2: That is easy!

Just go and do something like this:

<for-each-layer-node>
{
  sint32 metalength = The length of the metadata-table in BYTES.
  uint16 metacount = Size of the following metadata-table.

  for_each ENTRY in METATABLE (size = metacount)
  {
    str name = The name of the metadata entry.
    uint8 type = The type of the following metadata.
    // Type can be:
    // uint8, sint8, uint16, sint32, str (trough identifiers 0x00-x03)
    // trough the identifier 0xFF: Metadata-Table. A table in a table. Tableception! How fun is that?
    // Note that we CAN read unsigned bytes.
    <???> payload = The data, as specified in type by the previous 'type' value..
  }
}

On note 3: If you say we don't need to much compression, then that is just fine with me. Also, do NOT embed PNG into another format. It's a huge mess to do so in Java. (It involves the use of ByteArrayInput/Out-Streams, and copying alot of data around, which is bad, always)

On note 4: In the first draft, I am already using recursion for the image nodes and stuff. The fact that the image data itself is stored as list is, because there are some optimizations one can do if the image-data is all stored in a continguos line of bytes. I forgot my example on how its useful, maybe I can remember it later!

On comment : We should just go and try to think up a GOOD tree-layer/image format, which can then be used by anyone else. The format specification should just be published online later, without any license, since it is JUST a specification for a special format.

On all notes: Note that, if you specify the binary length of all the data in the file (and do so everywhere in the file), you can create a format that is built in a way of: "I can't read this datatype, I don't have to, I can just jump over it!"

That would be very useful for some systems that don't have the support for certain datatypes, and image-formats.

On another note: falls nearly asleep while writing

Good Night for now.

goes to bed

Quick-Before-Bed-Edit: Other format ideas: [ ] RGBA_FLOAT32 (Idk why), [ ] BITMASK (USEFUL!), [ ] Multi-Indexed-Binary-Color-Pallet (Fast, and easy to use/implement. PNG-Like. 8°16 Bit), [ ] HSV (8 bit x 3), [ ] CMYK (8 bit x 4), [ ] VGA_PALLET (8 bit), [ ] HSL, (8 bit x 3) [ ] Red-Green-Blue-Luminosity (HDR Image Format, 8 Bit x 4).

I can't think of any more formats.

[sylvia43]

Looking at the way gifs are done, I suggest we have a global color table and then each layer is basically a gif frame, which means that it has its own local color table. We can make the local color table optional and optionally override or add to the glov all color table. The image is stored in pairs of bytes, one for the color and one for the amount of consecutive pixels in that color. Color can be a variable amount of bytes allowing for efficient color depth.

[HeroesGrave]

I think the way we're going is to write all the raw image data and then gzip it or something.

That said, anyone know which kind of compression algorithm is most supported across different languages?

[BurntPizza]

The DEFLATE algorithm is probably what we want, at least for lossless. It's what's used in gzip and png. http://docs.oracle.com/javase/7/docs/api/java/util/zip/Deflater.html

Also, for a custom format, I had the thought of using individual zip entries for each layer, as a zip file is actually a compressed group of files.

[sylvia43]

@HeroesGrave -_- that ruins the point... An image compression algorithm needs to compress the image and write it, not just write the image and compress the file. We need an actual thing that compresses the image.

Assuming the color depth is C bytes:

3 bytes verification having the name of the format.
2 bytes width.
2 bytes height.
1 byte number of layers.
C bytes number of entries in global color table (number of global colors in image).
Global Color Table Entries:
    1 byte red channel.
    1 byte green channel.
    1 byte blue channel.
    1 byte alpha channel.
Layers:
    // Assume the layer has width * height pixels,
    // we can just count them, so we don't need
    // an end marker.
    C bytes color table index.
    1 byte length of consecutive pixels of that color.
Image end marker.

On top of that, we could compress it.

[HeroesGrave]

Another draft:

<Magic Number>
[Header] {
    Width
    Height
    Pixel Format
}
[Misc Metadata] {
    ...
}
[Background Layer] {
    <Layer Name>
    <Image Data Identifier> 
    <Blending Matrix>
    [Misc Metadata] {
        ...
    }
    <Child Count>
    ... Recursive Layers ...
}
[Raw Image Data] {
    <Image Data Identifier> (Corresponding to the layer with the same ID)
    <Compressed Size> (The size of the following data in bytes)
    <Compressed Data>
}
... And repeat raw data for all the layers ...

[BurntPizza]

I wrote a compressor that simply converts the image from sRGB to YCbCr and DEFLATEs it, with possible downsampling of the chroma channels; it was about on par with PNG IIRC.

Also, @anubiann00b , there's not much point to trying make color tables and such (if I understand you correctly) as any compression algorithm applied afterwards will do the same.

[BurntPizza]

@HeroesGrave , that draft looks pretty good. Also, by Data Identifier, is that encoding, like ARGB etc?

[HeroesGrave]

If you have a look at the ZLBIN importer/exporter it might give an idea of a rough attempt.

This won't be all that different from ZLBIN. It will just have a friendlier name, have metadata, and allow for reading of that metadata without having to read the entire file.

By data identifier I mean a key that links the layer to its image data.

[sylvia43]

Well for compressed data we could try something like [color] [pixels] [endmarker] and so on. It seems inefficient to store every pixel by color.

[BurntPizza]

I agree, it is extremely inefficient, please look at http://en.wikipedia.org/wiki/DEFLATE and see what Huffman coding and LZ77 does. They solve the problems you are talking about.

Another method of increasing compression performance in changing the color space, e.g. my YCbCr compressor: YCbCr is decorrelated and tends to have a lower entropy than RGB, and compresses better. I have thought a fair bit about data compression before, not just for this project either.

Chances are, if a standard has been widely accepted for many years, it is better than something an individual can throw together on a whim.

[HeroesGrave]

I'm going to do some experimenting on this over at PaintDotJava/Experimental.

I'm going with the name .tlgf (Tree-layered graphics format), but it's open to change.

[HeroesGrave]

YCbCr looks interesting.

[BurntPizza]

YCbCr looks interesting.

It's good when doing lossy compression, that's it mostly.

[BurntPizza]

Here's another, based mostly off @HeroesGrave 's Feb 22 draft:

long magicNumber = 0x50444a496d616765L; // "PDJImage" in ASCII
int width;
int height;
int format;         // index in format enum
long metadataBytes; // following x bytes are general metadata
// <metadata>

// this is repeated for x layers, ordered by index, bottom-up
{
    String name;        // serializes to {int size, char... chars}
    long metadataBytes; // following x bytes are layer metadata, includes blending info, etc
    //<metadata>
    long imageBytes;    // following x bytes is compressed image data
    // image data
}

I included blending info in the layer metadata, as I think that is more appropriate, as plugins could add proprietary optional blending / shading information, but it doesn't matter too much.

I scrapped the indirection as it would be buggy to do as some image data could easily coincide with an ID, and derail parsing. With blocks of [metadata, image data] it is easy to skip around in the file if needed, i.e. for progressive loading.

What really needs to be decided is what compression we're using. I vote for DEFLATE because it's pretty good, is ubiquitous, and it's already implemented for us: DelfaterOutputStream There is of course more: http://en.wikipedia.org/wiki/Category:Lossless_compression_algorithms but be wary of patent encumbered algorithms.

[HeroesGrave]

The image data identifier was so we could put all the image data (for all the layers) together for simpler compression.

The ID links the layer information to the actual image data. That way information about the image can be easily gathered without decompression.

[BurntPizza]

I'm betting that compressing each layer separately would result in better compression, and it's not much more complicated (arguably simpler) than linking different sections of the file together.

The layer info can still easily be parsed prior to decompression with this model as well, as each section is prefixed with it's size, and can easily be skipped over to parse the next layer.

[HeroesGrave]

Hmmm... I'm not sure.

Maybe @Longor1996 could explain the reason for suggesting that all image data be compressed together?

I do agree that compressing separately is more simple.

[Longor1996]

As I said before, I forgot the reason why I suggested that in the first place. But I do know that compressing all the imagedata at once is a bad idea, so I guess I wanted to say to compress the images one-by-one, and line them up in a big field of bytes, like:

for(IMG image : images)
{
  byte[] data = compressImage(image);
  out.write(data);
}

I don't know why I had that idea. Random me! Also, some changes/ideas for the latest 'suggestion' (+comments):

// This is probably one of the most important things for any good format.
long magicNumber = 0x50444a496d616765L;

// these two -could- be replaced with shorts, but its probably better to use int's, just to be sure.
int imageWidth;
int imageHeight;

// use a 'int', 'long' is a waste here: Why would you want more than 2 GB of metadata?
int imageMetadataBytes;
byte[imageMetadataBytes] imageMetadata;

// --> this is repeated for X layers, ordered by index, bottom-up
// You mean each node in the image-tree has its own absolute index, right?
// And the tree is saved bottom up? I think saving it Top-Bottom is better... or is it?
{
    // you can get the filesize down further by using different formats for each layer.
    // also, use a byte, we will never have (or find) more than 255 formats.
    // I think I listed some formats already somewhere ...
    byte layerFormat;

    // the type of compression used for this layer, which can be:
    // [
    //   INFDEF(-> Inflate and Deflate)
    //   RLE (-> RunLengthEncoding)
    //   LINEBYLINE(-> PNG)
    //   ZLIB(-> ZLBIN)
    //   RAW(-> BIN)
    // ]
    // by giving each layer its own compression, its possible to get the filesize down even further,
    // and its not really hard to put into code. Can also do things 'PNG-Crush' does to PNG's then.
    byte layerCompression;

    // serializes to {short size, char... chars}
    // You can use a short here for the string length!
    // Nobody is ever going to put a novel in there.
    String layerName;

    // use a 'int', 'long' is a waste here: Who needs more than 2 GB of metadata?
    int layerMetadataBytes;
    byte[layerMetadataBytes] layerMetadata;

    // use a 'int', 'long' is a waste here: Tell me about a Image that is larger than 2 GB.
    // On second thought, there ARE images that are this big (+2GB), but we cant possibly
    // load them, since we got some pretty hard limits in java with byte-array size.
    int layerImageBytes;
    // compressed image data
    byte[layerImageBytes] layerImageData;
}

Also, here is a small trick from the PNG format: One can compress each 'line' of a image seperately, which can give a extremely good compression for the whole image. Read the english Wikipedia-Article for the PNG-Format, specifically the 'compression' section, its explained there. Since we can already use multiple compression formats, why not use the PNG one too?

On the note of Metadata: Its probably a good idea if a NBT-style encoding (-> Minecrafts Dataformat) is used for the metadata.

Another important thing: When the user saves a image in this format, he should be asked if the application should determine the best possible compression format for the image, or if it should simply use INF/DEF. A manual selection of the compression format might also be a good idea.

So-Much-Text

Gist with Color-Formats I can come up with: https://gist.github.com/Longor1996/87276f53387b97e15681

[HeroesGrave]

Added to 1.0 milestone.

[Longor1996]

I think I will make another writeup of the format-proposal.

Things To-Do:

• Define exactly how metadata is to be stored. Simply using fields of bytes is a very bad idea.
• Determine a final list of all possible pixel-formats (/color formats) that can be used.
• Determine a final list of all possible compression algorithms that can be used.
• Make sure that the image-format is 'streamable'. (Example: In webbrowsers one can sometimes see how images load pixelline by pixelline. This is what I mean with 'streamable')

Additional things to think about:

• Add a way to store a 'thumbnail' of the image right in the header of a image.
• Add a way to store pre-calculated mip-maps for the image-layers.

[HeroesGrave]

I think I might denominate this one for 1.0 and create a solution specific to this project rather so we don't have to worry about portability.

[SuperDisk]

I've just been lurking on the project and noticed this issue; There's already a layered image format out there: TIFF. I don't know if Java's built-in loader will support layers but it's part of the specification.

I guess it's fun to reinvent the wheel sometimes, though. I certainly have done it many times.

[HeroesGrave]

If I read the wikipedia page correctly, layering is only supported through extensions to the format, and may not be available with all readers.

A layered TIFF importer/exporter could still be an option, but if we can make a more efficient format then why not have that too?

[SuperDisk]

I think layers are part of TIFF/IT which seems to be a standard. I fear that it's basically like JPEG2k though: Nobody implements it. Getting overambitious with creating a new format just leads to this:

I hate to jack this thread, but is there any IRC channel or something of the sort where project maintainers can correspond? The ephemeral nature of these bulletin boards may be sort of limiting. I like where this project came from and is going, but direct interaction would be nice.

[Longor1996]

@SuperDisk I looked up if TIFF supports layers and found that TIFF only supports layers as an extension, meaning that nobody is required to support them (Photoshop and GIMP have support).

A possibly better solution is OpenRaster: http://en.wikipedia.org/wiki/OpenRaster This format is already supported by a few image editors (GIMP and Pinta for example), so it shouldn't hurt to implement it.

[SuperDisk]

The format is just a zip file with individual PNGs comprising the layers, and an XML file holding metadata. I'm sure it would be quite easy to implement.