programmatically improve GIF size

[arnaudambro]

Hello, I try to improve the GIF size I am creating by playing with the setDispose and/or setTransparent parameters, but it doesn't have any effect at all on the GIF I created. Let say I have this easy GIF file, where each frame is a Uint8ClampedArray of pixels of a screenshot of a DOM Node ; it has 40 frames, coming from 40 screenshots ; what would be the options I should choose to reduce the size of the output GIF ?

Another question : is there a way I can improve the GIF size by playing around with the Uint8ClampedArray of pixels I am feeding the GIFEncoder with ? I mean, if I was filtering each frames, giving the "transparent color" index to each pixel identical on previous frame, would it improve the size the GIF output ?

Thanks very much for your previous help, and for this one too I hope ! Cheers, Arnaud

[twolfson]

setDispose changes a value in the GIF output, it doesn't save data by changing it. Similarly, setTransparent marks 1 color in the palette as a transparent color -- the color is still used during encoding though

I know 1 flaw this library has is outputting the palette every frame but that's not going to save you a huge amount

A deeper answer to your question would be this library is targetted at encoding the GIF, it cares less about optimizing the size of the output so it's unlikely you'll find the answer in our options/methods =/

Here's some options though:

• Reduce the palette size (unsupported by this library) -- this would allow simplification of data in each frame
• Move off of GIF format (e.g. explore movie formats, explore gifv, explore swapping JPGs/PNGs programmatically, http://www.sublimetext.com/~jps/animated_gifs_the_hard_way.html)
• Try running result through a compressor (e.g. https://github.com/jihchi/imagemin-giflossy, https://github.com/imagemin/imagemin-gifsicle)

[arnaudambro]

Alright, thank you very much for your time and answer !

[arnaudambro]

My first try with the palette ended up by returning a 8.6Mo GIF instead of a 57Ko without the palette 🥇

Could you tell me if it is the right approche to use the palette ?

const splitArray = (array, length) => {
  try {
    return [...array]
      .filter((_, ind) => ind < array.length / length)
      .map((_, ind) => [...array.slice(ind * length, ind * length + length)])
      .map(array => JSON.stringify(array));
  } catch (e) {
    throw new Error('cannot split array', array, length);
  }
};

const pixelsToAnimatedGIF = (frames, width, height, fps) =>
  new Promise((resolve, reject) => {
    const gif = new GifEncoder(width, height);
    gif.pipe(concat(resolve));
    gif.writeHeader();
    gif.setFrameRate(fps);

    /* build the palette and the indexed frames */
    let jsonPalette = [];
    let indexedFrames = [];

    for (let frame of frames) {
      /* extract colors from the frame and push to the palette if needed */
      splitArray(frame, 4).forEach(color => {
        if (colors.indexOf(color) < 0) {
          jsonPalette.push(color);
        }
      })
      /* build the frame indexed to the palette colors */
      const indexedFrame = splitArray(frame, 4).map(color => jsonPalette.indexOf(color));
      indexedFrames.push(indexedFrame);
    }

    /* get the palette as an Uint8ClampedArray, not an array of jsoned colors */
    let arrayPalette = [];
    jsonPalette.map(json => JSON.parse(json)).forEach(arr => arrayPalette.push(...arr))
    const palette = Uint8ClampedArray.from(arrayPalette)

    /* add indexed frames to the gif */
    for (let indexedFrame of indexedFrames.filter((_, ind) => ind !== 0)) {
      gif.addFrame(indexedFrame, { palette, indexedPixels: true });
    }

    gif.finish();
    gif.on('error', reject);
  });

[twolfson]

It looks like you're creating a massive pallete with a color for every pixel in a frame. gif-encoder without the palette option sets the limit of colors and maps some to nearby other colors (this is NeuQuant in case you're curious

Also, remember that currently we write the palette on every frame so it's re-dumping this large data set over and over =/

[arnaudambro]

OK @twolfson thanks for your reply. Unfortunately, my mission is to improve the GIF export only in the front, without any back, so imagine or gifsicle can't work for me. I try to give a last shot which might I thought might improve the size as much as what is doable, by comparing each frame with its previous one, and set to transparent the identical pixels between each other. This is actually improving the size a little bit, but the output GIF is not exactly what I expected. I have a pink background which I want to be there all the time, and I set the transparent color to be a color not present in any frames (here it is green). I set the disposal value to be 3 (restore to previous), and here is my code :

import GifEncoder from 'gif-encoder';
import concat from 'concat-stream';

const joinArray = array => {
 // from ["[r, g, b, a]", "[r, g, b, a]"] to [r, g, b, a, r, g, b, a]
  let newArray = [];
  array.map(json => JSON.parse(json)).forEach(arr => newArray.push(...arr));
  return Uint8ClampedArray.from(newArray);
};

const splitArray = (array, length = 4) => {
 // from [r, g, b, a, r, g, b, a] to ["[r, g, b, a]", "[r, g, b, a]"]
  try {
    return [...array]
      .filter((_, ind) => ind < array.length / length)
      .map((_, ind) => [...array.slice(ind * length, ind * length + length)])
      .map(array => JSON.stringify(array));
  } catch (e) {
    throw new Error('cannot split array', array, length);
  }
};

const HEXToJSON = hex => {
  //from "#ABCDEF" to "[171, 205, 239, 1]"
  const letters = hex.replace('#', '');
  return JSON.stringify([
    parseInt(`${letters[0]}${letters[1]}`, 16),
    parseInt(`${letters[2]}${letters[3]}`, 16),
    parseInt(`${letters[4]}${letters[5]}`, 16),
    1])
  }

export const optimizeFrames = (frames, gif, transparentColor) => {
  try {
    let prevFrame = null;
    const jsonTransColor = HEXToJSON(transparentColor);

    for (let frame of frames) {
      let newFrame = [];
      let currentFrame = splitArray(frame);
      if (prevFrame !== null) {
        prevFrame.forEach((color, i) => {
          if (color === currentFrame[i]) {
            newFrame.push(jsonTransColor);
          } else {
            newFrame.push(currentFrame[i]);
          }
        })
      } else {
        newFrame = currentFrame;
      }
      const frameToExport = joinArray(newFrame);
      gif.setDispose(3); // restore to previous
      gif.addFrame(frameToExport);
      prevFrame = currentFrame;
    }
  } catch (e) {
    console.error(e);
  }
}

const pixelsToAnimatedGIF = (frames, width, height, fps = 8, transparentColor = "#000000") =>
  new Promise((resolve, reject) => {
    const gif = new GifEncoder(width, height);
    gif.pipe(concat(resolve));
    gif.writeHeader();
    gif.setFrameRate(fps);
    gif.setTransparent(parseInt(transparentColor.replace('#', '0x'), 16));

    optimizeFrames(frames, gif, transparentColor);

    gif.finish();
    gif.on('error', reject);
  });

I guess that somehow, I need to give a "fallback background", so that the transparent color shows the purple background, isn't it ? If so, I don't have a clue of how to do it, or if it is feasible. Do you ?

[twolfson]

Sorry, don't have time to answer this today or tomorrow. I'll commit to answering by the end of the week

In the meantime, maybe you could provide me more information on the bigger picture of what you're trying to accomplish (e.g. are you generating an image for download? are you drawing something on screen?)

[arnaudambro]

What I am trying to do is creating an image for download, so I do it from the DOM Node to the animated GIF File, only from the front-end if I can. So far, it's a success, except for the size. Think you so much for your time, I am learning a lot thanks to your help !

[twolfson]

Still don't have time to address this yet. Still will give a full response by end of week. I'm going to guess that the content from the DOM is dynamically generated somehow? Is it on canvas or something?

[arnaudambro]

No, it is actually DOM Nodes encapsulated in one single root DOM Node, and animated with CSS animations. Then I take a snapshot of the root with dom-to-image's toPixelData() function, at regular intervals (depending on the FPS), which I can do by giving a negative value to the animation-delay property and pausing it with animation-play-state: paused. After that I pass all those frames to your GIFEncoder.

[twolfson]

Alright, finally took a look at your code. I've never used the dispose method. Based on my reading in Wikipedia though, it seems like it redraws over the entire frame including with transparent colors

The nuance is that you indicate where the upper-left corner + width + height of the next frame are so it will only redraw that small area

https://en.wikipedia.org/wiki/GIF#Animated_GIF

https://www.w3.org/Graphics/GIF/spec-gif89a.txt

I don't have time to dig into this further since it seems like this is micro-optimizing. The GIF format isn't super efficient and we have newer video formats for a reason =/

My recommendation is to use the library as a vanilla encoder with its built-in palette generator and let users optimize further if they want

A second recommendation is to try downloading the normally generated image and passing it through an optimizer then seeing how much space is saved

[arnaudambro]

Hi @twolfson ,

So I made a lot of tests with the code posted just before, and I finally succeed to improve the GIF size a lot. If I take the GIF underneath, I took it from 2.7Mo to 854Ko, almost 70% size optimization, isn't it nice ?

So what I did specifically for this GIF is that I put a white background div behind the scene, set the transparent color to a green screen and setDispose value to 1 (graphic is to be left in place). The result is almost perfect, except for the black writing at the bottom right, which may be solved if I choose another color that the green screen.

Here is the code below.

import GifEncoder from 'gif-encoder';
import concat from 'concat-stream';

const joinArray = array => {
 // from ["[r, g, b, a]", "[r, g, b, a]"] to [r, g, b, a, r, g, b, a]
  let newArray = [];
  array.map(json => JSON.parse(json)).forEach(arr => newArray.push(...arr));
  return Uint8ClampedArray.from(newArray);
};

const splitArray = (array, length = 4) => {
 // from [r, g, b, a, r, g, b, a] to ["[r, g, b, a]", "[r, g, b, a]"]
  try {
    return [...array]
      .filter((_, ind) => ind < array.length / length)
      .map((_, ind) => [...array.slice(ind * length, ind * length + length)])
      .map(array => JSON.stringify(array));
  } catch (e) {
    throw new Error('cannot split array', array, length);
  }
};

const HEXToJSON = hex => {
  //from "#00f868" to "[0, 248, 104, 1]"
  const letters = hex.replace('#', '');
  return JSON.stringify([
    parseInt(`${letters[0]}${letters[1]}`, 16),
    parseInt(`${letters[2]}${letters[3]}`, 16),
    parseInt(`${letters[4]}${letters[5]}`, 16),
    1])
  }

export const optimizeFrames = (frames, gif, transparentColor) => {
  try {
    let prevFrame = null;
    const jsonTransColor = HEXToJSON(transparentColor); //green screen #00f868

    for (let frame of frames) {
      let newFrame = [];
      let currentFrame = splitArray(frame);
      if (prevFrame !== null) {
        prevFrame.forEach((color, i) => {
          if (color === currentFrame[i]) {
            newFrame.push(jsonTransColor);
          } else {
            newFrame.push(currentFrame[i]);
          }
        })
      } else {
        newFrame = currentFrame;
      }
      const frameToExport = joinArray(newFrame);
      gif.setDispose(1); // The graphic is to be left in place.
      gif.addFrame(frameToExport);
      prevFrame = currentFrame;
    }
  } catch (e) {
    console.error(e);
  }
}

const pixelsToAnimatedGIF = (frames, width, height, fps = 8, transparentColor = "#000000") =>
  new Promise((resolve, reject) => {
    const gif = new GifEncoder(width, height);
    gif.pipe(concat(resolve));
    gif.writeHeader();
    gif.setFrameRate(fps);
    gif.setTransparent(parseInt(transparentColor.replace('#', '0x'), 16));

    optimizeFrames(frames, gif, transparentColor);

    gif.finish();
    gif.on('error', reject);
  });

That's when I thank you so much for your giant help all along, and share with you my happiness of succeeding this challenge ! Cheers Arnaud

[arnaudambro]

I have one last question though : does setDispose and setDelay or setFrameRate can be set for each frame ? For setDelay for example, can we have a delay of 500ms between two frames, then 3000ms between two other frames of the same GIF ?

[twolfson]

Glad to hear everything is working as desired

setDispose and setDelay/setFrameRate can be set on each frame. We can verify this by looking at the specs:

https://en.wikipedia.org/wiki/GIF#Animated_GIF

320:   21 F9                  Graphic Control Extension for frame #1
322:   04           4          - four bytes of data follow
323:   08                      - bit-fields 3x:3:1:1, 000|010|0|0 -> Restore to bg color
324:   09 00                   - 0.09 sec delay before painting next frame

https://www.w3.org/Graphics/GIF/spec-gif89a.txt

            ii) Graphic Control Label - Identifies the current block as a
            Graphic Control Extension. This field contains the fixed value
            0xF9.

            iii) Block Size - Number of bytes in the block, after the Block
            Size field and up to but not including the Block Terminator.  This
            field contains the fixed value 4.

            iv) Disposal Method - Indicates the way in which the graphic is to
            be treated after being displayed.

            Values :    0 -   No disposal specified. The decoder is
                              not required to take any action.
                        1 -   Do not dispose. The graphic is to be left
                              in place.
                        2 -   Restore to background color. The area used by the
                              graphic must be restored to the background color.
                        3 -   Restore to previous. The decoder is required to
                              restore the area overwritten by the graphic with
                              what was there prior to rendering the graphic.
                     4-7 -    To be defined.

Also, for reference, setDelay and setFrameRate set the same variable:

https://github.com/twolfson/gif-encoder/blob/0.7.2/lib/GIFEncoder.js#L129-L138

[arnaudambro]

Thanks !