UPNG stacks frames for transparent backgrounds.

[BannerBomb]

In my nodejs project, I take a gif image, break up each frame into png files (using gif-frames), then apply an overlay to them then recombine then into an apng. When the frames have a transparent background it seems UPNG stacks the frames which make the frames stay instead of removing them to display the next one. This is an example of the APNG result when I have cumulative enabled when extracting the frames of the gif then recombine them using UPNG. https://cdn.discordapp.com/attachments/626649734953566218/814236015316434954/1614199393560.png

And here's another if I disable cumulative when extracting the frames then recombine using UPNG. The stacking isn't as bad but it still is keeping data from previous frames. https://cdn.discordapp.com/attachments/626649734953566218/814238946128953354/1614200092424.png

The original gif I used for the above examples is https://media3.giphy.com/media/Wwe6DNWWOvfBbr9Gbc/giphy.gif.

All of the png frames that were extracted from gif-frames look fine when I output them one by one before combining them into a apng image which the only other thing it can be is UPNG.

The result is fine if the image doesn't have a transparent background around the moving parts. https://cdn.discordapp.com/attachments/626649734953566218/814229966102331427/1614197951358.png

[photopea]

Hi, are you talking about the UPNG.encode() method, which you use to generate the APNG file with the animation?

I need to reproduce it somehow. Can you send me a ZIP archive of images (a file for each frame), so that I try to call UPNG.encode() on it and see the result?

[photopea]

Also, make sure you are using the latest version of UPNG.js from here, not old versions from various NPMs.

[BannerBomb]

I am using the latest version from the repo. And yes I mean the UPNG.encode method.

Here's some example frames. frame examples.zip

In each folder of the archive I have the frame images as pngs that I extracted from their gif images using gif-frames and a frameData.txt, the frameData.txt contains information of each frame like the delay in milliseconds, the x,y width and height and some other details if needed. This is the description for what each key represents.

The only folder that doesn't end up having stacked frames is folder 2 because there are no transparent pixels around the moving image. Though it might but just may not be noticeable since all pixels are colored.

Also if needed the buffers I pass to the encode method are what I get from using ctx.getImageData(0, 0, width, height).data with canvas.

[photopea]

I was able to get the APNG like this:

Is that what you expect? I don't think your APNG above was made from the frames that you sent me, because it has a white background and some circle at the bottom right corner. It also has a different resolution.

[photopea]

Also, the table of your values is completely redundant for UPNG.encode(), there is no input for such values, just the delays can be useful.

[BannerBomb]

Yes, that is the results I expected, I only use the delays from the frameData, but I sent all the data for the frames in case you needed anything from it.

The reason the frames I sent at first were different is that I go through each frame that was extracted, convert them to base64 data add them to SVG code which is what changes the shape to a circle, and adds the grey circle to the bottom right of it then I convert the SVG data back to a png and recombine the frames.

But I did try using the unmodified frames that I sent in the zip file and I got the same issue. If you want to try here's the code I use.

const UPNG = require('./Modules/UPNG');
const gifFrames = require('./Modules/gif-frames');
const { Canvas, loadImage } = require('canvas');

const frameData = {
    buffers: [],
    width: 0,
    height: 0,
    delays: []
}

function frameToBuffer(frame) {
    const stream = frame.getImage();
    if (stream instanceof Stream) {
        const chunks = [];
        stream.on('data', (data) => chunks.push(data));
        return new Promise((resolve, reject) => stream.once('end', () => resolve(Buffer.concat(chunks))));
    } else if (stream instanceof Canvas) {
        return stream.toBuffer();
    } else {
        return stream;
    }
}

const url = 'https://media3.giphy.com/media/Wwe6DNWWOvfBbr9Gbc/giphy.gif';
const frames = await gifFrames({ frames: 'all', outputType: 'png', url });
for (const frame of frames) {
    const imageBuffer = await frameToBuffer(frame);
    const canvasImage = await loadImage(imageBuffer);
    if (!frameData.width) frameData.width = canvasImage.width;
    if (!frameData.height) frameData.height = canvasImage.height;
    const canvas = new Canvas(canvasImage.width, canvasImage.height);
    const context = canvas.getContext('2d');
    context.drawImage(canvasImage, 0, 0);

    frameData.buffers.push(context.getImageData(0, 0, canvasImage.width, canvasImage.height).data);
    frameData.delays.push((0.01 * frame.frameInfo.delay) * 1000);
}

const encodedPNG = UPNG.encode(frameData.buffers, frameData.width, frameData.height, 0, frameData.delays);
const finalBuffer = Buffer.from(encodedPNG);
const dataURI = finalBuffer.toString('base64');
console.log(`data:image/png;base64,${dataURI}`); // You don't have to resolve the buffer to base64. I just did this assuming you don't want to save it into a file. I just send the buffer to Discord which sends it as an attachment. But I tested and also have the issue when using the base64 data uri.

Here is a setup node environment that reproduces the issue for me. test.zip

I use the UPNG module locally because there was an issue using it in nodejs since the UPNG object isn't exported and pako isn't being required in the file. Other than adding const pako = require('pako') to the top and module.exports = UPNG to the bottom of the script and removing the time variables because nodejs was erroring due to them having duplicate variable names within the same context everything else is exactly how it is from the script in this repo.

Also if you test the reproduction environment if you're nodejs version is greater than version 13 you must install canvas by building from source as mentioned here because they haven't built binaries for the newer versions yet. Otherwise, if you're on node 13 or lower you can install it like any other packages.

Also just to note, my OS is Ubuntu 18.04.3 LTS and my nodejs version is v15.8.0

[BannerBomb]

And here's a screen capture I took so you can see each individual frame that is extracted from the gif along with the final apng output and how it shows up for me. I also opened the image in a different browser and I still got the issue. And tried it outside of discord to see if it was something to do with discord's image compression in which it wasn't discord doing it either.

https://user-images.githubusercontent.com/11788894/109096947-7b6c8100-76ec-11eb-9aaa-157bc04af7f9.mp4

[photopea]

I was not able to run your code. I see an error: "await is only valid in async function".

But I see this:

 UPNG.encode(frameData.buffers ...

Could you convert these frameData.buffers to static PNGs or GIFs and show me a ZIP of them? I just need an input of UPNG.encode() to reproduce the bug.

[BannerBomb]

Yea, the code block I sent was an incomplete snippet to illustrate how it was used. The archive should contain the full working code.

Here, The folder named 1 contains the frames that are used to write to canvas which is the buffers that come straight from gif-frames. Which is the result I get from lines 27 to 29 as outlined in the image below. Those buffers are then written to canvas right after and then I use context.getImageData to get the Uint8ClampedArray which is the buffer that I pass to UPNG.

The folder named buffer data contains the raw Uint8ClampedArray values which is what is output from context.getImageData and is what I pass straight to UPNG.encode. I wasn't able to save them into a png file from the clamped array because fs was just outputting corrupt files (I tried converting the Uint8ClampedArray to a buffer then writing that buffer to a png file but that didn't seem to work), and I didn't want to include another encoder to convert the clamped array otherwise it wouldn't be the same data that I was passing to the encode method. Anyways, each text file is what the frameData.buffers array includes. images.zip

[BannerBomb]

Trying to simplify what I wrote above in case you don't get what I was trying to explain, the folder buffer data is the Uint8ClamedArray buffer data that is in frameData.buffers I decided to output the raw buffer values because it wouldn't let me save it to a png file unless I exported the canvas image which wouldn't have contained the same data being passed to the encode method. You can reconstruct the array buffer with new Uint8ClampedArray([...values here...]) (I assume you already know that considering you made a library that handles that type of stuff) and push those buffers to a plain array then pass the plain array into the encode method.

[photopea]

I am still confused. Your ZIP contains images of 150 x 150 pixels (both PNGs and TXTs), but the APNG you shared is 80 x 80 pixels.

What is going on here? UPNG.encode() does not resize the input frames. Either your APNG is not a result of UPNG.encode(), or the images you sent me are not an input of UPNG.encode().

[BannerBomb]

the very first 80x80 images I shared was resized and put into svg code which was then converted back into a png by me. After I opened this issue, I started going through my code and removed any code that didn't affect the outcome that I was getting. They're the same frames just the 80x80 frames had extra processing on them, and both the 150x150 and 80x80 gives me the same output with the stacking issue.

[BannerBomb]

Wait, I just found what was causing the issue. I looked back at this example https://github.com/photopea/UPNG.js#encoder

// Read RGBA from canvas and encode with UPNG
var dta = ctx.getImageData(0,0,200,300).data;  // ctx is Context2D of a Canvas
//  dta = new Uint8Array(200 * 300 * 4);       // or generate pixels manually
var png = UPNG.encode([dta.buffer], 200, 300, 0);   console.log(new Uint8Array(png));

I was using var dta = ctx.getImageData(0,0,200,300).data; to get the buffer data. But wasn't appending .buffer to the end of it. I was passing the raw Uint8ClampedArray. It is fixed after I changed my code to var dta = ctx.getImageData(0,0,200,300).data.buffer;

[photopea]

So everything is working now?

[BannerBomb]

yes, I just retested all the other images with the change and it works fine now. Sorry, for wasting your time on this issue. I didn't realize the buffer property in the example at first.

[photopea]

But I am surprised that providing an Uint8Array instead of ArrayBuffer made such weird effect.