Filesize concerns

[jvanasco]

I've noticed that the images created by PIL are quite large.

gifs seem to be over 4x larger than they could be.
pngs are around 130% larger

i tested this just off of pillow created files vs the same ones imported/exported through a graphics app. ie, i used lossless compression on the second time.

as a quick example, i drew some text on a canvas. a (256 color) gif from pillow was around 12k; the same gif was opened & exported to 2k using an app. pillow generated it as 3 color gif at 3k (larger filesize than the much better looking 256 color), while it was a few hundred bytes via an app.

there are a handful of advanced image 'compressing' apps out there too, which do lossless optimizations. i didn't run the images through any of them.

i'm wondering if anyone here knows any techniques or ways to address this. i thought this might have something to do with the old gif patents, but they're long expired and pngs seem to be affected.

[wiredfool]

There's a comment in the GIF c code that it's not a particularly optimized quantizer, and that there are other algorithims that would be better. There's also a outstanding issue to add pngquant support, which is an optimized png quantization library. (not sure if that would work in the gif case, it might).

[jvanasco]

i have a workaround, where i save files to a named tempfile , then run pngcrush / gifsicle / jpegtran on it... outputting into another named tempfile.

It works well on Mac/Linux. The 3 apps are available on Linux and ImageOptim.app ( http://imageoptim.com/ ) has a resources folder that has those 3 ( plus a few more ) compiled; I just add that to my path.

It has 2 huge caveats though:

1- The overhead and limitations of external processes (which can create issues when you're in a daemonized/threaded scenario )

2- It's not windows-safe. I don't think i'd ever use windows at this point, but the NamedTempfile support in windows doesn't support a second process opening up the file ( linux/bsd does )

[wiredfool]

There are a couple of methods in Pillow for saving to Gif (and jpeg) using external programs. I'm not really happy with them for a few reasons, those caveats among them.

[hugovk]

From https://github.com/python-pillow/Pillow/issues/2480 (closed as duplicate of this):

---

What did you do?

Reverse a gif get it much bigger size on disk

What did you expect to happen?

Still the original size or only a bit more

What versions of Pillow and Python are you using?

latest Pillow and 3.6 Python

from PIL import Image, ImageSequence
from PIL import ImagePalette
with Image.open('sd.gif') as im:
    if im.is_animated:
        frames = [f.copy() for f in ImageSequence.Iterator(im)]
        frames.reverse()
        frames[0].save('out.gif', save_all=True, append_images = frames[1:])  

[jvanasco]

I'm not familiar with the exact inner workings, but #2480 is probably not the same as this (or an issue at all).

I should probably address some of the earlier comments in here too.

This ticket shouldn't have to do with quantization (which is lossy) but is meant to address bloat issues when doing a lossless encoding as gif. Perhaps Pillow is running the gif through the quantizer when it doesn't have to, and could just ignore it if the number of colors is small enough?

In terms of #2480, animated gifs are built off of deltas -- not individual frames. The bloat could also be created by:

• the original gif uses diffs, the Pillow created gif uses full frames
• reversing the image drastically changes the filesize of a few frames.
• other things (It's the end of the day!)

[wiredfool]

Actually, it's simpler than that. Pillow doesn't do compression on GIFs. It only implements the uncompressed frame format. So anything that's going to do GIF compression is going to beat it. There was code for that, but it was in the closed source PILplus extension.

Pillow does diffs of frames, though there's some subtle details of the disposal methods when writing that would mess with that. There is a new quantizer that's been merged since this issue came in, but that's also probably not the main issue anymore.

[TheRockettek]

Meanwhile my pillow just made a 800kb file into a 1.2Gb file somehow

[wiredfool]

If you’d like to contribute a gif compressor under the PIL license, go for it.

[radarhere]

5291 adds LZW encoding to GIFs, reducing filesize by 30% - or in the case of #3788, by 75%.

[radarhere]

Does anyone in this issue have a specific image that demonstrates this problem?

[jvanasco]

I can try to recreate my older test images this week.

[xmo-odoo]

I've got an animated gif which increases 10x in size when read, then re-stored as-is (with save_all=True): I've also found this SO entry where a user went from a 2MB GIF to a 7.5MB GIF after resizing (incidentally it's a bit frustrating that resizing a gif will leave only the first frame but not signal the issue, maybe it should at least warn if the gif has multiple frames?)

[xmo-odoo]

Complement: I was on a version of Pillow preceding #5291, updating to 9.2.0 does improve things significantly (the image still increases after saving but only to 400k).

[radarhere]

I've also found this SO entry where a user went from a 2MB GIF to a 7.5MB GIF after resizing

There isn't a complete example in the StackOverflow post of the resizing, but for the initial example where the user is just resaving the image, testing it with main now it is only output as 2.3mb.

incidentally it's a bit frustrating that resizing a gif will leave only the first frame but not signal the issue, maybe it should at least warn if the gif has multiple frames?

This is because resize() returns a copy of the image, and copying an image only includes the first frame. My feeling is that raising a warning when resizing a multiframe file would be an annoyance both to any users who were used to this behaviour, and also to users who were only concerned with the first frame.

[radarhere]

I'm hoping that between #5291 in Pillow 8.2.0 and #7568 in the upcoming Pillow 10.2.0 (reducing https://github.com/python-pillow/Pillow/issues/617#issuecomment-1188984138 from 408kb to 345kb), the GIF filesize concerns in this issue are now resolved.