php-imagine / Imagine

PHP Object Oriented image manipulation library
https://imagine.readthedocs.io
Other
4.42k stars 530 forks source link

2.7 Mb animated GIF breaking the server #680

Closed ThomasLabstep closed 6 years ago

ThomasLabstep commented 6 years ago

Issue description

We are trying to resize a 2.7mb animated gif and it kills our server. On my mac, it reaches the timeout of 1min and uses 6gb of memory and 100% of cpu.

What's the PHP version you are using?

PHP 7.1.13 (cli) (built: Feb  1 2018 13:38:42) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.1.13, Copyright (c) 1999-2017, by Zend Technologies

What's the imaging library you are using [gd/imagick/gmagick/any]?

Version: ImageMagick 7.0.8-11 Q16 x86_64 2018-08-29 https://www.imagemagick.org
Copyright: © 1999-2018 ImageMagick Studio LLC
License: https://www.imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules
Delegates (built-in): bzlib freetype jng jpeg ltdl lzma png tiff xml zlib

What's the imaging library configuration

imagick

imagick module => enabled
imagick module version => 3.4.3
imagick classes => Imagick, ImagickDraw, ImagickPixel, ImagickPixelIterator, ImagickKernel
Imagick compiled with ImageMagick version => ImageMagick 7.0.8-11 Q16 x86_64 2018-08-29 https://www.imagemagick.org
Imagick using ImageMagick library version => ImageMagick 7.0.8-11 Q16 x86_64 2018-08-29 https://www.imagemagick.org
ImageMagick copyright => © 1999-2018 ImageMagick Studio LLC
ImageMagick release date => 2018-08-29
ImageMagick number of supported formats:  => 222
ImageMagick supported formats => 3FR, 3G2, 3GP, AAI, AI, ART, ARW, AVI, AVS, BGR, BGRA, BGRO, BMP, BMP2, BMP3, BRF, CAL, CALS, CANVAS, CAPTION, CIN, CIP, CLIP, CMYK, CMYKA, CR2, CRW, CUR, CUT, DCM, DCR, DCRAW, DCX, DDS, DFONT, DNG, DOT, DPX, DXT1, DXT5, EPDF, EPI, EPS, EPS2, EPS3, EPSF, EPSI, EPT, EPT2, EPT3, ERF, FAX, FILE, FITS, FRACTAL, FTP, FTS, G3, G4, GIF, GIF87, GRADIENT, GRAY, GRAYA, GROUP4, GV, HALD, HDR, HISTOGRAM, HRZ, HTM, HTML, HTTP, HTTPS, ICB, ICO, ICON, IIQ, INFO, INLINE, IPL, ISOBRL, ISOBRL6, JNG, JNX, JPE, JPEG, JPG, JPS, JSON, K25, KDC, LABEL, M2V, M4V, MAC, MAP, MASK, MAT, MATTE, MEF, MIFF, MKV, MNG, MONO, MOV, MP4, MPC, MPEG, MPG, MRW, MSL, MSVG, MTV, MVG, NEF, NRW, NULL, ORF, OTB, OTF, PAL, PALM, PAM, PANGO, PATTERN, PBM, PCD, PCDS, PCL, PCT, PCX, PDB, PDF, PDFA, PEF, PES, PFA, PFB, PFM, PGM, PGX, PICON, PICT, PIX, PJPEG, PLASMA, PNG, PNG00, PNG24, PNG32, PNG48, PNG64, PNG8, PNM, PPM, PS, PS2, PS3, PSB, PSD, PTIF, PWP, RADIAL-GRADIENT, RAF, RAS, RAW, RGB, RGBA, RGBO, RGF, RLA, RLE, RMF, RW2, SCR, SCT, SFW, SGI, SHTML, SIX, SIXEL, SPARSE-COLOR, SR2, SRF, STEGANO, SUN, SVG, SVGZ, TEXT, TGA, THUMBNAIL, TIFF, TIFF64, TILE, TIM, TTC, TTF, TXT, UBRL, UBRL6, UIL, UYVY, VDA, VICAR, VID, VIFF, VIPS, VST, WBMP, WMV, WPG, X3F, XBM, XC, XCF, XPM, XPS, XV, YCbCr, YCbCrA, YUV

Directive => Local Value => Master Value
imagick.locale_fix => 0 => 0
imagick.skip_version_check => 0 => 0
imagick.progress_monitor => 0 => 0

Minimal PHP code to reproduce the error:

The code that I want to be able to disable is:

    /**
     * {@inheritdoc}
     */
    public function resize(BoxInterface $size, $filter = ImageInterface::FILTER_UNDEFINED)
    {
        try {
            if ($this->layers->count() > 1) {
                $this->imagick = $this->imagick->coalesceImages();
                foreach ($this->imagick as $frame) {
                    $frame->resizeImage($size->getWidth(), $size->getHeight(), $this->getFilter($filter), 1);
                }
                $this->imagick = $this->imagick->deconstructImages();
            } else {
                $this->imagick->resizeImage($size->getWidth(), $size->getHeight(), $this->getFilter($filter), 1);
            }
        } catch (\ImagickException $e) {
            throw new RuntimeException('Resize operation failed', $e->getCode(), $e);
        }
        return $this;
    }

How can we skip the "$this->layers->count() > 1" condition ? For now we will fork this library to leave only the else part. The dangerous code is "imagick->coalesceImages". We don't want animated resized gif, only a thumbnail is fine for us.

We are using LiipImagine. They have a 'animated' option that is false by default. But we still go down to this method.

        file_small:
            data_loader: stream.amazon_s3
            quality: 100
            format: jpg
            filters:
                upscale:   { min:  [60, 60] }
                thumbnail: { size: [60, 60], mode: outbound, allow_upscale: true }
        file_medium:
            data_loader: stream.amazon_s3
            quality: 100
            format: jpg
            filters:
                upscale:   { min:  [150, 150] }
                thumbnail: { size: [150, 150], mode: outbound, allow_upscale: true }
        file_large:
            data_loader: stream.amazon_s3
            quality: 100
            format: jpg
            filters:
                upscale:   { min:  [900, 900] }
                thumbnail: { size: [900, 900], mode: inset, allow_upscale: true }

Just wanted to report this issue.

Please advise.

mlocati commented 6 years ago

If you only need a resized version of just one frame, you can use a code like this:

$imagine = new Imagine\Imagick\Imagine();
$image = $imagine->open('path/to/your/huge.gif');
$firstFrame = $image->layers()->current();
$firstFrame->resize(new Imagine\Image\Box(60, 60));
$firstFrame->save('path/to/thumbnail.gif');
ThomasLabstep commented 6 years ago

We switched back to gd. But then tiff are not supported. We will try with gmagick.

If it doesn't work, we will fork this library and remove the call to coalesceImages.

Thanks for the feedback.

mlocati commented 6 years ago

Why don't you just call $image->layers()->current() and work on it? It returns a Imagine\Image\ImageInterface: you can call resize on it...

ThomasLabstep commented 6 years ago

You mean implement my own resize method at LiipImagineBundle level instead of using the default resize method?

That's a good idea!

namespace Liip\ImagineBundle\Imagine\Filter;

use Imagine\Exception\InvalidArgumentException;
use Imagine\Filter\FilterInterface;
use Imagine\Image\ImageInterface;

/**
 * Filter for resizing an image relative to its existing dimensions.
 *
 * @author Jeremy Mikola <jmikola@gmail.com>
 */
class RelativeResize implements FilterInterface
{
    private $method;
    private $parameter;

    /**
     * Constructs a RelativeResize filter with the given method and argument.
     *
     * @param string $method    BoxInterface method
     * @param mixed  $parameter Parameter for BoxInterface method
     *
     * @throws \Imagine\Exception\InvalidArgumentException
     */
    public function __construct($method, $parameter)
    {
        if (!in_array($method, array('heighten', 'increase', 'scale', 'widen'))) {
            throw new InvalidArgumentException(sprintf('Unsupported method: ', $method));
        }

        $this->method = $method;
        $this->parameter = $parameter;
    }

    /**
     * {@inheritdoc}
     */
    public function apply(ImageInterface $image)
    {
        return $image->resize(call_user_func(array($image->getSize(), $this->method), $this->parameter));
    }
}
mlocati commented 6 years ago

I don't know LiipImagineBundle, but you can extend the existing RelativeResize class and do something like this:

use Liip\ImagineBundle\Imagine\Filter\RelativeResize;
use Imagine\Image\ImageInterface;

class MyRelativeResize extends RelativeResize
{
    /**
     * {@inheritdoc}
     *
     * @see \Liip\ImagineBundle\Imagine\Filter\RelativeResize::apply()
     */
    public function apply(ImageInterface $image)
    {
        return parent::apply($image()->layers()->current());
    }
}
ThomasLabstep commented 6 years ago

Thank you for your suggestion, I will try it later.

I'm looking into the config animated parameter. Because I don't want to animate the GIF.

For some reason, it's defined by default to false in LiipImagineBundle, and I'm trying to debug what imagine does with this variable.

/**
     * @param array  $options
     * @param string $path
     */
    private function prepareOutput(array $options, $path = null)
    {
        if (isset($options['format'])) {
            $this->imagick->setImageFormat($options['format']);
        }

        var_dump($options['animated']);
        exit;

        if (isset($options['animated']) && true === $options['animated']) {
            $format = isset($options['format']) ? $options['format'] : 'gif';
            $delay = isset($options['animated.delay']) ? $options['animated.delay'] : null;
            $loops = isset($options['animated.loops']) ? $options['animated.loops'] : 0;

            $options['flatten'] = false;

            $this->layers->animate($format, $delay, $loops);
        } else {
            $this->layers->merge();
        }
        $this->applyImageOptions($this->imagick, $options, $path);

        // flatten only if image has multiple layers
        if ((!isset($options['flatten']) || $options['flatten'] === true) && count($this->layers) > 1) {
            $this->flatten();
        }
    }
mlocati commented 6 years ago

If animated is not set (or is not true), we call the merge method, which basically assign the layers (frames in case of animated GIFs) to the image. This is useful in case you add new layers (frames) to the image, otherwise it basically does nothing.

ThomasLabstep commented 6 years ago

Thanks for the clarification!

I'll now try your solution with 'apply($image()->layers()->current());'

You're so helpful 👍

ThomasLabstep commented 6 years ago

A huge thank you. It works flawlessly. ❤️ ❤️ ❤️

class ThumbnailFilterLoader extends BaseThumbnailFilterLoader
{
    /**
     * @param ImageInterface $image
     * @param array          $options
     *
     * @return ImageInterface
     */
    public function load(ImageInterface $image, array $options = [])
    {
        return parent::load($image->layers()->current(), $options);
    }
}