emcconville / wand

The ctypes-based simple ImageMagick binding for Python
http://docs.wand-py.org/
Other
1.4k stars 198 forks source link

Wand fails on Heroku #241

Closed briankung closed 5 years ago

briankung commented 9 years ago

Wand/ImageMagick not working in Heroku

I have a Stamp class that uses Wand 0.4.0 to draw some text onto images. The relevant portions are below:

from wand.drawing import Drawing
from wand.image import Image

from wand.color import Color

class Stamp(object):
    # ...
    def imprint():
        annotate = Drawing()
        image = Image(filename=file_path)

        # ...
        # Customer number annotation
        annotate.font_size = customer_number.font_size
        annotate.text(
            customer_number.x, customer_number.y, str(customer_number.text))

        # Certification year annotation
        annotate.font_size = certification_year.font_size
        annotate.text(
            certification_year.x, certification_year.y,
            "MEMBER SINCE {}".format(certification_year.text))

        annotate(image)

This works locally with no complaints. On Heroku, I am using the Heroku Cedar-14 imagemagick buildpack. When I try to to execute the ImageMagick related code directly from the heroku instance's shell (heroku run python manage.py shell or bash), I get various errors. If I supply a file path, it complains that it is missing a decode delegate:

>>> stamp.white() # Calls imprint()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/app/stamps/models.py", line 45, in white
    'white.png', '#001c3e', customer_number, certification_year)
  File "/app/stamps/models.py", line 70, in imprint
    image = Image(filename=file_path)
  File "/app/.heroku/python/lib/python3.4/site-packages/wand/image.py", line 2050, in __init__
    self.read(filename=filename, resolution=resolution)
  File "/app/.heroku/python/lib/python3.4/site-packages/wand/image.py", line 2107, in read
    self.raise_exception()
  File "/app/.heroku/python/lib/python3.4/site-packages/wand/resource.py", line 222, in raise_exception
    raise e
wand.exceptions.MissingDelegateError: no decode delegate for this image format `/app/stamps/lib/templates/white.png' @ error/constitute.c/ReadImage/544
  1. resource.py, line 222
  2. image.py, line 2107
  3. image.py, line 2050

...and if I provide it a file (and/or a format), it gives me a UnicodeDecodeError:

>>> white = open('white.png')
>>> img = Image(file=white)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/app/.heroku/python/lib/python3.4/site-packages/wand/image.py", line 2038, in __init__
    self.read(file=file, resolution=resolution)
  File "/app/.heroku/python/lib/python3.4/site-packages/wand/image.py", line 2094, in read
    blob = file.read()
  File "/app/.heroku/python/lib/python3.4/codecs.py", line 319, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte
  1. image.py, line 2094
  2. image.py, line 2038

Via shell or python manage.py shell, ImageMagick reports that it has all the necessary delegates to handle PNG:

>>> from subprocess import call
>>> call(['identify', '-list', 'format'])
   Format  Module    Mode  Description
-------------------------------------------------------------------------------
...
      PNG* PNG       rw-   Portable Network Graphics (libpng 1.2.51,1.2.50)
           See http://www.libpng.org/ for details about the PNG format.
    PNG00* PNG       rw-   PNG inheriting bit-depth and color-type from original
    PNG24* PNG       rw-   opaque or binary transparent 24-bit RGB (zlib 1.2.8)
    PNG32* PNG       rw-   opaque or transparent 32-bit RGBA
    PNG48* PNG       rw-   opaque or binary transparent 48-bit RGB
    PNG64* PNG       rw-   opaque or transparent 64-bit RGBA
     PNG8* PNG       rw-   8-bit indexed with optional binary transparency

convert in shell

Here's a standard filetype convert working just fine without annotation:

~/stamps/lib/templates $ ls
blue_opaque.png  blue.png  white.png
~/stamps/lib/templates $ convert white.png -debug 'Configure' white.jpg
2015-06-02T17:01:21+00:00 0:00.000 0.000u 6.8.9 Configure convert[7]: utility.c/ExpandFilenames/945/Configure
  Command line: convert {white.png} {-debug} {Configure} {white.jpg}
2015-06-02T17:01:21+00:00 0:00.020 0.000u 6.8.9 Configure convert[7]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/share/ImageMagick-6/coder.xml"
2015-06-02T17:01:21+00:00 0:00.020 0.000u 6.8.9 Configure convert[7]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/lib/x86_64-linux-gnu/ImageMagick-6.8.9//config-Q16/coder.xml"
2015-06-02T17:01:21+00:00 0:00.020 0.000u 6.8.9 Configure convert[7]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/etc/ImageMagick-6/coder.xml"
2015-06-02T17:01:21+00:00 0:00.020 0.000u 6.8.9 Configure convert[7]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/share/doc/ImageMagick-6/coder.xml"
2015-06-02T17:01:21+00:00 0:00.020 0.000u 6.8.9 Configure convert[7]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/app/.config/ImageMagick/coder.xml"
2015-06-02T17:01:21+00:00 0:00.020 0.000u 6.8.9 Configure convert[7]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/app/.magick/coder.xml"
2015-06-02T17:01:21+00:00 0:00.020 0.010u 6.8.9 Configure convert[7]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/share/ImageMagick-6/magic.xml"
2015-06-02T17:01:21+00:00 0:00.030 0.010u 6.8.9 Configure convert[7]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/lib/x86_64-linux-gnu/ImageMagick-6.8.9//config-Q16/magic.xml"
2015-06-02T17:01:21+00:00 0:00.030 0.010u 6.8.9 Configure convert[7]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/etc/ImageMagick-6/magic.xml"
2015-06-02T17:01:21+00:00 0:00.030 0.010u 6.8.9 Configure convert[7]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/share/doc/ImageMagick-6/magic.xml"
2015-06-02T17:01:21+00:00 0:00.030 0.010u 6.8.9 Configure convert[7]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/app/.config/ImageMagick/magic.xml"
2015-06-02T17:01:21+00:00 0:00.030 0.010u 6.8.9 Configure convert[7]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/app/.magick/magic.xml"
~/stamps/lib/templates $ ls
blue_opaque.png  blue.png  white.jpg  white.png

However, convert with the example annotation from the ImageMagick docs throws two errors about missing a delegates.xml, which looks familiar, and a type.xml.

~/stamps/lib/templates $ ls
blue_opaque.png  blue.png  white.png
~/stamps/lib/templates $ convert white.png -debug "Configure" -gravity south -stroke '#000C' -strokewidth 2 -annotate 0 'Faerie Dragon' white_annotated.png
2015-06-02T17:03:28+00:00 0:00.000 0.000u 6.8.9 Configure convert[34]: utility.c/ExpandFilenames/945/Configure
  Command line: convert {white.png} {-debug} {Configure} {-gravity} {south} {-stroke} {#000C} {-strokewidth} {2} {-annotate} {0} {Faerie Dragon} {white_annotated.png}
2015-06-02T17:03:28+00:00 0:00.000 0.000u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/share/ImageMagick-6/coder.xml"
2015-06-02T17:03:28+00:00 0:00.000 0.000u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/lib/x86_64-linux-gnu/ImageMagick-6.8.9//config-Q16/coder.xml"
2015-06-02T17:03:28+00:00 0:00.000 0.000u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/etc/ImageMagick-6/coder.xml"
2015-06-02T17:03:28+00:00 0:00.000 0.000u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/share/doc/ImageMagick-6/coder.xml"
2015-06-02T17:03:28+00:00 0:00.010 0.000u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/app/.config/ImageMagick/coder.xml"
2015-06-02T17:03:28+00:00 0:00.010 0.000u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/app/.magick/coder.xml"
2015-06-02T17:03:28+00:00 0:00.010 0.000u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/share/ImageMagick-6/magic.xml"
2015-06-02T17:03:28+00:00 0:00.010 0.000u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/lib/x86_64-linux-gnu/ImageMagick-6.8.9//config-Q16/magic.xml"
2015-06-02T17:03:28+00:00 0:00.010 0.000u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/etc/ImageMagick-6/magic.xml"
2015-06-02T17:03:28+00:00 0:00.010 0.000u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/share/doc/ImageMagick-6/magic.xml"
2015-06-02T17:03:28+00:00 0:00.010 0.000u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/app/.config/ImageMagick/magic.xml"
2015-06-02T17:03:28+00:00 0:00.010 0.000u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/app/.magick/magic.xml"
2015-06-02T17:03:29+00:00 0:00.590 0.110u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/share/ImageMagick-6/type.xml"
2015-06-02T17:03:29+00:00 0:00.590 0.110u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/lib/x86_64-linux-gnu/ImageMagick-6.8.9//config-Q16/type.xml"
2015-06-02T17:03:29+00:00 0:00.590 0.110u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/etc/ImageMagick-6/type.xml"
2015-06-02T17:03:29+00:00 0:00.590 0.110u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/share/doc/ImageMagick-6/type.xml"
2015-06-02T17:03:29+00:00 0:00.590 0.110u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/app/.config/ImageMagick/type.xml"
2015-06-02T17:03:29+00:00 0:00.590 0.110u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/app/.magick/type.xml"
2015-06-02T17:03:29+00:00 0:00.590 0.110u 6.8.9 Configure convert[34]: type.c/LoadTypeCache/1102/Configure
  Loading type configure file "built-in" ...
2015-06-02T17:03:29+00:00 0:00.650 0.130u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/share/ImageMagick-6/delegates.xml"
2015-06-02T17:03:29+00:00 0:00.660 0.130u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/lib/x86_64-linux-gnu/ImageMagick-6.8.9//config-Q16/delegates.xml"
2015-06-02T17:03:29+00:00 0:00.660 0.130u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/etc/ImageMagick-6/delegates.xml"
2015-06-02T17:03:29+00:00 0:00.660 0.130u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/usr/share/doc/ImageMagick-6/delegates.xml"
2015-06-02T17:03:29+00:00 0:00.660 0.130u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/app/.config/ImageMagick/delegates.xml"
2015-06-02T17:03:29+00:00 0:00.660 0.130u 6.8.9 Configure convert[34]: configure.c/GetConfigureOptions/679/Configure
  Searching for configure file: "/app/.magick/delegates.xml"
2015-06-02T17:03:29+00:00 0:00.660 0.130u 6.8.9 Configure convert[34]: delegate.c/LoadDelegateCache/1475/Configure
  Loading delegate configuration file "built-in" ...
convert: UnableToOpenConfigureFile `type.xml' @ warning/configure.c/GetConfigureOptions/706.
convert: UnableToReadFont `(null)' @ error/annotate.c/RenderFreetype/1127.
convert: UnableToOpenConfigureFile `delegates.xml' @ warning/configure.c/GetConfigureOptions/706.
convert: UnableToReadFont `(null)' @ error/annotate.c/RenderFreetype/1127.
~/stamps/lib/templates $ ls
blue_opaque.png  blue.png  white_annotated.png  white.png

The files, delegates.xml and type.xml, both exist on my Heroku instance. If I copy them from /etc/ImageMagick/ or /app/.apt/etc/ImageMagick-6/ to a directory that convert is expecting, like /app/.magick/, then all but the Font errors disappear. And I can make the font errors go away by specifying a font (ex. convert ... -font 'Nimbus Mono L) from convert -list font.

However, this is not a long-term solution, as Heroku uses an ephemeral writeable filesystem and changes are lost. It's also a configuration step that would ideally be handled by the buildpack. Unfortunately, I wasn't able to figure out if there was something in the buildpack that I could tweak.

Other attempts

I halfheartedly customized the buildpack to use the same version of ImageMagick that I was using locally (6.9.1-3), which failed to install properly. I also tried specifying a runtime.txt to make the Heroku Python environment match my local environment (3.4.3), which worked just fine, but it wasn't the problem.

Help!

If anyone has gotten Wand working on Heroku and I'm missing something obvious, please do tell! I'm new to Python. For what it's worth, this is within a Django app, though the Stamp model itself is just vanilla Python model.

Thanks!

emcconville commented 5 years ago

This is not a wand-py issue, but more of a case-study for devops. Wand is a C-API binding for ImageMagick's MagickWand library, and it can only assume that the underlining environment + dependencies are installed correctly. For Heroku, a customized buildpack is the correct method for deploying your solution, and GitHub is full of examples that may work for your business requirements. It might also be time to evaluate if a configuration management solution is needed in you development cycle. Something like Ansible, Chef, or Puppet to name a few.