netlify / build-image

This is the build image used for running automated builds
MIT License
497 stars 196 forks source link

build fails with imagemagick and .webp file format #266

Closed kalwalt closed 5 years ago

kalwalt commented 5 years ago

I have my personal site made with jekyll and amp, lately i add an image fallback and i migrated to the .webp format, I use the jekyll-responsive-image plugin, the problem is that i need to use the imagemagick6 version build with --webp suppor,t i used linuxbrew for this part of the process, as i uploaded my new version to netlify server the build process, of course, fails:

7:08:28 PM:   Liquid Exception: no decode delegate for this image format `/opt/build/repo/assets/images/uploads/Art_Is_a_joke_interactive_slice.webp' @ error/constitute.c/ReadImage/544 in /opt/build/repo/_posts/2018-24-10-new-ideas-about-my-interactive-artworks.md
7:08:29 PM: failed during stage 'building site': Build script returned non-zero exit code: 1
7:08:28 PM: jekyll 3.8.5 | Error:  no decode delegate for this image format `/opt/build/repo/assets/images/uploads/Art_Is_a_joke_interactive_slice.webp' @ error/constitute.c/ReadImage/544
7:08:30 PM: Shutting down logging, 16 messages pending

I read in the docs that is possible "install" the missing packages but i stopped because i didn't really understand the process, and also because maybe should be interesting adding this feature (.webp) as default. i would make some simple questions in respect of the installing process:

"A frequent use pattern is to get a copy of the binary we need to build your site, and add it to your repository so you can run it directly during your build."

What does it means? Should i get the source code and run a build with it? May instead install with a .deb package?

By default your build starts in the root of your repository, so it could be as easy as using ./my-binary as part of your build command.

I don't understand what is intended with build command, i'm testing in local with the build-image, i run build jekyll build for build my site what addictional command should i use?

kalwalt commented 5 years ago

additional infos: from buildbot:

convert -version
Version: ImageMagick 6.7.7-10 2018-09-28 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP    
buildbot@ea87ced0d696:/$ convert -list delegate

Path: /etc/ImageMagick/delegates.xml

Delegate                Command
-------------------------------------------------------------------------------
    blender =>          "blender" -b "%i" -F PNG -o "%o""\n"convert" -concatenate "%o*.png" "%o"
        cdr =>          "uniconvertor" "%i" "%o.svg"; mv "%o.svg" "%o"
        cgm =>          "ralcgm" -d ps -oC < "%i" > "%o" 2> "%Z"
 dng:decode =>          "ufraw-batch" --silent --create-id=also --out-type=png --out-depth=16 "--output=%u.png" "%i"
        dot =>          "dot" -Tsvg "%i" -o "%o"
        dvi =>          "dvips" -q -o "%o" "%i"
        eps<=>pdf       "gs" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 "-sDEVICE=pdfwrite" "-sOutputFile=%o" "-f%i"
        eps<=>ps        "gs" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=nodevice" "-sOutputFile=%o" "-f%i"
        fig =>          "fig2dev" -L ps "%i" "%o"
        hpg =>          "hp2xx" -q -m eps -f `basename "%o"` "%i";     mv -f `basename "%o"` "%o"
       hpgl =>          "if [ -e hp2xx -o -e /usr/bin/hp2xx ]; then     hp2xx -q -m eps -f `basename "%o"` "%i";     mv -f `basename "%o"` "%o";   else     echo "You need to install hp2xx to use HPGL files with ImageMagick.";     exit 1;   fi"
        htm =>          "html2ps" -U -o "%o" "%i"
       html =>          "html2ps" -U -o "%o" "%i"
      https =>          "curl" -s -k -o "%o" "https:%F"
       ilbm =>          "ilbmtoppm" "%i" > "%o"
        man =>          "groff" -man -Tps "%i" > "%o"
       miff<= show      "/usr/bin/display" -delay 0 -window-group %[group] -title "%l " "ephemeral:%i"
mpeg:decode =>          "ffmpeg" -v -1 -i "%i" -vframes %S -vcodec pam -an -f rawvideo -y "%u.pam" 2> "%Z"
        pdf<=>eps       "gs" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=nodevice" "-sOutputFile=%o" "-f%i"
        pdf<=>ps        "gs" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=nodevice" "-sOutputFile=%o" "-f%i"
        pnm<= ilbm      "ppmtoilbm" -24if "%i" > "%o"
        pov =>          "povray" "+i%i" -D0 "+o%o" +fn%q +w%w +h%h +a -q9 "-kfi%s" "-kff%n";"convert" -concatenate "%o*.png" "%o"
         ps<=>eps       "gs" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=nodevice" "-sOutputFile=%o" "-f%i"
         ps<=>pdf       "gs" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=pdfwrite" "-sOutputFile=%o" "-f%i"
         ps<= print     "lpr "%i"
       rgba<= rle       "rawtorle" -o "%o" -v "%i"
       scan =>          "scanimage" -d "%i" > "%o"
      scanx =>          "scanimage" > "%o"
      shtml =>          "html2ps" -U -o "%o" "%i"
        sid =>          "mrsidgeodecode" -if sid -i "%i" -of tif -o "%o" > "%u"
        svg =>          "rsvg-convert" -o "%o" "%i"
       tiff<= launch    "gimp" "%i"
        txt<=>ps        "enscript" -o "%o" "%i"
        wmf =>          "wmf2eps" -o "%o" "%i"

from my system:

convert -version
Version: ImageMagick 6.9.7-4 Q16 x86_64 20170114 http://www.imagemagick.org
Copyright: © 1999-2017 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP 
Delegates (built-in): bzlib djvu fftw fontconfig freetype jbig jng jpeg lcms lqr ltdl lzma openexr pangocairo png tiff wmf x xml zlib
convert -list delegate

Path: /etc/ImageMagick-6/delegates.xml

Delegate                Command
-------------------------------------------------------------------------------
    blender =>          "blender' -b '%i' -F PNG -o '%o''\n'convert' -concatenate '%o*.png' '%o"
        bmp<= jxr       "/bin/mv '%i' '%i.bmp'; 'JxrEncApp' -i '%i.bmp' -o '%o.jxr'; /bin/mv '%i.bmp' '%i'; /bin/mv '%o.jxr' '%o"
        bmp<= wdp       "/bin/mv '%i' '%i.bmp'; 'JxrEncApp' -i '%i.bmp' -o '%o.jxr'; /bin/mv '%i.bmp' '%i'; /bin/mv '%o.jxr' '%o"
        bpg =>          "bpgdec' -b 16 -o '%o.png' '%i'; /bin/mv '%o.png' '%o"
        cdr =>          "uniconvertor' '%i' '%o.svg'; /bin/mv '%o.svg' '%o"
        cgm =>          "uniconvertor' '%i' '%o.svg'; /bin/mv '%o.svg' '%o"
 dng:decode =>          "ufraw-batch' --silent --create-id=also --out-type=png --out-depth=16 '--output=%u.png' '%i"
        doc =>          "soffice' --convert-to pdf -outdir `dirname '%i'` '%i' 2> '%u'; /bin/mv '%i.pdf' '%o"
       docx =>          "soffice' --convert-to pdf -outdir `dirname '%i'` '%i' 2> '%u'; /bin/mv '%i.pdf' '%o"
        dot =>          "dot' -Tsvg '%i' -o '%o"
        dvi =>          "dvips' -sstdout=%%stderr -o '%o' '%i"
        dxf =>          "uniconvertor' '%i' '%o.svg'; /bin/mv '%o.svg' '%o"
        eps<=>pdf       "gs' -sstdout=%%stderr -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 '-sDEVICE=pdfwrite' '-sOutputFile=%o' '-f%i"
        eps<=>ps        "gs' -sstdout=%%stderr -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 '-sDEVICE=ps2write' '-sOutputFile=%o' '-f%i"
        fig =>          "uniconvertor' '%i' '%o.svg'; /bin/mv '%o.svg' '%o"
        hpg =>          "hp2xx' -sstdout=%%stderr -m eps -f `basename '%o'` '%i';     /bin/mv -f `basename '%o'` '%o"
       hpgl =>          "hp2xx' -sstdout=%%stderr -m eps -f `basename '%o'` '%i';     /bin/mv -f `basename '%o'` '%o"
        htm =>          "html2ps' -U -o '%o' '%i"
       html =>          "html2ps' -U -o '%o' '%i"
      https =>          "curl' -s -k -L -o '%o' 'https:%M"
       ilbm =>          "ilbmtoppm' '%i' > '%o"
        jpg<= lep       "lepton' '%i' '%o"
        jxr =>          "/bin/mv '%i' '%i.jxr'; 'JxrDecApp' -i '%i.jxr' -o '%o.bmp' -c 0; /bin/mv '%i.jxr' '%i'; /bin/mv '%o.bmp' '%o"
        lep =>          "lepton' '%i' '%o"
       miff<= show      "display' -immutable -delay 0 -title '%M' '%i"
mpeg:decode =>          "ffmpeg' -nostdin -v -1 -i '%i' -vframes %S -vcodec pam -an -f rawvideo -y '%u.pam' 2> '%u"
        odt =>          "soffice' --convert-to pdf -outdir `dirname '%i'` '%i' 2> '%u'; /bin/mv '%i.pdf' '%o"
        pdf<=>eps       "gs' -sstdout=%%stderr -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 '-sDEVICE=eps2write' -sPDFPassword='%a' '-sOutputFile=%o' '-f%i"
        pdf<=>ps        "gs' -sstdout=%%stderr -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 '-sDEVICE=ps2write' -sPDFPassword='%a' '-sOutputFile=%o' '-f%i"
        png<= bpg       "bpgenc' -b 12 -sstdout=%%stderr '%Q' -o '%o' '%i"
        png<= webp      "cwebp' -quiet %Q '%i' -o '%o"
        pnm<= ilbm      "ppmtoilbm' -24if '%i' > '%o"
        ppt =>          "soffice' --convert-to pdf -outdir `dirname '%i'` '%i' 2> '%u'; /bin/mv '%i.pdf' '%o"
       pptx =>          "soffice' --convert-to pdf -outdir `dirname '%i'` '%i' 2> '%u'; /bin/mv '%i.pdf' '%o"
         ps<=>eps       "gs' -sstdout=%%stderr -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 '-sDEVICE=eps2write' '-sOutputFile=%o' '-f%i"
         ps<=>pdf       "gs' -sstdout=%%stderr -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 '-sDEVICE=pdfwrite' '-sOutputFile=%o' '-f%i"
         ps<= print     "lpr '%i"
      shtml =>          "html2ps' -U -o '%o' '%i"
        sid =>          "mrsidgeodecode' -if sid -i '%i' -of tif -o '%o' > '%u"
        svg =>          "rsvg-convert' -o '%o' '%i"
       tiff<= launch    "gimp' '%i"
        wdp =>          "/bin/mv '%i' '%i.jxr'; 'JxrDecApp' -i '%i.jxr' -o '%o.bmp'; /bin/mv '%i.jxr' '%i'; /bin/mv '%o.bmp' '%o"
       webp =>          "dwebp' -pam '%i' -o '%o"
        xls =>          "soffice' --convert-to pdf -outdir `dirname '%i'` '%i' 2> '%u'; /bin/mv '%i.pdf' '%o"
       xlsx =>          "soffice' --convert-to pdf -outdir `dirname '%i'` '%i' 2> '%u'; /bin/mv '%i.pdf' '%o"
rybit commented 5 years ago

What it means is that you’d need to copy the binaries into your repo and then access them directly during your build. There is no sudo access so installing deb packages won’t work during the build.

If you have more troubles, please write into support for further assistance.

kalwalt commented 5 years ago

@rybit thank you for the answer, i added them but maybe not in the right place. Definetely if I will write to the support. Have you future plans for an Imagemagick version with .webp support?

bcomnes commented 5 years ago

@kalwalt yes see https://github.com/netlify/build-image/issues/157

kalwalt commented 5 years ago

@bcomnes thank you!