search

lovell / sharp-libvips

Packaging scripts to prebuild libvips and its dependencies - you're probably looking for https://github.com/lovell/sharp
Apache License 2.0
178 stars 100 forks source link

What/where is the expected output of ./build.sh 8.6.3 linux-x64 ? #4

Closed Nantris closed 6 years ago

Nantris commented 6 years ago

I'm running this command with Docker installed and it seems to complete successfully, but I can't for the life of me find any output from this command anywhere.

lovell commented 6 years ago

Hello, when you say "output", are you referring to stdout logging or an output file (or both? or something else?).

The build script maps the current working directory $PWD into the containers so the resultant .tar.gz files end up there.

My best guess would be that an error is occurring.

lovell commented 6 years ago

I'll assume from your question opened against sharp that you got this working.

Nantris commented 6 years ago

Ah unfortunately I did not, at least on Linux. Works beautifully elsewhere.

I've not had time to go back to this yet and gather enough info to meaningfully move this issue forward. Thanks for all your help always @lovell.

Nantris commented 6 years ago

@lovell thanks a lot for your help. I finally got around to looking back into this.

I'm running these commands from a Linux Mint 18.3 x64 machine.

If I run ./build.sh 8.6.3 I get a libvips-8.6.3-win32-x64.tar.gz in my current working directory, but if I run ./build.sh 8.6.3 linux-x64 I get nothing.

This is the full console output I get when running ./build.sh 8.6.3 linux-x64

wheezy: Pulling from library/debian
Digest: sha256:f3dc825cffd4d98f9af923a704f67b790c741a4f706296f0f77000ec580d3a1e
Status: Image is up to date for debian:wheezy
stretch: Pulling from library/debian
Digest: sha256:f1f61086ea01a72b30c7287adee8c929e569853de03b7c462a8ac75e0d0224c4
Status: Image is up to date for debian:stretch
Using default tag: latest
latest: Pulling from socialdefect/raspbian-jessie-core
b4c8471bb3fe: Already exists 
Digest: sha256:cc5cc638f11fa93709a44f99b42395e90e2d5c4089e903698404c36401b3497b
Status: Image is up to date for socialdefect/raspbian-jessie-core:latest
Building linux-x64...
Sending build context to Docker daemon   2.56kB
Step 1/4 : FROM debian:wheezy
 ---> 2d79c539d0db
Step 2/4 : MAINTAINER Lovell Fuller <npm@lovell.info>
 ---> Using cache
 ---> 23c60565365c
Step 3/4 : RUN   echo "deb http://ftp.debian.org/debian wheezy-backports main" | tee /etc/apt/sources.list.d/wheezy-backports.list &&   apt-get update &&   apt-get install -y build-essential autoconf libtool nasm gtk-doc-tools texinfo gperf advancecomp &&   apt-get -t wheezy-backports install -y jq &&   curl https://sh.rustup.rs -sSf | sh -s -- -y
 ---> Using cache
 ---> 94cffe5f846c
Step 4/4 : ENV   PATH="/root/.cargo/bin:$PATH"   PLATFORM="linux-x64"   FLAGS="-O3"
 ---> Using cache
 ---> 4dbd9388a1b6
Successfully built 4dbd9388a1b6
Successfully tagged vips-dev-linux-x64:latest
glib version 2.55.1 has been superseded by 2.57.1
xml2 version 2.9.7 has been superseded by 2.9.8
gsf version 1.14.42 has been superseded by 1.14.43
jpeg version 1.5.3 has been superseded by 1.5.90
webp version 0.6.1 has been superseded by 1.0.0
gdkpixbuf version 2.36.11 has been superseded by 2.37.0
freetype version 2.9 has been superseded by 2.9.1
harfbuzz version 1.7.4 has been superseded by 1.8.4
pango version 1.41.0 has been superseded by 1.42.1
svg version 2.42.0 has been superseded by 2.43.2

I wasn't sure if using just linux was valid, so I tried it, ./build.sh 8.6.3 linux results in this output:

wheezy: Pulling from library/debian
Digest: sha256:f3dc825cffd4d98f9af923a704f67b790c741a4f706296f0f77000ec580d3a1e
Status: Image is up to date for debian:wheezy
stretch: Pulling from library/debian
Digest: sha256:f1f61086ea01a72b30c7287adee8c929e569853de03b7c462a8ac75e0d0224c4
Status: Image is up to date for debian:stretch
Using default tag: latest
latest: Pulling from socialdefect/raspbian-jessie-core
b4c8471bb3fe: Already exists 
Digest: sha256:cc5cc638f11fa93709a44f99b42395e90e2d5c4089e903698404c36401b3497b
Status: Image is up to date for socialdefect/raspbian-jessie-core:latest
6614ccf119f081e86e2429cc0286d850a0659281341896c141a37e6c32df5eef  libvips-8.6.3-win32-x64.tar.gz

Running ./test-linux-x64.sh results in:

Testing debian:jessie...
jessie: Pulling from library/debian
Digest: sha256:a64a7d8ff7ff87edb78004ef0b159661546d2ddbd82772128b344c90cf8422ab
Status: Image is up to date for debian:jessie
./test-linux-x64.sh: 16: ./test-linux-x64.sh: cannot create packaging/debian:jessie.log: Directory nonexistent
debian:jessie fail
cat: 'packaging/debian:jessie.log': No such file or directory
Testing debian:stretch...
Error response from daemon: Get https://registry-1.docker.io/v2/library/debian/manifests/stretch: net/http: TLS handshake timeout
./test-linux-x64.sh: 16: ./test-linux-x64.sh: cannot create packaging/debian:stretch.log: Directory nonexistent
debian:stretch fail
cat: 'packaging/debian:stretch.log': No such file or directory
Testing ubuntu:trusty...
trusty: Pulling from library/ubuntu
Digest: sha256:71529e96591eb36a4100cd0cc5353ff1a2f4ee7a85011e3d3dd07cb5eb524a3e
Status: Image is up to date for ubuntu:trusty
./test-linux-x64.sh: 16: ./test-linux-x64.sh: cannot create packaging/ubuntu:trusty.log: Directory nonexistent
ubuntu:trusty fail
cat: 'packaging/ubuntu:trusty.log': No such file or directory
Testing ubuntu:xenial...
xenial: Pulling from library/ubuntu
Digest: sha256:14066a391d902c386d6164d44ade3460ba044abcdf8df88b0ff79a6f635be8d3
Status: Image is up to date for ubuntu:xenial
./test-linux-x64.sh: 16: ./test-linux-x64.sh: cannot create packaging/ubuntu:xenial.log: Directory nonexistent
ubuntu:xenial fail
cat: 'packaging/ubuntu:xenial.log': No such file or directory
Testing centos7...
7: Pulling from library/centos
Digest: sha256:b67d21dfe609ddacf404589e04631d90a342921e81c40aeaf3391f6717fa5322
Status: Image is up to date for centos:7
./test-linux-x64.sh: 25: ./test-linux-x64.sh: cannot create packaging/centos7.log: Directory nonexistent
centos7 fail
cat: packaging/centos7.log: No such file or directory
Testing archlinux...
latest: Pulling from pritunl/archlinux
Digest: sha256:c111496ffe9a106c6dd4d183bbacbb7136ed729008aa158a92dca37a976b4ec0
Status: Image is up to date for pritunl/archlinux:latest
./test-linux-x64.sh: 33: ./test-linux-x64.sh: cannot create packaging/archlinux.log: Directory nonexistent
archlinux fail
cat: packaging/archlinux.log: No such file or directory
lovell commented 6 years ago

The glib version 2.55.1 has been superseded by 2.57.1 messages mean you'll need to update the VERSION_GLIB etc. variables to point to the latest versions of their respective libraries in https://github.com/lovell/sharp-libvips/blob/master/build/lin.sh#L18

Nantris commented 6 years ago

Thanks very much for the tip @lovell! Going to continue to work on this today and hopefully finally crack the case.

Nantris commented 6 years ago

@lovell I updated the dependency version numbers, but now I get: autoreconf: 'configure.ac' or 'configure.in' is required. Any thoughts?

Below is the message with some preceding lines. I can provide the full output of the command in a gist if needed.

Making install in testbed
make[2]: Entering directory `/deps/lcms2/testbed'
make[3]: Entering directory `/deps/lcms2/testbed'
make[3]: Nothing to be done for `install-exec-am'.
make[3]: Nothing to be done for `install-data-am'.
make[3]: Leaving directory `/deps/lcms2/testbed'
make[2]: Leaving directory `/deps/lcms2/testbed'
make[2]: Entering directory `/deps/lcms2'
make[3]: Entering directory `/deps/lcms2'
 /bin/mkdir -p '/target/lib/pkgconfig'
 /usr/bin/install -c -m 644 lcms2.pc '/target/lib/pkgconfig'
make[3]: Leaving directory `/deps/lcms2'
make[2]: Leaving directory `/deps/lcms2'
make[1]: Leaving directory `/deps/lcms2'
autoreconf: 'configure.ac' or 'configure.in' is required
Nantris commented 6 years ago

Downloading the lcms2-2.9.tar.gz and extracting and building with the same instructions you used (minus --target and --host) builds without any issue on my local machine.

I also haphazardly threw a autoreconf -fiv into the lin.sh just after cd ${DEPS}/lcms2 and just before: ./configure --host=${CHOST} --prefix=${TARGET} --enable-shared --disable-static --disable-dependency-tracking, but no luck with that either.

Nantris commented 6 years ago

@lovell I know you're already offering a tremendous library for free and I am very appreciative for that and for all the help you've offered. I'm having a lot of trouble with this script and feel bad troubling you with it too given all that you've already put out for free use.

All that being said, is there any chance you'd consider support for prebuilt binaries for Linux which expect zlib 1.2.11? I'd love to be able to have my project (https://recollectr.io) support Linux and have gotten quite a few requests, but I've been stuck on packaging a version that works. Part of this has been that it seems like many distros skipped straight from zlib 1.2.8 to zlib 1.2.11.

The zlib homepage states, "Due to the bug fixes, any installations of 1.2.9 or 1.2.10 should be immediately replaced with 1.2.11." 1.2.11 has been the most recent version of zlib since January 15, 2017, so it looks to be very stable, and all new major Linux distributions include it with the exception of Debian, which does not include 1.2.9 in any case.

First release supporting 1.2.11 by distro

Distribution Release date Zlib version
Ubuntu 17.10 19 October 2017 1.2.11
openSUSE Leap 15 25 May 2018 1.2.11
Fedora 26 11 July 2017 1.2.11
Mageia 6 16 July 2017 1.2.11
*Debian Stretch 17 June 2017 1.2.8

*Debian looks like it won't include zlib 1.2.11 until Q2 or Q3 of 2019, but everything else is now using 1.2.11. I'd love to simply build my own, but so far I've been unable to do anything in this regard.

Zlib bug fixes since 1.2.9

Any help you can offer either with troubleshooting this script or supporting prebuilt binaries for zlib 1.2.11 would be greatly appreciated by myself and indirectly quite a few Linux users I'm sure. Sorry for the wall of text and thank you again @lovell.

lovell commented 6 years ago

The lin.sh script already compiles and provides zlib 1.2.11. If you want to use the binaries it creates with Electron on a Linux OS that only provides zlib 1.2.8 then you'll need to downgrade VERSION_ZLIB in the script to match that version (and remove the version_latest "zlib" check).

Nantris commented 6 years ago

@lovell thanks for your reply. I know it should compile for zlib 1.2.11 and that's what I'd very much like to do, but the script fails for me with:

make[1]: Leaving directory `/deps/lcms2'
autoreconf: 'configure.ac' or 'configure.in' is required

Execution stops at this point. I've tried the script on 2 Linux machines and got the same message. I don't know if building for Linux should work from Windows. I assumed not, but I tried that too without luck.

lovell commented 6 years ago

I'll have to try locally to see what's up but might not have time to do this for a week or two.

Nantris commented 6 years ago

Thanks @lovell! I appreciate it a lot

Nantris commented 6 years ago

I thought perhaps this was a transient issue, so I came back to this to try again. Unfortunately I get a different issue now. I tried running ./build.sh 8.6.1 linux-x64 and I got:

make[3]: Entering directory `/deps/ffi'
 /bin/mkdir -p '/target/lib/../lib'
 /bin/bash ./libtool   --mode=install /usr/bin/install -c -s  libffi.la '/target/lib/../lib'
libtool: install: /usr/bin/install -c .libs/libffi.so.6.0.4 /target/lib/../lib/libffi.so.6.0.4
libtool: install: strip --strip-unneeded /target/lib/../lib/libffi.so.6.0.4
libtool: install: (cd /target/lib/../lib && { ln -s -f libffi.so.6.0.4 libffi.so.6 || { rm -f libffi.so.6 && ln -s libffi.so.6.0.4 libffi.so.6; }; })
libtool: install: (cd /target/lib/../lib && { ln -s -f libffi.so.6.0.4 libffi.so || { rm -f libffi.so && ln -s libffi.so.6.0.4 libffi.so; }; })
libtool: install: /usr/bin/install -c .libs/libffi.lai /target/lib/../lib/libffi.la
libtool: finish: PATH="/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/target/bin:/sbin" ldconfig -n /target/lib/../lib
----------------------------------------------------------------------
Libraries have been installed in:
   /target/lib/../lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
 /bin/mkdir -p '/target/share/info'
 /usr/bin/install -c -m 644 ./doc/libffi.info '/target/share/info'
 install-info --info-dir='/target/share/info' '/target/share/info/libffi.info'
install-info: warning: nothing done since /usr/bin/install-info doesn't exist,
install-info: warning: you might want to install an info-browser package.
 /bin/mkdir -p '/target/lib/pkgconfig'
 /usr/bin/install -c -m 644 libffi.pc '/target/lib/pkgconfig'
make[3]: Leaving directory `/deps/ffi'
make[2]: Leaving directory `/deps/ffi'
make[1]: Leaving directory `/deps/ffi'
/packaging/build/lin.sh: 105: /packaging/build/lin.sh: ./configure: not found

Full console output: https://gist.github.com/Slapbox/062bb2c3fdd105b7e251a4837f491a2e

Any thoughts? Several related package versions were "superseded" when I came back to this, and I had to update their version numbers in the build script.

lovell commented 6 years ago

I've started updating these scripts ahead of the forthcoming libvips v8.7.0. The latest glib requires a newer automake than Debian Wheezy provides so for now you'll need to peg glib at v2.56.1.

Nantris commented 6 years ago

Thanks very much for the update and the advice @lovell!

Nantris commented 6 years ago

@lovell will the new version you're working on support zlib@1.2.11? I've revisited this issue several times in recent weeks, but I've not had any success unfortunately.

The closest I've gotten is getting back to the issue I had here after disabling the check for a newer version of glib.

According to this it seems like lcms might be a viable alternative to lcms2?

Nantris commented 6 years ago

@lovell any update on the possibility of official support for zlib@1.2.11 ? I know you've got a lot going on with this project, but I figured it wouldn't hurt to check in. I've tried a few more times to make headway with the script, without luck unfortunately.

lovell commented 6 years ago

Both the current v8.6.1 and forthcoming v8.7.0 bundles provide zlib v1.2.11.

Nantris commented 6 years ago

Thanks for the update @lovell. Unfortunately 8.6.1 won't build for me no matter what I try, but I'll definitely give 8.7.0 a shot. But is there any plan to offer a precompiled version for zlib@1.2.11 as you do with zlib@1.2.9, in light of its fast growing adoption?

I'm hopeful that 8.7.0 will build for me, but I'm worried that I'll hit a snag somewhere given my luck so far.

lovell commented 6 years ago

"But is there any plan to offer a precompiled version for zlib@1.2.11 as you do with zlib@1.2.9"

The precompiled binaries have provided zlib v1.2.11 for at least a year.

Nantris commented 6 years ago

Thanks for that @lovell. I have no idea how I've ended up so off the mark. I'll re-investigate to find out where I've been going wrong. Perhaps in the past I was trying to build with 1.2.11 on a system with 1.2.9 and now that I have a system with 1.2.11 I'm mixing up timelines and versions. Not sure, but that's really excellent news (at least it's news to me.)

Thank again very much for all that you do and for setting me right here.

Nantris commented 6 years ago

Hey @lovell. I got a chance to give this another shot on Linux 18.3, which uses zlib@1.2.11, but I'm still running into the same issues as before with the prebuilts. Namely, that it looks for zlib@1.2.9:

Uncaught Error: /lib/x86_64-linux-gnu/libz.so.1: version `ZLIB_1.2.9' not found (required by /home/myUser/projects/myApp/app/node_modules/sharp/build/Release/../../vendor/lib/libpng16.so.16)

I don't understand why it's still looking for 1.2.9. Can you shine any light on this for me?

My Sharp/vendor/versions.json looks like this:

{
  "cairo": "1.14.12",
  "croco": "0.6.12",
  "exif": "0.6.21",
  "expat": "2.2.5",
  "ffi": "3.2.1",
  "fontconfig": "2.12.6",
  "freetype": "2.9",
  "gdkpixbuf": "2.36.11",
  "gif": "5.1.4",
  "glib": "2.55.1",
  "gsf": "1.14.42",
  "harfbuzz": "1.7.4",
  "jpeg": "1.5.3",
  "lcms": "2.9-",
  "orc": "0.4.28",
  "pango": "1.41.0",
  "pixman": "0.34.0",
  "png": "1.6.34",
  "svg": "2.42.0",
  "tiff": "4.0.9-cda4b06",
  "vips": "8.6.1",
  "webp": "0.6.1",
  "xml": "2.9.7",
  "zlib": "1.2.11"
}
lovell commented 6 years ago

I think you may have got this backwards. In your local version try VERSION_ZLIB=1.2.8 and remove the version_latest "zlib" ... check.

Nantris commented 6 years ago

@lovell sorry for the lack of clarity. These latest messages have moved off the topic of running the sharp-libvips. My interest in this script was for getting Sharp working on Linux distros using zlib@1.2.11, but since there are prebuilt binaries which support this I've moved on to trying to get the relevant prebuilt binary working. I've followed up here in the Sharp repo: https://github.com/lovell/sharp/issues/1095#issuecomment-422551755

Nantris commented 6 years ago

Thanks for your reply. I was on Mint 19 previous which actually DOES have 1.2.11. I can't believe how dumb I am. I've been shuffling versions and trying a lot to get Sharp working on Linux and got mixed up.

Sorry to waste your time with that.