search

JuliaPackaging / BinaryBuilder.jl

Binary Dependency Builder for Julia
https://binarybuilder.org
Other
392 stars 101 forks source link

building tarballs locally fails #377

Closed bjarthur closed 4 years ago

bjarthur commented 6 years ago

sorry for yet another issue. i finally got all the way through the wizard and chose to "get me a local git repo". when i julia build_tarballs.jl it fails, even though i successfully built from within the wizard, as well as with the --debug flag. this is on OS X, master branch of BinaryBuilder, julia 1.0.

$ julia build_tarballs.jl 
[ Info: Building for i686-linux-gnu, x86_64-linux-gnu, aarch64-linux-gnu, arm-linux-gnueabihf, powerpc64le-linux-gnu, i686-linux-musl, x86_64-linux-musl, aarch64-linux-musl, arm-linux-musleabihf, x86_64-apple-darwin14, x86_64-unknown-freebsd11.1, i686-w64-mingw32, x86_64-w64-mingw32
ERROR: LoadError: Build for GperfBuilder on i686-linux-gnu did not complete successfully

Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at ./logging.jl:318 [inlined]
 [3] #build#175(::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Array{String,1}, ::Bool, ::Function, ::BinaryBuilder.QemuRunner, ::String, ::Array{ExecutableProduct,1}, ::String, ::Linux, ::Prefix) at /Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/src/AutoBuild.jl:473
 [4] (::getfield(BinaryBuilder, Symbol("#kw##build")))(::NamedTuple{(:verbose, :ignore_manifests, :debug),Tuple{Bool,Array{String,1},Bool}}, ::typeof(build), ::BinaryBuilder.QemuRunner, ::String, ::Array{ExecutableProduct,1}, ::String, ::Linux, ::Prefix) at ./none:0
 [5] (::getfield(BinaryBuilder, Symbol("##164#170")){Bool,Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:debug,),Tuple{Bool}}},String,String,VersionNumber,String,Array{Platform,1},typeof(products),Array{Any,1}})(::String) at /Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/src/AutoBuild.jl:391
 [6] mktempdir(::getfield(BinaryBuilder, Symbol("##164#170")){Bool,Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:debug,),Tuple{Bool}}},String,String,VersionNumber,String,Array{Platform,1},typeof(products),Array{Any,1}}, ::String) at ./file.jl:576
 [7] mktempdir at ./file.jl:574 [inlined]
 [8] macro expansion at ./task.jl:263 [inlined]
 [9] #autobuild#162(::Bool, ::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:debug,),Tuple{Bool}}}, ::Function, ::String, ::String, ::VersionNumber, ::Array{Pair{String,String},1}, ::String, ::Array{Platform,1}, ::typeof(products), ::Array{Any,1}) at /Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/src/AutoBuild.jl:265
 [10] (::getfield(BinaryBuilder, Symbol("#kw##autobuild")))(::NamedTuple{(:verbose, :debug),Tuple{Bool,Bool}}, ::typeof(autobuild), ::String, ::String, ::VersionNumber, ::Array{Pair{String,String},1}, ::String, ::Array{Platform,1}, ::typeof(products), ::Array{Any,1}) at ./none:0
 [11] macro expansion at ./logging.jl:307 [inlined]
 [12] #build_tarballs#151(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Array{String,1}, ::String, ::VersionNumber, ::Array{Pair{String,String},1}, ::String, ::Array{Platform,1}, ::typeof(products), ::Array{Any,1}) at /Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/src/AutoBuild.jl:114
 [13] build_tarballs(::Array{String,1}, ::String, ::VersionNumber, ::Array{Pair{String,String},1}, ::String, ::Array{Platform,1}, ::Function, ::Array{Any,1}) at /Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/src/AutoBuild.jl:21
 [14] top-level scope at none:0
 [15] include at ./boot.jl:317 [inlined]
 [16] include_relative(::Module, ::String) at ./loading.jl:1041
 [17] include(::Module, ::String) at ./sysimg.jl:29
 [18] exec_options(::Base.JLOptions) at ./client.jl:229
 [19] _start() at ./client.jl:421
in expression starting at /Users/arthurb/src/GperfBuilder/build_tarballs.jl:53
$ cat build_tarballs.jl 
# Note that this script can accept some limited command-line arguments, run
# `julia build_tarballs.jl --help` to see a usage message.
using BinaryBuilder

name = "GperfBuilder"
version = v"3.1.0"

# Collection of sources required to build GperfBuilder
sources = [
    "http://ftp.gnu.org/pub/gnu/gperf/gperf-3.1.tar.gz" =>
    "588546b945bba4b70b6a3a616e80b4ab466e3f33024a352fc2198112cdbb3ae2",

]

# Bash recipe for building across all platforms
script = raw"""
cd $WORKSPACE/srcdir/gperf-3.1/
./configure --prefix=$prefix --host=$target
make
make install

"""

# These are the platforms we will build for by default, unless further
# platforms are passed in on the command line
platforms = [
    Linux(:i686, libc=:glibc),
    Linux(:x86_64, libc=:glibc),
    Linux(:aarch64, libc=:glibc),
    Linux(:armv7l, libc=:glibc, call_abi=:eabihf),
    Linux(:powerpc64le, libc=:glibc),
    Linux(:i686, libc=:musl),
    Linux(:x86_64, libc=:musl),
    Linux(:aarch64, libc=:musl),
    Linux(:armv7l, libc=:musl, call_abi=:eabihf),
    MacOS(:x86_64),
    FreeBSD(:x86_64),
    Windows(:i686),
    Windows(:x86_64)
]

# The products that we will ensure are always built
products(prefix) = [
    ExecutableProduct(prefix, "gperf", :gperf)
]

# Dependencies that must be installed before this package can be built
dependencies = [

]

# Build the tarballs, and possibly a `build.jl` as well.
build_tarballs(ARGS, name, version, sources, script, platforms, products, dependencies)
$ cat build/i686-linux-gnu/LBOzBhPP/destdir/logs/GperfBuilder.log 
`/Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/deps/qemu/bin/qemu-system-x86_64 -kernel /Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/deps/qemu/bzImage -m 12288M -cpu host -smp 2 -M accel=hvf -nographic -nodefaults -rtc base=utc,driftfix=slew -device virtio-serial -chardev stdio,id=charconsole0 -device virtconsole,chardev=charconsole0,id=console0 -device virtserialport,chardev=charcomm0,id=comm0 -device virtio-net-pci,netdev=networking -netdev user,id=networking -drive if=virtio,file=/Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/deps/downloads/Rootfs.v2018.9.18.x86_64-linux-gnu.squashfs,format=raw -fsdev local,security_model=none,id=fsdev0,path=/Users/arthurb/src/GperfBuilder/build/i686-linux-gnu/LBOzBhPP -device virtio-9p-pci,id=fs0,fsdev=fsdev0,mount_tag=workspace0 -fsdev local,security_model=none,id=fsdev1,path=/Users/arthurb/src/GperfBuilder/build/i686-linux-gnu/LBOzBhPP/metadir -device virtio-9p-pci,id=fs1,fsdev=fsdev1,mount_tag=workspace1 -drive if=virtio,file=/Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/deps/downloads/BaseCompilerShard-i686-linux-gnu.v2018.10.10.x86_64-linux-gnu.squashfs,format=raw -drive if=virtio,file=/Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/deps/downloads/GCC-i686-linux-gnu.v4.8.5.x86_64-linux-gnu.squashfs,format=raw -drive if=virtio,file=/Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/deps/downloads/LLVM.v6.0.1-0.x86_64-linux-gnu.squashfs,format=raw -drive if=virtio,file=/Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/deps/downloads/BaseCompilerShard-x86_64-linux-gnu.v2018.10.10.x86_64-linux-gnu.squashfs,format=raw -drive if=virtio,file=/Users/arthurb/.julia/packages/BinaryBuilder/hGAV0/deps/downloads/GCC-x86_64-linux-gnu.v4.8.5.x86_64-linux-gnu.squashfs,format=raw -device virtio-serial -chardev socket,path=/tmp/tmpYYjeTP/qemu_comm.socket,server,nowait,id=charcomm0 -append 'quiet console=hvc0 root=/dev/vda rootflags=ro rootfstype=squashfs noinitrd init=/sandbox'`
qemu-system-x86_64: virtio_bus_set_host_notifier: unable to init event notifier: Too many open files (-24)
qemu-system-x86_64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
rsync: pipe: Address family not supported by protocol (97)
rsync error: error in IPC code (code 14) at pipe.c(122) [sender=3.1.3]
reboot: Power down
bjarthur commented 6 years ago

note that i get further along using the same build_tarball.jl file on linux, before erroring out b/c of a shared file system. so this issue is something OS X specific.

bjarthur commented 6 years ago

specifically, with BINARYBUILDER_RUNNER=privileged, BINARYBUILDER_STORAGE_DIR pointing to a local drive, popfirst!(DEPOT_PATH), and julia 1.0.1 intalled locally, i get the following on scientific linux:

$ ../julia-1.0.1/bin/julia build_tarballs.jl 
[ Info: Building for i686-linux-gnu, x86_64-linux-gnu, aarch64-linux-gnu, arm-linux-gnueabihf, powerpc64le-linux-gnu, i686-linux-musl, x86_64-linux-musl, aarch64-linux-musl, arm-linux-musleabihf, x86_64-apple-darwin14, x86_64-unknown-freebsd11.1, i686-w64-mingw32, x86_64-w64-mingw32
[ Info: Downloading https://julialangmirror.s3.amazonaws.com/binarybuilder/Rootfs.v2018.9.18.x86_64-linux-gnu.squashfs to /opt/users/julia/downloads/Rootfs.v2018.9.18.x86_64-linux-gnu.squashfs...
[23:25:56] ######################################################################## 100.0%
[ Info: Downloading https://julialangmirror.s3.amazonaws.com/binarybuilder/BaseCompilerShard-i686-linux-gnu.v2018.10.10.x86_64-linux-gnu.squashfs to /opt/users/julia/downloads/BaseCompilerShard-i686-linux-gnu.v2018.10.10.x86_64-linux-gnu.squashfs...
[23:25:58] ######################################################################## 100.0%
[ Info: Downloading https://julialangmirror.s3.amazonaws.com/binarybuilder/GCC-i686-linux-gnu.v4.8.5.x86_64-linux-gnu.squashfs to /opt/users/julia/downloads/GCC-i686-linux-gnu.v4.8.5.x86_64-linux-gnu.squashfs...
[23:26:01] ######################################################################## 100.0%
[ Info: Downloading https://julialangmirror.s3.amazonaws.com/binarybuilder/LLVM.v6.0.1-0.x86_64-linux-gnu.squashfs to /opt/users/julia/downloads/LLVM.v6.0.1-0.x86_64-linux-gnu.squashfs...
[23:26:06] ######################################################################## 100.0%
[ Info: Downloading https://julialangmirror.s3.amazonaws.com/binarybuilder/BaseCompilerShard-x86_64-linux-gnu.v2018.10.10.x86_64-linux-gnu.squashfs to /opt/users/julia/downloads/BaseCompilerShard-x86_64-linux-gnu.v2018.10.10.x86_64-linux-gnu.squashfs...
[23:26:09] ######################################################################## 100.0%
[ Info: Downloading https://julialangmirror.s3.amazonaws.com/binarybuilder/GCC-x86_64-linux-gnu.v4.8.5.x86_64-linux-gnu.squashfs to /opt/users/julia/downloads/GCC-x86_64-linux-gnu.v4.8.5.x86_64-linux-gnu.squashfs...
[23:26:12] ######################################################################## 100.0%
[ Info: Running privileged container via `sudo`, may ask for your password:
At line 1151, ABORTED (Invalid argument)!
ERROR: LoadError: Build for GperfBuilder on i686-linux-gnu did not complete successfully

Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at ./logging.jl:318 [inlined]
 [3] #build#175(::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Array{String,1}, ::Bool, ::Function, ::BinaryBuilder.UserNSRunner, ::String, ::Array{ExecutableProduct,1}, ::String, ::Linux, ::Prefix) at /opt/users/julia/julia-1.0.1/local/share/julia/packages/BinaryBuilder/hGAV0/src/AutoBuild.jl:473
 [4] (::getfield(BinaryBuilder, Symbol("#kw##build")))(::NamedTuple{(:verbose, :ignore_manifests, :debug),Tuple{Bool,Array{String,1},Bool}}, ::typeof(build), ::BinaryBuilder.UserNSRunner, ::String, ::Array{ExecutableProduct,1}, ::String, ::Linux, ::Prefix) at ./none:0
 [5] (::getfield(BinaryBuilder, Symbol("##164#170")){Bool,Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:debug,),Tuple{Bool}}},String,String,VersionNumber,String,Array{Platform,1},typeof(products),Array{Any,1}})(::String) at /opt/users/julia/julia-1.0.1/local/share/julia/packages/BinaryBuilder/hGAV0/src/AutoBuild.jl:391
 [6] mktempdir(::getfield(BinaryBuilder, Symbol("##164#170")){Bool,Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:debug,),Tuple{Bool}}},String,String,VersionNumber,String,Array{Platform,1},typeof(products),Array{Any,1}}, ::String) at ./file.jl:576
 [7] mktempdir at ./file.jl:574 [inlined]
 [8] macro expansion at ./task.jl:263 [inlined]
 [9] #autobuild#162(::Bool, ::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:debug,),Tuple{Bool}}}, ::Function, ::String, ::String, ::VersionNumber, ::Array{Pair{String,String},1}, ::String, ::Array{Platform,1}, ::typeof(products), ::Array{Any,1}) at /opt/users/julia/julia-1.0.1/local/share/julia/packages/BinaryBuilder/hGAV0/src/AutoBuild.jl:265
 [10] (::getfield(BinaryBuilder, Symbol("#kw##autobuild")))(::NamedTuple{(:verbose, :debug),Tuple{Bool,Bool}}, ::typeof(autobuild), ::String, ::String, ::VersionNumber, ::Array{Pair{String,String},1}, ::String, ::Array{Platform,1}, ::typeof(products), ::Array{Any,1}) at ./none:0
 [11] macro expansion at ./logging.jl:307 [inlined]
 [12] #build_tarballs#151(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Array{String,1}, ::String, ::VersionNumber, ::Array{Pair{String,String},1}, ::String, ::Array{Platform,1}, ::typeof(products), ::Array{Any,1}) at /opt/users/julia/julia-1.0.1/local/share/julia/packages/BinaryBuilder/hGAV0/src/AutoBuild.jl:114
 [13] build_tarballs(::Array{String,1}, ::String, ::VersionNumber, ::Array{Pair{String,String},1}, ::String, ::Array{Platform,1}, ::Function, ::Array{Any,1}) at /opt/users/julia/julia-1.0.1/local/share/julia/packages/BinaryBuilder/hGAV0/src/AutoBuild.jl:21
 [14] top-level scope at none:0
 [15] include at ./boot.jl:317 [inlined]
 [16] include_relative(::Module, ::String) at ./loading.jl:1041
 [17] include(::Module, ::String) at ./sysimg.jl:29
 [18] exec_options(::Base.JLOptions) at ./client.jl:229
 [19] _start() at ./client.jl:421
in expression starting at /opt/users/julia/bar/build_tarballs.jl:54
staticfloat commented 6 years ago

Please show me the output of the following:

using BinaryBuilder
BinaryBuilder.runshell(Linux(:x86_64); verbose=true)
bjarthur commented 6 years ago

ooops, seemingly i didn't hit the "comment" button three days ago. here's the output for OS X:

julia> using BinaryBuilder
[ Info: Recompiling stale cache file /Users/arthurb/.julia/compiled/v1.0/BinaryBuilder/WHzgM.ji for BinaryBuilder [12aac903-9f7c-5d81-afc2-d9565ea332ae]

julia> BinaryBuilder.runshell(Linux(:x86_64); verbose=true)
[ Info: Downloading https://julialangmirror.s3.amazonaws.com/binarybuilder/Rootfs.v2018.9.18.x86_64-linux-gnu.squashfs to /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/Rootfs.v2018.9.18.x86_64-linux-gnu.squashfs...
[07:46:48] ######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash ea97920cb26dedf55dc09195968e4ecfb2b5e5b01126ace761d30ab87b20bdb5 for file /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/Rootfs.v2018.9.18.x86_64-linux-gnu.squashfs
[ Info: Rewriting Rootfs.v2018.9.18.x86_64-linux-gnu.squashfs from UID 0 -> 1772020894
[ Info: Downloading https://julialangmirror.s3.amazonaws.com/binarybuilder/BaseCompilerShard-x86_64-linux-gnu.v2018.10.10.x86_64-linux-gnu.squashfs to /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/BaseCompilerShard-x86_64-linux-gnu.v2018.10.10.x86_64-linux-gnu.squashfs...
[07:47:02] ######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash 6b8635430269b4d30652b0f8cd14cabf53389587c462379b4b42b2a22dece5db for file /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/BaseCompilerShard-x86_64-linux-gnu.v2018.10.10.x86_64-linux-gnu.squashfs
[ Info: Rewriting BaseCompilerShard-x86_64-linux-gnu.v2018.10.10.x86_64-linux-gnu.squashfs from UID 0 -> 1772020894
[ Info: Downloading https://julialangmirror.s3.amazonaws.com/binarybuilder/GCC-x86_64-linux-gnu.v4.8.5.x86_64-linux-gnu.squashfs to /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/GCC-x86_64-linux-gnu.v4.8.5.x86_64-linux-gnu.squashfs...
[07:48:24] ######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash a6227eac0630907fd2c5f809276eed9836a75e3d32fbd55b9475b266ab89cd49 for file /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/GCC-x86_64-linux-gnu.v4.8.5.x86_64-linux-gnu.squashfs
[ Info: Rewriting GCC-x86_64-linux-gnu.v4.8.5.x86_64-linux-gnu.squashfs from UID 0 -> 1772020894
[ Info: Downloading https://julialangmirror.s3.amazonaws.com/binarybuilder/LLVM.v6.0.1-0.x86_64-linux-gnu.squashfs to /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/LLVM.v6.0.1-0.x86_64-linux-gnu.squashfs...
[07:49:07] ######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash 80578c2f73b6d3cd4a5138bd732f4b2aff28968e0f2cf1b35e290f2a202de9f8 for file /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/LLVM.v6.0.1-0.x86_64-linux-gnu.squashfs
[ Info: Rewriting LLVM.v6.0.1-0.x86_64-linux-gnu.squashfs from UID 0 -> 1772020894
[ Info: Downloading https://github.com/staticfloat/PixmanBuilder/releases/download/v0.34.0-0/Pixman.x86_64-apple-darwin14.tar.gz to /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/Pixman.x86_64-apple-darwin14.tar.gz...
[07:49:10] ######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash 7d10eba60ac7824093199d557c7fb9ec0750cd2d3f2314d5a2480677fc1f88aa for file /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/Pixman.x86_64-apple-darwin14.tar.gz
[ Info: Downloading https://github.com/staticfloat/GlibBuilder/releases/download/v2.54.2-2/Glib.x86_64-apple-darwin14.tar.gz to /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/Glib.x86_64-apple-darwin14.tar.gz...
[07:49:14] ######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash 29b52a6a95ef48a0da40e2c3a9385dd2fc93d472a965763f932a15fcd96a100b for file /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/Glib.x86_64-apple-darwin14.tar.gz
[ Info: Downloading https://github.com/staticfloat/PcreBuilder/releases/download/v8.41-0/pcre.x86_64-apple-darwin14.tar.gz to /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/pcre.x86_64-apple-darwin14.tar.gz...
[07:49:16] ######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash e9b99a17525ed9d694e312be7e556e0666f64515c99c7cb87257a411abf8fb9e for file /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/pcre.x86_64-apple-darwin14.tar.gz
[ Info: Downloading https://github.com/staticfloat/GettextBuilder/releases/download/v0.19.8-0/libgettext.x86_64-apple-darwin14.tar.gz to /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/libgettext.x86_64-apple-darwin14.tar.gz...
[07:49:22] ######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash b7f64f9c34705c157f6c6a241454a2984cc80cefd11f46c63345341791cc012f for file /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/libgettext.x86_64-apple-darwin14.tar.gz
[ Info: Downloading https://github.com/staticfloat/LibffiBuilder/releases/download/v3.2.1-0/libffi.x86_64-apple-darwin14.tar.gz to /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/libffi.x86_64-apple-darwin14.tar.gz...
[07:49:23] ######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash 287f289b428cfae58084aa4bab21cb07cc405a7c5535b62a39dc8b68d163d250 for file /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/libffi.x86_64-apple-darwin14.tar.gz
[ Info: Downloading https://github.com/staticfloat/ZlibBuilder/releases/download/v1.2.11-3/Zlib.x86_64-apple-darwin14.tar.gz to /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/Zlib.x86_64-apple-darwin14.tar.gz...
[07:49:24] ######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash d921114efca229f53b4faaac3d6d153862559aa38bc6df1b702732036394c634 for file /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/Zlib.x86_64-apple-darwin14.tar.gz
[ Info: Downloading https://github.com/Keno/QemuBuilder/releases/download/v2.12.50-0/Qemu.v2.12.50.x86_64-apple-darwin14.tar.gz to /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/Qemu.v2.12.50.x86_64-apple-darwin14.tar.gz...
[07:49:29] ######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash 98210f754e39008d982f447cad60043f75ed8831b3ff36d464856ac24d7c603c for file /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/Qemu.v2.12.50.x86_64-apple-darwin14.tar.gz
[ Info: Downloading https://github.com/Keno/LinuxBuilder/releases/download/v4.15-rc2/linux.x86_64-linux-gnu.tar.gz to /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/linux.x86_64-linux-gnu.tar.gz...
[07:49:32] ######################################################################## 100.0%
[ Info: No hash cache found
[ Info: Calculated hash 0d02413529e635d4af6d2122f6aba22d261f43616bb286da3855325988f9ac3b for file /Users/arthurb/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/linux.x86_64-linux-gnu.tar.gz
Setting parent shell interrupt to ^] (use ^C as usual, ^] to interrupt Julia)
qemu-system-x86_64: virtio_bus_set_host_notifier: unable to init event notifier: Too many open files (-24)
qemu-system-x86_64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
verbose sandbox enabled (running in init mode)
Parsed --cd as "/workspace/"
Parsed --workspace as "9p/workspace0" -> "/workspace"
Parsed --map as "/dev/vdb" -> "/opt/x86_64-linux-gnu/BaseCompilerShard-2018.10.10"
Parsed --map as "/dev/vdc" -> "/opt/x86_64-linux-gnu/GCC-4.8.5"
Parsed --map as "/dev/vdd" -> "/opt/x86_64-linux-gnu/LLVM-6.0.1-0"
Reading 77 environment mappings
--> Creating overlay workdir at /proc
--> Mounting overlay of / at /tmp (modifications in /proc/upper/rootfs, workspace in /proc/work/rootfs)
--> mapping /dev/vdd to /tmp/opt/x86_64-linux-gnu/LLVM-6.0.1-0
--> mapping /dev/vdc to /tmp/opt/x86_64-linux-gnu/GCC-4.8.5
--> mapping /dev/vdb to /tmp/opt/x86_64-linux-gnu/BaseCompilerShard-2018.10.10
--> Mounting overlay of /tmp/opt/x86_64-linux-gnu/BaseCompilerShard-2018.10.10:/tmp/opt/x86_64-linux-gnu/GCC-4.8.5:/tmp/opt/x86_64-linux-gnu/LLVM-6.0.1-0 at /tmp/opt/x86_64-linux-gnu (modifications in /proc/upper/x86_64-linux-gnu, workspace in /proc/work/x86_64-linux-gnu)
--> Mounting procfs at /tmp/proc
--> Mounting /dev at /tmp/dev
--> workspacing 9p/workspace0 to /tmp/workspace
--> Mounting procfs at /proc
About to run `/bin/bash` 
bash-4.4#
staticfloat commented 6 years ago 
qemu-system-x86_64: virtio_bus_set_host_notifier: unable to init event notifier: Too many open files (-24)
qemu-system-x86_64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).

I have never seen this before; I'm not certain if this is qemu itself spitting out errors because your OSX kernel is running out of open files, or if it is passing along serial messages from Linux, complaining about too many open files within the virtualized environment. I'm going to bet the former. Two potential solutions:

You can update your limits via things like sysctl -w kern.maxfiles=<N>. Be sure to update your shell limits as well, with ulimit -S -n <N>. Read this for more detail.

bjarthur commented 6 years ago

changing kern.maxfiles[perproc] to 65535 and ulimit to 16384 (the max it would let me) got rid of the qemu errors emitted by runshell. but the exact same error as in the OP results when trying to execute build_tarballs.jl.

dpo commented 6 years ago

@bjarthur I had the same issue and resolved it with Docker: https://github.com/JuliaPackaging/BinaryBuilder.jl/issues/380#issuecomment-431236276.

bjarthur commented 6 years ago

docker works locally, thanks!

i tried to get it working on github via travis too, but have failed so far:

.ERROR: LoadError: IOError: could not spawn `docker inspect --type=image julia_binarybuilder_rootfs:v2018.9.18`: no such file or directory (ENOENT).

seems to me we need to get this working so that we can support OS X without having to build locally.

TeroFrondelius commented 5 years ago

I hit the same error message:

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.1.0 (2019-01-21)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using BinaryBuilder

julia> BinaryBuilder.run_wizard()
Would you like to resume the previous incomplete wizard run?
   Resume previous run
 > Start from scratch

        o      `.
       o*o      \'-_                 00000000: 01111111  .
         \\      \;"".     ,;.--*    00000001: 01000101  E
          \\     ,\''--.--'/         00000003: 01001100  L
          :=\--<' `""  _   |         00000003: 01000110  F
          ||\\     `" / ''--         00000004: 00000010  .
          `/_\\,-|    |              00000005: 00000001  .
              \\/     L
               \\ ,'   \
             _/ L'   `  \
            /  /    /   /          Julia Binzard
           /  /    |    \          JuliaPackaging/BinaryBuilder.jl
          "_''--_-''---__=;

Welcome to the BinaryBuilder wizard.
We'll get you set up in no time.

            # Step 1: Select your platforms

Make a platform selection
 > All supported architectures
   Specific operating system
   Specific architecture
   Custom

            # Step 2a: Obtain the source code

Please enter a URL (git repository or compressed archive) containing the source code to build:
> http://sourceforge.net/projects/gnuplot/files/gnuplot/5.2.7/gnuplot-5.2.7.tar.gz

[23:18:20] ######################################################################## 100.0%

Would you like to download additional sources?  [y/N]: 

            # Step 2b: Obtain binary dependencies (if any)

Do you require any (binary) dependencies?  [y/N]: 
            # Step 3: Build for Linux(:x86_64, libc=:glibc)

You will now be dropped into the cross-compilation environment.
Please compile the library. Your initial compilation target is Linux(:x86_64, libc=:glibc).
The $prefix environment variable contains the target directory.
Once you are done, exit by typing `exit` or `^D`

You have the following contents in your working directory:
  - gnuplot-5.2.7
Hints:
  - gnuplot-5.2.7/configure

    This file is a configure file generated by GNU Autoconf. 
     The recommended options for GNU Autoconf are `./configure --prefix=$prefix --host=$target`
    followed by `make` and `make install`. Since the prefix environment
    variable is set already, this will automatically perform the installation
    into the correct directory.

Setting parent shell interrupt to ^] (use ^C as usual, ^] to interrupt Julia)
qemu-system-x86_64: virtio_bus_set_host_notifier: unable to init event notifier: Too many open files (-24)
qemu-system-x86_64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
sandbox:$WORKSPACE/srcdir#

After running ./configureand makethey fail because the cnfigurecannot write to the file system.

staticfloat commented 4 years ago

We don't use QEMU anymore.