search

JuliaPackaging / BinaryBuilder.jl

Binary Dependency Builder for Julia
https://binarybuilder.org
Other
390 stars 99 forks source link

No version of BinaryBuilder usable? #380

Closed dpo closed 4 years ago

dpo commented 5 years ago

I tried to Pkg.add("BinaryBuilder") on Julia 0.7 and 1.0.1, but both fail because of https://github.com/JuliaLang/PkgDev.jl/issues/143.

I fell back on Julia 0.6.4, where Pkg.add("BinaryBuilder") succeeds, but when I try to run make on Linux 64bit, building a static library fails with the message

ar: No file descriptors available

Google didn't help.

I tried master on Julia 0.7 and 1.0.1. If I set BINARYBUILDER_RUNNER=docker, BinaryBuilder bails with the message

IOError: could not spawn `docker inspect --type=image julia_binarybuilder_rootfs:v2018.9.18`: no such file or directory (ENOENT)

when it tries to drop me to a command prompt.

Using qemu on 0.7 or 1.0.1 gives me the message

qemu-system-x86_64: virtio_bus_set_host_notifier: unable to init event notifier: Too many open files (-24)
qemu-system-x86_64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).

It does drop me to a command prompt, but running make fails with

Assembler messages:
Fatal error: can't create somefile.o: Too many open files

I'm on macOS 10.13.6.

Is there anything else I can try?

dpo commented 5 years ago

I ran into https://github.com/JuliaPackaging/BinaryBuilder.jl/issues/171. Setting ulimit -n 10000 allowed me to run the wizard with Julia07 and the qemu runner. I managed to build for several platforms, but trying to run build_tarballs.jl locally fails with an error similar to https://github.com/JuliaPackaging/BinaryBuilder.jl/issues/377. The log file is as follows:

`/Users/dpo/.julia/packages/BinaryBuilder/Bj1V9/deps/qemu/bin/qemu-system-x86_64 -kernel /Users/dpo/.julia/packages/BinaryBuilder/Bj1V9/deps/qemu/bzImage -m 12288M -cpu host -smp 2 -M accel=hvf -nographic -nodefaults -rtc base=utc,driftfix=slew -device virtio-serial -chardev stdio,id=charconsole0 -device virtconsole,chardev=charconsole0,id=console0 -device virtserialport,chardev=charcomm0,id=comm0 -device virtio-net-pci,netdev=networking -netdev user,id=networking -drive if=virtio,file=/Users/dpo/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/Rootfs.v2018.9.18.x86_64-linux-gnu.squashfs,format=raw -fsdev local,security_model=none,id=fsdev0,path=/Users/dpo/dev/julia/JSO/ThinASLBuilder.jl/build/i686-linux-gnu/y9CWRvJZ -device virtio-9p-pci,id=fs0,fsdev=fsdev0,mount_tag=workspace0 -fsdev local,security_model=none,id=fsdev1,path=/Users/dpo/dev/julia/JSO/ThinASLBuilder.jl/build/i686-linux-gnu/y9CWRvJZ/metadir -device virtio-9p-pci,id=fs1,fsdev=fsdev1,mount_tag=workspace1 -drive if=virtio,file=/Users/dpo/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/BaseCompilerShard-i686-linux-gnu.v2018.10.10.x86_64-linux-gnu.squashfs,format=raw -drive if=virtio,file=/Users/dpo/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/GCC-i686-linux-gnu.v4.8.5.x86_64-linux-gnu.squashfs,format=raw -drive if=virtio,file=/Users/dpo/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/LLVM.v6.0.1-0.x86_64-linux-gnu.squashfs,format=raw -drive if=virtio,file=/Users/dpo/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/BaseCompilerShard-x86_64-linux-gnu.v2018.10.10.x86_64-linux-gnu.squashfs,format=raw -drive if=virtio,file=/Users/dpo/.julia/packages/BinaryBuilder/Bj1V9/deps/downloads/GCC-x86_64-linux-gnu.v4.8.5.x86_64-linux-gnu.squashfs,format=raw -device virtio-serial -chardev socket,path=/tmp/tmpHT6ZbY/qemu_comm.socket,server,nowait,id=charcomm0 -append 'quiet console=hvc0 root=/dev/vda rootflags=ro rootfstype=squashfs noinitrd init=/sandbox'`
rsync: pipe: Address family not supported by protocol (97)
rsync error: error in IPC code (code 14) at pipe.c(122) [sender=3.1.3]
reboot: Power down
dpo commented 5 years ago

Resolved the issue with brew cask install docker, firing up docker.app (which apparently starts the docker daemon), and running BINARYBUILDER_RUNNER=docker julia build_tarballs.jl.

staticfloat commented 5 years ago

Wow, this sounds pretty annoying. I'm sorry you ran into so many problems.

I tried to Pkg.add("BinaryBuilder") on Julia 0.7 and 1.0.1, but both fail because of JuliaLang/PkgDev.jl#143.

I think what happened here is that you had an old METADATA, because we dropped the requirement for PkgDev a while back. (The first release without it was 18 days ago, but the master version of BB dropped it back in June.

If I set BINARYBUILDER_RUNNER=docker, BinaryBuilder bails with the message:

Unfortunately, I need more of the error message than what you posted. Looking at your later message though, it sounds like perhaps your docker install was broken/too old perhaps?

rsync: pipe: Address family not supported by protocol (97) rsync error: error in IPC code (code 14) at pipe.c(122) [sender=3.1.3] reboot: Power down

Oooh, this is new and exciting. I will have to try and reproduce locally.

IanButterworth commented 5 years ago

I had the Too many open files issue too, which required the following to fix that on yosemite:

$ echo kern.maxfiles=65536 | sudo tee -a /etc/sysctl.conf
$ echo kern.maxfilesperproc=65536 | sudo tee -a /etc/sysctl.conf
$ sudo sysctl -w kern.maxfiles=65536
$ sudo sysctl -w kern.maxfilesperproc=65536
$ ulimit -n 65536 65536

And I've arrived at the same

rsync: pipe: Address family not supported by protocol (97)
rsync error: error in IPC code (code 14) at pipe.c(122) [sender=3.1.3]
reboot: Power down

The docker method works well, but seems to run a fair bit slower (but that might be more to do with my docker setup)

jonathanBieler commented 5 years ago

I've got the same problem when trying to build SDL2, setting ulimit -n 65536 65536 before make doesn't help.

staticfloat commented 4 years ago

We don't use QEMU anymore, so we will be releasing a docker-only version presently.