Closed TheChymera closed 6 years ago
Chrooting into the file and starting the emerge process, it really does look like it's filling up:
newbuilder / # df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 10M 0 10M 0% /dev
tmpfs 7.7G 0 7.7G 0% /dev/shm
none 6.0G 6.0G 0 100% /var/tmp/portage
none 1.0G 0 1.0G 0% /tmp
I looked into the occurences of 6G
, and it appears that this is the only file where such a size is specified, editing it (to 16G
, see below), however, did not fix the problem:
newbuilder /usr/share/gebuilder # cat scripts/common/mount_basic.sh
#!/bin/bash
for file in dev dev/pts proc sys var/tmp/portage tmp
do
on_exit "umount -R \"${ROOT}/$file\""
done
pushd "${ROOT}"
debug "recursively bind-mount /dev to dev/"
mount --rbind /dev/ dev
debug "mount proc/"
mount -t proc none proc
debug "mount sys/"
mount -t sysfs none sys
debug "mount devpts with gid=5 for glibc[-suid] at /dev/pts/"
mount -t devpts -o gid=5 none dev/pts/
debug "mount tmpfd on /var/tmp/portage"
mkdir -p var/tmp/portage
mount -t tmpfs -o size=16G none var/tmp/portage
debug "mounting tmpfs on /tmp/"
mount -t tmpfs -o size=1G none tmp
popd
Yeah, maybe we should just stop mounting var/tmp/portage as tmpfs in the first place. This might be detrimental when building openstack images, because we build some smaller software in a tight space-constraint, but should not have a large impact otherwise.
Usually, packages (when sufficiently resource consuming) check whether they will exceed the capacity of /var/tmp/portage/ via check-reqs.eclass, but sci-biology/mouse-brain-atlases
doesn't seem to do that.
I'll first try to fix it package-side, then. It's my package.
Defining the respective variables for the package does not seem to have done the trick:
# Copyright 1999-2018 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
EAPI=6
inherit check-reqs
DESCRIPTION="A collection of mouse brain atlases in NIfTI format"
HOMEPAGE="http://imaging.org.au/AMBMC/Model"
SRC_URI="http://chymera.eu/pkgdata/${P}.zip"
LICENSE="fairuse"
SLOT="0"
KEYWORDS="~amd64 ~x86 ~amd64-linux ~x86-linux"
IUSE=""
RDEPEND=""
DEPEND=""
CHECKREQS_DISK_BUILD=4G
CHECKREQS_DISK_USR=4G
CHECKREQS_DISK_VAR=8G
src_install() {
insinto "/usr/share/${PN}"
doins *
}
Results in (note how it does check for space a the beginning):
[32;01m * [39;49;00mPackage: sci-biology/mouse-brain-atlases-0.1.20180623
[32;01m * [39;49;00mRepository: science
[32;01m * [39;49;00mMaintainer: horea.christ@gmail.com sci@gentoo.org
[32;01m * [39;49;00mUSE: abi_x86_64 amd64 elibc_glibc kernel_linux userland_GNU
[32;01m * [39;49;00mFEATURES: preserve-libs sandbox userpriv usersandbox
[32;01m*[0m Checking for at least 4 GiB disk space at "/var/tmp/portage/sci-biology/mouse-brain-atlases-0.1.20180623/temp" ...
[A[218C [34;01m[ [32;01mok[34;01m ][0m
[32;01m*[0m Checking for at least 4 GiB disk space at "//usr" ...
[A[218C [34;01m[ [32;01mok[34;01m ][0m
[32;01m*[0m Checking for at least 8 GiB disk space at "//var" ...
[A[218C [34;01m[ [32;01mok[34;01m ][0m
>>> Unpacking source...
>>> Unpacking mouse-brain-atlases-0.1.20180623.zip to /var/tmp/portage/sci-biology/mouse-brain-atlases-0.1.20180623/work
>>> Source unpacked in /var/tmp/portage/sci-biology/mouse-brain-atlases-0.1.20180623/work
>>> Preparing source in /var/tmp/portage/sci-biology/mouse-brain-atlases-0.1.20180623/work/mouse-brain-atlases-0.1.20180623 ...
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/sci-biology/mouse-brain-atlases-0.1.20180623/work/mouse-brain-atlases-0.1.20180623 ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/sci-biology/mouse-brain-atlases-0.1.20180623/work/mouse-brain-atlases-0.1.20180623 ...
>>> Source compiled.
>>> Test phase [not enabled]: sci-biology/mouse-brain-atlases-0.1.20180623
>>> Install mouse-brain-atlases-0.1.20180623 into /var/tmp/portage/sci-biology/mouse-brain-atlases-0.1.20180623/image/ category sci-biology
ERROR:root:Failed to copy file: _parsed_options=Namespace(group=-1, mode=420, owner=-1, preserve_timestamps=False), source=b'dsurqec_40micron_mask.nii', dest_dir=b'/var/tmp/portage/sci-biology/mouse-brain-atlases-0.1.20180623/image/usr/share/mouse-brain-atlases'
Traceback (most recent call last):
File "/usr/lib/portage/python3.5/doins.py", line 209, in run
copyfile(source, dest)
File "/usr/lib64/python3.5/site-packages/portage/util/file_copy/__init__.py", line 30, in _optimized_copyfile
_file_copy(src_file.fileno(), dst_file.fileno())
OSError: [Errno 28] No space left on device
ERROR:root:Failed to copy file: _parsed_options=Namespace(group=-1, mode=420, owner=-1, preserve_timestamps=False), source=b'dsurqec_40micron_masked.nii', dest_dir=b'/var/tmp/portage/sci-biology/mouse-brain-atlases-0.1.20180623/image/usr/share/mouse-brain-atlases'
Traceback (most recent call last):
File "/usr/lib/portage/python3.5/doins.py", line 209, in run
copyfile(source, dest)
File "/usr/lib64/python3.5/site-packages/portage/util/file_copy/__init__.py", line 30, in _optimized_copyfile
_file_copy(src_file.fileno(), dst_file.fileno())
OSError: [Errno 28] No space left on device
ERROR:root:Failed to copy file: _parsed_options=Namespace(group=-1, mode=420, owner=-1, preserve_timestamps=False), source=b'lambmc_15micron.nii', dest_dir=b'/var/tmp/portage/sci-biology/mouse-brain-atlases-0.1.20180623/image/usr/share/mouse-brain-atlases'
Traceback (most recent call last):
File "/usr/lib/portage/python3.5/doins.py", line 209, in run
copyfile(source, dest)
File "/usr/lib64/python3.5/site-packages/portage/util/file_copy/__init__.py", line 30, in _optimized_copyfile
_file_copy(src_file.fileno(), dst_file.fileno())
OSError: [Errno 28] No space left on device
ERROR:root:Failed to copy file: _parsed_options=Namespace(group=-1, mode=420, owner=-1, preserve_timestamps=False), source=b'lambmc_180micron.nii', dest_dir=b'/var/tmp/portage/sci-biology/mouse-brain-atlases-0.1.20180623/image/usr/share/mouse-brain-atlases'
Traceback (most recent call last):
File "/usr/lib/portage/python3.5/doins.py", line 209, in run
copyfile(source, dest)
File "/usr/lib64/python3.5/site-packages/portage/util/file_copy/__init__.py", line 30, in _optimized_copyfile
_file_copy(src_file.fileno(), dst_file.fileno())
OSError: [Errno 28] No space left on device
ERROR:root:Failed to copy file: _parsed_options=Names
Interesting. Not sure why the check-reqs doesn't do its job here...
Anyways, checking the resources only results in an earlier build error, and does not really address the problem. We either need to enlargen the tmpfs or just not mount it at all.
you want to make the choice yourself or should we discuss pro/cons?
Fixed as per https://github.com/IBT-FMI/gebuilder/commit/2a6d939d08b650aa7144c47af59bfc2f245fce0a --- reopen if you want to discuss this further.
I keep getting a lot of these errors:
Strangely enough (and this is with
#DELETE_ON_FAIL=1
)df -h
gives me:@Doeme ideas?