Container build still crashes on apt-get issue

[tailhook]

The crashed process:

ERROR:vagga::builder: Error building container "py": step Install(["ca-certificates", "python3.6=3.6.2-1~16.04.york0"]) failed: error running <Command "/usr/bin/apt-get" "install" "-y" "ca-certificates" "python3.6=3.6.2-1~16.04.york0"; environ[6]; chroot="/vagga/root"; work-dir="/work"> killed by signal SIGBUS[7]

The waiting process:

Get:15 http://ua.archive.ubuntu.com xenial-security/restricted Translation-en [2428 B]
Fetched 3653 kB in 0s (4937 kB/s)                                
Reading package lists... Done
WARN:vagga::file_util: Another build process is executing `apt-get install` command against the same apt cache. Waiting ...
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  locales

Probably the problem is because the first one has PPAs enabled and they might be cleaned on clean install of the second container (as you might note, it's installing the locales package, which means it's the first Ubuntu step)

Probably our lock doesn't help with this :(

/cc @anti-social

[anti-social]

Could we use overlayfs for apt cache? I have a patch and it works fine at first sight.

Structure of mounted cache:

lowerdir=/vagga/cache/apt-cache
upperdir=/vagga/container/apt-overlay/upper
workdir=/vagga/container/apt-overlay/work

Possibly it is worth to make the feature optional as it may require extended attributes for opaque directories (although in our case apt-get should not remove anything from its cache). For example the option could be named apt-cache-use-overlayfs.

[anti-social]

Also maybe it would be better to place upperdir and workdir in /vagga/cache to guarantee they are at the same mountpoint with lowerdir. That will give us opportunity to just rename new cache entries when copying them back.

As a drawback we need somehow to clean those directories if build fails.

[tailhook]

Yes sounds good (I think overlayfs didn't work in unprivileged mode in 2017 when issue was created :) ). And I think it's fine to put upper dir in container's directory (somewhere below root). I think it's common enough to put cache and container directories into the same volume (and by default vagga puts everything into the project dir anyways).

Possibly it is worth to make the feature optional

Can we detect whether this works, rather than requiring user to set the option manually? We can probably try mount and write (delete?) file. So we check both: whether unprivileged unionfs works and extended attributes.