NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
18.19k stars 14.2k forks source link

dockerTools: buildImage contents recursively copies tzdata because of a rel. symlink `posix -> .` #30432

Closed siers closed 6 years ago

siers commented 7 years ago

Issue description

Prelude: When building a docker image, my code required files in /etc to function properly. (e.g. /etc/profile and /etc/services). As size is important to me, I unpacked and inspected the contents with ncdu, which presented the following issue to me:


When including the config.system.build.toplevel in the contents of the image, there are thousands of directories posix, which contain copies of themselves, due to the way dockerTools.buildImage creates the image, https://github.com/NixOS/nixpkgs/blob/82e955325012524c96a7d0fbb0901cc21bc5203e/pkgs/build-support/docker/default.nix#L330 together with the posix symlink in /etc/zoneinfo https://github.com/NixOS/nixpkgs/blob/82e955325012524c96a7d0fbb0901cc21bc5203e/pkgs/data/misc/tzdata/default.nix#L36 which gets set in environment.etc.zoneinfo here: https://github.com/NixOS/nixpkgs/blob/82e955325012524c96a7d0fbb0901cc21bc5203e/nixos/modules/config/timezone.nix#L49

Steps to reproduce (i.e. build an image)

default.nix: (it's both valid bash and nix at the same time)

#! @bash@/bin/bash -e

/* &> /dev/null || :

# 1. get all the paths stored in the layer.tar of the docker image
# 2. measure size of the /etc/zoneinfo stored in layer.tar
tar xOf @image@ "$(tar tf @image@ | grep -m1 layer)" |
    tee >(tar t > paths) |
    tar xO ./etc/zoneinfo |
    wc -c | awk '{print "etc/zoneinfo size in tar: "$1/1000/1000"M"}'

echo ...though the total disk usage is more like 200M
echo

# print *the number* of the recursively added paths and *a sample* of them
du -hs @image@
grep posix/posix/posix paths |
    tee >(wc -l > /dev/tty) |
    sed -n '4000,4004p'

exit # */

{ pkgs ? import <nixpkgs> {} }:

let
  toplevel = (import <nixpkgs/nixos/lib/eval-config.nix> {
    modules = [
      {
        imports = [ <nixpkgs/nixos/modules/profiles/minimal.nix> ];
        boot.isContainer = true;
      }
    ];
  }).config.system.build.toplevel;

  image = pkgs.dockerTools.buildImage {
    name = "testing";
    tag = "latest";
    contents = toplevel;
  };

in
  pkgs.substituteAll {
    src = ./default.nix;
    isExecutable = true;

    inherit image;
    inherit (pkgs) bash;
  }

its output:

% nix-build && ./result
/nix/store/bf4cxh1nsbmvx5pa5zfwhvp35pkl7h9g-default.nix
etc/zoneinfo size in tar: 68.5483M
...though the total [edit: unpacked] disk usage is more like 200M

121M    /nix/store/qsn5p54sw07k7qw7xdypczh33laqkqrr-docker-image-testing.tar.gz
./etc/zoneinfo/posix/posix/posix/posix/posix/posix/posix/posix/posix/W-SU
./etc/zoneinfo/posix/posix/posix/posix/posix/posix/posix/posix/posix/Universal
./etc/zoneinfo/posix/posix/posix/posix/posix/posix/posix/posix/posix/UTC
./etc/zoneinfo/posix/posix/posix/posix/posix/posix/posix/posix/posix/UCT
./etc/zoneinfo/posix/posix/posix/posix/posix/posix/posix/posix/posix/Turkey
44390

Technical details

nix-instantiate --eval '<nixpkgs>' -A lib.nixpkgsVersion                                        
"18.03pre117690.ab2cc75f78"

Last words

Not sure how solvable this is; I just wanted to make people aware of what I encountered.

siers commented 6 years ago

Rejoice, fellow programmers,

For this seems to be fixed on master via https://github.com/NixOS/nixpkgs/commit/9d2bd32b7970fceafd2b20cb97f676932a89c54a: tzdata: avoid symlink loop "./posix" -> "..".