Docker Hub images, Void Dockerfile all unusable

[kwshi]

I've been playing with Darch the past few days and trying to get it to work on one of my computers, but I've been unable to get any sort of usable system with Darch so far. In particular, none of the existing pre-built images on Docker Hub (godarch/{void,void-musl,arch,ubuntu,debian}) successfully stage, and while your personal recipes do stage successfully, they result in either an out-of-memory kernel panic or a blank screen for me.

I've included a list of all the different things I've tried so far and what I've found at the bottom. Let me just start by saying thank you for doing this; it's a very cool, attractive project that feels like exactly what I'm looking for (having come from NixOS, I enjoy having reproducible build steps but did not enjoy all the extra Nix layers and the complexity it adds); and thank you for the effort you put into making this accessible to other folks! I also want to say that I don't intend to put any stress/pressure on you to maintain this project and fix all my problems, and I totally understand if you're busy with other things and don't have the time for a project you consider "done"!

So all I ask is this: it would be fantastic if you can help give me some coarse descriptions/point me to some resources about what Linux boot-loading magic (in your words, "special scripts that take into account the kernel, initramfs and root file system") involves and how those steps fit into the Darch staging + booting pipeline. I would love to help out with fixing/updating the documentation and these Dockerfiles (as well as add even more distributions or contribute to Darch V1.0) to make it even more accessible, but I currently have minimal clue how any of this works, so if you can help me get started I'd gladly contribute back!

---

List of unsuccessful attempts:

• darch images pull godarch/arch && darch stage upload godarch/arch: pulls successfully; staging installs some packages, appears to successfully produce /extract/rootfs.squash, then fails with "cp: cannot stat '/boot/vmlinuz-linux': No such file or directory". Nothing is successfully staged (per darch stage list). I don't know what's going on here.
• darch images pull godarch/debian && darch stage upload godarch/debian: pulls successfully; staging tries to install some packages but immediately fails with a "404 Not Found" / "E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?" message. My guess is that the Docker Hub image here is out of date and has an obsolete repo URL cached; or, it is trying to install a specific version of squashfs-tools (the package it fails to download) that is no longer in the apt repos.
• darch images pull godarch/ubuntu && darch stage upload godarch/ubuntu: same as debian.
• darch images pull godarch/void && darch stage upload godarch/void: pulls fine; staging tries to install some package but fails with "Not Found" as well. The repo URL printed (http://alpha.us.repo.voidlinux.org/current) seems correct/up-to-date to me, so my best guess is that it's trying to install a specific version that either no longer exists or has been renamed.
• darch images pull pauldotknopf/darch-ubuntu-desktop && darch stage upload pauldotknopf/darch-ubuntu-base: pulls and stages successfully, and an entry appears in darch stage list as well as GRUB on reboot. However, booting quickly results in an out-of-memory error (old laptop only has 2GB RAM, maybe that's relevant) and thereby causes a kernel panic.
• darch images pull pauldotknopf/darch-ubuntu-base && darch stage upload pauldotknopf/darch-ubuntu-base: pulls and stages fine, and even seems to boot successfully for the most part--until right when I think a login prompt is about the pop up, the entire screen goes black (but still on), with a single cursor blinking in the top-left; nothing else. Ctrl-Alt-F# doesn't do anything, so I doubt it's related to a broken desktop config, especially since I'm pretty sure base doesn't even have a desktop installed. So I don't know what's wrong, but one of your last commits on pauldotknopf/darch-recipes mentions something about Ubuntu cosmic missing graphical drivers, so maybe it has something to do with that?

• Manually clone the godarch/distributions repository and try to rebuild the images from Dockerfile:

  • Void: the build script tries to fetch the 20181111 rootfs archive from the "current" path under the Void mirror, but the actual current version is 20191109, so the build script merely fetches a 404 HTML page and names it root.tar.xz. Two fixes were tried:

  • Changing the build script URL to the correct 20181111 path downloads the correct tar, but building fails on

    Step 4/17 : RUN xbps-install -y -S -R http://alpha.us.repo.voidlinux.org/current
    ---> Running in d26d89a29227
    [*] Updating `http://alpha.us.repo.voidlinux.org/current/x86_64-repodata' ...
    x86_64-repodata: [1701KB 0%] 195MB/s ETA: 00m00s
    x86_64-repodata: 1701KB [avg rate: 81GB/s]
    The command '/bin/sh -c xbps-install -y -S -R http://alpha.us.repo.voidlinux.org/current' returned a non-zero code: 95

    Modifying the Dockerfile to run xbps-install with the -d flag (verbose/"debug" logging) reveals that the error arises from a change in the Void repodata format no longer compatible with releases earlier than June 2019, per https://docs.voidlinux.org/xbps/troubleshooting/common-issues.html#repodata-errors.

  • Changing the build script URL to download the latest 20191109 version downloads successfully and makes it through most of the build but fails on

    Step 16/17 : RUN dracut --force /boot/initramfs-$(ls /usr/lib/modules).img $(ls /usr/lib/modules)
    ---> Running in 716c28136f03
    ls: cannot access '/usr/lib/modules': No such file or directory
    ls: cannot access '/usr/lib/modules': No such file or directory
    /bin/sh: 1: dracut: not found
    The command '/bin/sh -c dracut --force /boot/initramfs-$(ls /usr/lib/modules).img $(ls /usr/lib/modules)' returned a non-zero code: 127

    which I don't really understand (I haven't ever used dracut manually before and don't really understand the Linux initramfs boot process), but it suggests to me that 20191109 has introduced some differences in the rootfs since 20181111 such that /usr/lib/modules and dracut no longer exist. I have no idea how to fix this.

  • Ubuntu: I ran the build script, and the tar -czpf ../root.x86_64.tar.gz * first spit out some error/warning messages

  tar: proc/sys/net/ipv4/route/flush: Cannot open: Permission denied
  tar: proc/sys/net/ipv6/route/flush: Cannot open: Permission denied
  tar: proc/sys/vm/compact_memory: Cannot open: Permission denied
  tar: proc/sys/vm/drop_caches: Cannot open: Permission denied

  and has since been running silently for the past ~2 hours. Maybe that's to be expected, but in any case, I can't tell whether it's actually working correctly, even doing anything, or just frozen. (htop says gzip is taking up 126% of my CPU, so I can only guess that it's at least doing something.) In any case, I'm not sure what to do other than wait for I-don't-know-how-long-I-should.

  • Arch: per the distributions page on the current documentation, the Arch image is missing an initramfs and needs to have a kernel installed first. As such, I added the line

    RUN pacman -S --noconfirm lvm2 linux

    to the Dockerfile, and subsequently the build logs seem to indicate that mkinitcpio was run during the build successfully. Subsequently, I pushed (kwshi/darch-arch), pulled onto my Darch machine, and ran darch stage upload kwshi/darch-arch. Most of the staging seemed to be successful, save for a cryptic error message at the end:

    darch: rename /var/lib/darch/stage/tmp/inzjzzlvc78waw3e35cx1uwbz986262600/hooks /var/lib/darch/stage/live/xh7snhd7hy1a4xai099kqmepe/hooks: no such file or directory

    Nevertheless, an entry appeared in darch stage list, and an entry appeared in GRUB after darch stage sync-bootloader (or was it bootloader-sync? I forget.). However, when I tried to boot it, I received the error message

    ERROR: device '' not found.  Skipping fsck.
    Darch mount handler
    Trying to mount Darch source (UUID=d4605d2a-8d4d-.......omitted....)
    :: running late hook [lvm2]
    :: running cleanup hook [lvm2]
    :: running cleanup hook [udev]
    ERROR: Root device mounted successfully, but /sbin/init does not exist.
    Bailing out, you are on your own.  Good luck.

    It does then open a shell that I can play with, so I seem to at least get a functioning/usable Arch "core", but clearly this is not a fully successful boot, so I must be missing something. But the documentation does not go into much detail on how exactly I was supposed to perform the "install kernel and mkinitcpio" steps, so I'm not sure whether I did it right.

[kwshi]

Update: following the "test drive" instructions to pull and run pauldotknopf/darch-arch-base doesn't work either. Pulling fails with a

INFO[0057] apply failure, attempting cleanup        error="failed to extract layer sha256:81e2ef41a8ebca.....: mkparent /var/lib/darch/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/35/fs/usr/lib/systemd/system-generators: not a directory: unknown" key="extract-49363929-2dho sha256:32ac130aa52dff51193......."
darch: failed to unpack image on snapshotter overlayfs: failed to extract layer ........... not a directory: unknown

[kwshi]

Oh well--I have since given up on figuring Darch out, so I'm going to close this issue.

For those still interested in a solution/doing this for their own distro of choice on a UEFI system: I've managed to achieve essentially the same setup as Darch "from scratch", i.e. via manually building a container & exporting the contents to a squashfs live image. The high-level steps are as follows:

1. Write some Dockerfiles that build an OS image with everything you want installed & configured.
   • It's important that this image also includes "system-level" software not normally found in container images, e.g. bootloaders, kernels, etc.
   • It's also important to configure whatever builds your initramfs (e.g., dracut for Void), to support booting from a live image, by adding to /etc/dracut.conf.d/ a file specifying add_dracutmodules+='dmsquash-live' before dracut is run/installed.
2. docker run the container to do some last-minute configs, e.g. setting user passwords.
3. docker export the container to a tar archive; unpack the output to a folder in order to modify the filesystem's etc/hostname (because of how containers work, you can't set hostnames within the container--hence this last-minute hack).
4. Re-pack the folder contents into a tar archive, with something like tar --xform 's:^\./::' -cf live.tar -C <root-path> ..
5. Use tar2sqfs to pack the archive into a SquashFS image.
6. Depending on how your bootloader works, add a bootloader entry to boot from the initramfs & vmlinuz from your extracted filesystem (you'll probably need to copy those into a folder under /boot or whatever your EFI system partition is).

Those instructions are not very detailed and will still require you to do some digging into how to set each part up, but hopefully it gives future hobbyists a solid starting point. I have a repo with the Containerfiles & dotfiles I use to implement this setup; if anyone comes across this and wants me to explain more about how to do this, feel free to open an issue on that repo & I'll be happy to help!