Simplify mount using losetup directly

[konsumer]

related to #18

This will use --show -fP flags to get info about the loop, directly, so we don't need to manage sizes & start/ends.

[konsumer]

Testing on mac, it seems to not quite be assigning the correct devices in docker-space, for some reason. I'm pretty sure this is how I did it on a linux host, but I need to do some testing when I have a chance.

[konsumer]

Looks like it relates to this and seems to work fine if I do --privileged -v /dev:/dev

[konsumer]

Looks like CI is failing for the issue I found, above, which is for the -P flag to work, losetup needs access to /dev as well as --privileged. Again, locally, I fixed with -v /dev:/dev, so we should probly decide if we want that (which sketches me out a lil, even though it makes total sense) in trade for simpler losetup. If that trade is ok, I'll update the tests to volume-mount /dev.

[konsumer]

For my own project, I started using my own script (with your docker image as base) and realized that I might be able to do all the privileged stuff outside of the container. I will make a lil proof-of-concept that mounts the loopbacks outside of the docker container.

[konsumer]

I came up with this Makefile:

# this uses qemu + make to manage your pi image

APP_NAME = demo
CWD = $(shell pwd)

# run bash in context of pi image
emu: loopback
    @echo "Chrooting to pi image-root"
    @docker run -it --rm \
        -v ${CWD}/images:/usr/rpi/images \
        ryankurte/docker-rpi-emu \
        chroot /usr/rpi/images/diskmount/ bash
    @sudo umount images/diskmount/boot/
    @sudo umount images/diskmount/

# mount image as loopback on host
loopback: images/$(APP_NAME).img
    @echo "Creating a loopback filesystem"
    $(eval LOOP = $(shell sudo losetup --show -fP "images/${APP_NAME}.img"))
    @mkdir -p images/diskmount
    @sudo mount "$(LOOP)p2" images/diskmount/
    @sudo mount "$(LOOP)p1" images/diskmount/boot/

# download a zip of current raspbian-lite
images/raspbian_lite.zip:
    @echo "Getting zip of latest raspbian_lite."
    @mkdir -p images
    @curl -L https://downloads.raspberrypi.org/raspbian_lite_latest -o images/raspbian_lite.zip

# unzip image to images/$(IMAGE)
images/$(APP_NAME).img: images/raspbian_lite.zip
    @echo "Extracting disk image for raspbian_lite."
    @unzip -p images/raspbian_lite.zip '*.img' > images/$(APP_NAME).img

You can run make and get a bash prompt inside the image that you can edit however you like. I get preload errors (which cane be fixed with echo "" > /etc/ld.so.preload), but it seems to work fine to install software and stuff, directly in the image. Much faster than emulating full qemu boot!

It has the added bonus that you can easily edit the files on the host in a nice editor:

[ryankurte]

hey thanks for all your work looking at this!

we should probly decide if we want that (which sketches me out a lil, even though it makes total sense) in trade for simpler losetup. If that trade is ok, I'll update the tests to volume-mount /dev.

i think i prefer the nastier setup to avoid having to mount everything into /dev, but maybe the better approach could be added as documentation for anyone else who comes along / with different requirements?

You can run make and get a bash prompt inside the image that you can edit however you like Much faster than emulating full qemu boot!

hey that’s neat! could be a useful additional command perhaps?

[konsumer]

Yeh, I did some testing on OSX, and it doesn't seem to be able to do the loopback stuff directly, so it's not ideal, and it's also not ideal to expose dev to docker.

hey that’s neat! could be a useful additional command perhaps?

I think we can get the same effect by passing through the inside-docker mount-point with volume to host, which is basically what I am doing in that above Makefile, but I am mounting the image outside (which I discovered is not great for OSX.) I think your example run-emu make-target is similar, just needs the in-docker image-mount dir to be volume-mounted to a host dir.

For now, I think the other solution we talked about in #18 (using the extra sizelimit param) seems to be a better direction than this PR. I could live with passing /dev or linux-only (so I can do losetup on host) in my own stuff, but I totally see how that sucks for a general no-install pi disk-builder/emulator.

[konsumer]

Another idea I had is to wrap a mount-trap around the commands I want to run in chroot, and tie it all into the method we came up with for better loop-mounting.

I found it really useful to trap the exit of my scripts to unmount the disk, so if there is an error, or an exit of any kind it will still cleanup & unmount loops, and I don't have to do it at the end of my script:

# handle mounting/unmounting of loopback
set -e
function cleanup {
  sudo umount -f "${ROOT}/boot/"
  sudo umount -f "${ROOT}"
  rm -rf "${ROOT}"
}
trap cleanup EXIT
# setup loop and mount it
# do my stuff in chroot

but for every script I have to copy/paste this + all the loop setup stuff. If the script was a chroot + image-manager all-in-one, I think it could be more general, and keep loop-stuff atomic:

# trap from above
# setup loop & mount it
# use arguments as chroot arguments

This way you could do this:

@docker run -it --rm -privilaged \
        -v $(PWD)/images:/usr/rpi/images/ \
        ryankurte/docker-rpi-emu \
            /usr/rpi/chroot.sh /usr/rpi/images/myimage.img /usr/rpi/images/mount  bash

I think if we wrapped everything in this chroot script, it would greatly simplify the scripts and make it work for lots more use-cases. Instead of referencing a script to do the mounting, and then managing the unmount, or putting the trap in several scripts, we could just pass it the command directly.

Lemme know if you are interested in this style, and I will make another PR. For now, I will play around with it on my project.

[ryankurte]

Another idea I had is to wrap a mount-trap around the commands I want to run in chroot

ooh yeah this seems great to me!

and tie it all into the method we came up with for better loop-mounting.

this method? i think your conclusion about exposing dev and not working on osx precludes the nicer approach?

[konsumer]

ooh yeah this seems great to me!

I will make a PR once I have it worked out. I was getting errors running out of memory when I tried to cp -R or mkdir -p, but the non-recursive versions of those commands work fine. I think it has something to do with how I was using the image-loop, so I will keep playing with it. An exit-trap is super-cool for these kinda things because you don't have to manage the loop once it's mounted, and when you are done or an error happens, it just cleans up itself. I sure hope that isn't causing the memory error, somehow.

this method?

Yep.

i think your conclusion about exposing dev and not working on osx precludes the nicer approach?

Totally agreed. Feel free to close this PR.