mcopy fails with "Disk full"

erichiggins commented 5 years ago

Background

I'm using Buildroot to create a custom image for a Raspberry Pi Zero W. I'd like to add an extra VFAT partition that can be mounted on the root filesystem. I've run into a problem trying to do this and I suspect that it's with genimage.

Any help would be greatly appreciated!

Changes to .cfg

The following is the only change I made to the board/raspberrypi/genimage-raspberrypi0w.cfg file. I intentionally have not yet defined the partition on image sdcard.img so that I can narrow down the source of the failure.

Removing these lines allows the make command to complete successfully.

image eric.vfat {
  vfat {}
  size = 1M
}

`make` error: Disk full

The following shows the relevant output from running the make command. Note the Disk full error after the mcopy command.

INFO: vfat(eric.vfat): cmd: "dd if=/dev/zero of="/home/vagrant/buildroot-2019.08/output/images/eric.vfat" seek=1048576 count=0 bs=1 2>/dev/null" (stderr):
INFO: vfat(eric.vfat): cmd: "mkdosfs  '/home/vagrant/buildroot-2019.08/output/images/eric.vfat'" (stderr):
INFO: vfat(eric.vfat): cmd: "MTOOLS_SKIP_CHECK=1 mcopy -bsp -i '/home/vagrant/buildroot-2019.08/output/images/eric.vfat' '/home/vagrant/buildroot-2019.08/output/build/genimage.tmp/root'/* ::" (stderr):
Disk full
INFO: vfat(eric.vfat): cmd: "rm -f "/home/vagrant/buildroot-2019.08/output/images/eric.vfat"" (stderr):
ERROR: vfat(eric.vfat): failed to generate eric.vfat
Makefile:812: recipe for target 'target-post-image' failed
make[1]: *** [target-post-image] Error 1

michaelolbrich commented 5 years ago

Do you create the rootfs with the same genimage config? By default, any filesystem will be filled with the content from the specified rootpath. This is probably more than 1M in your case.

If the fat partition is supposed to be empty then you need to specify an existing empty directory within the rootpath as mountpoint. Maybe /mnt.

So mountpoint = /mnt will move ${rootpath}/mnt aside and fill your fat partition with the content (in this case nothing. So basically you can split the specified rootpath into multiple partitions that are then mounted in the specified hierarchy at runtime to get the same file tree.

erichiggins commented 5 years ago

Do you create the rootfs with the same genimage config? By default, any filesystem will be filled with the content from the specified rootpath. This is probably more than 1M in your case.

Yes, though as I mentioned, I intentionally did not add the vfat image as a partition of the image w/ rootfs. I've been trying to distill to find the root-cause of the error. For clarity, I've added the full file contents below.

If the fat partition is supposed to be empty then you need to specify an existing empty directory within the rootpath as mountpoint. Maybe /mnt.

It's not supposed to be empty in the end, but in trying to reduce noise while I debug, it is for now. I have also tried using mountpoint and files within the rootfs-overlay and I have not had any success.

The end goal is to provide a VFAT directory that will contain the code run by the Pi, so that some developers on the team can simply mount the SD card on their windows computers to make changes more easily. (SSH & CLI were options I suggested, but weren't feasible for various reasons)

So mountpoint = /mnt will move ${rootpath}/mnt aside and fill your fat partition with the content (in this case nothing. So basically you can split the specified rootpath into multiple partitions that are then mounted in the specified hierarchy at runtime to get the same file tree.

Going from memory, when I tried to use mountpoint, genimage would throw another error about a file/path not existing in the output/build/ directory, though in checking after the failure, it was clearly there. Trying it now (just adding mountpoint = "/eric" to the vfat image as you suggest, I get the following error:

INFO: cmd: "mv "/home/vagrant/buildroot-2019.08/output/build/genimage.tmp/root/eric" "/home/vagrant/buildroot-2019.08/output/build/genimage.tmp/eric"" (stderr):
mv: cannot stat '/home/vagrant/buildroot-2019.08/output/build/genimage.tmp/root/eric': No such file or directory
Makefile:812: recipe for target 'target-post-image' failed

So then I created an empty directory: mkdir board/raspberrypi/raspberrypi0w/rootfs-overlay/eric Then got this error:

INFO: vfat(eric.vfat): cmd: "MTOOLS_SKIP_CHECK=1 mcopy -bsp -i '/home/vagrant/buildroot-2019.08/output/images/eric.vfat' '/home/vagrant/buildroot-2019.08/output/build/genimage.tmp/eric'/* ::" (stderr):
/home/vagrant/buildroot-2019.08/output/build/genimage.tmp/eric/*: No such file or directory
INFO: vfat(eric.vfat): cmd: "rm -f "/home/vagrant/buildroot-2019.08/output/images/eric.vfat"" (stderr):
ERROR: vfat(eric.vfat): failed to generate eric.vfat
Makefile:812: recipe for target 'target-post-image' failed

Which makes me think it has an issue with the directory being empty for some reason? Then I added an empty file: touch board/raspberrypi/raspberrypi0w/rootfs-overlay/eric/foobar.txt

This worked!

There's surprisingly little documentation about how to do something like this and even the folks in the buildroot IRC support channel haven't had much to say.

More details, for clarity:

Buildroot version: buildroot-2019.08
Environment: Vagrant/Ubuntu (as provided by Buildroot, but modified for an additional 8G of disk space), running on top of Ubuntu 18 LTS

Full contents of my `board/raspberrypi/genimage-raspberrypi0w.cf`

image boot.vfat {
  vfat {
    files = {
      "bcm2708-rpi-0-w.dtb",
      "rpi-firmware/bootcode.bin",
      "rpi-firmware/cmdline.txt",
      "rpi-firmware/config.txt",
      "rpi-firmware/fixup.dat",
      "rpi-firmware/start.elf",
      "rpi-firmware/overlays",
      "zImage"
    }
  }
  size = 32M
}

image eric.vfat {
  vfat {}
  size = 1M
}

image sdcard.img {
  hdimage {
  }

  partition boot {
    partition-type = 0xC
    bootable = "true"
    image = "boot.vfat"
  }

  partition rootfs {
    partition-type = 0x83
    image = "rootfs.ext4"
  }
}

michaelolbrich commented 5 years ago

Right, I think creating empty vfat filesystems does not work right now. Filling it is a bit more difficult and it seems we don't get the syntax quite correct. Can you try #73? This adds a 'empty = true' option to make this explicit.

erichiggins commented 5 years ago

I'd love to try, though not sure about the best upgrade path. The Vagrant image that buildroot provides uses a zipfile/directory instead of git.

michaelolbrich commented 5 years ago

I'm not familiar with buildroot, but I expect, that you can add a patch for it. It should apply to that latest release. The buildroot folks can probably help you with that.

erichiggins commented 5 years ago

Ok, I'll ask in the buildroot support IRC and get back to you. Thanks for the quick turnaround!

michaelolbrich commented 4 years ago

This should be fixed with #73. Please speak up if there is still something missing.

pengutronix / genimage