Add FreeBSD builds to NOOBS

[yzgyyang]

I use FreeBSD on my Raspberry Pis and would love to help to add FreeBSD builds to NOOBS. Are there any procedures to follow, or should I just open a PR after my own investigations and modifications? Thanks.

[procount]

The instructions for how to create a custom OS are detailed here -> https://github.com/raspberrypi/noobs#how-to-create-a-custom-os-version

But, first check that the file system / partitioning used is compatible with NOOBS. I think *BSD distros mostly use UFS...? I'm not sure that will be compatible with the current release of NOOBS.

[maxnet]

It is necessary that the file system is supported if you want to the size of your partition to be dynamic and occupy all remaining space on the SD card.

You can still use fixed size "raw" (dd) or "partclone" partition images if not though.

[XECDesign]

We generally only add operating systems if the NOOBS install is officially supported. If the FreeBSD devs are behind it and are willing to generate test and host all of the necessary files, it would definitely be a welcome addition.

[procount]

Also, if the "raw" partition images are used to get around an unsupported filesystem, I think any fstab references to the boot partition, for example, will have to be referenced using LABEL or PARTUUID, since NOOBS will not be able to adjust the partition reference in the partition_setup.sh script.

[emaste]

It is necessary that the file system is supported if you want to the size of your partition to be dynamic and occupy all remaining space on the SD card.

We generally use a UFS filesystem which I suspect is indeed not supported for resizing. But we could have NOOBS write a raw image, and have FreeBSD resize itself to the size of the SD card on boot.

[XECDesign]

I don't think there's any resizing needed for NOOBS installs. It just needs to be mountable from linux so that the fstab entry can be corrected. If the FreeBSD kernel can use a root= parameter instead, then the boot partition could be mounted instead.

[emaste]

We generally only add operating systems if the NOOBS install is officially supported. If the FreeBSD devs are behind it and are willing to generate test and host all of the necessary files, it would definitely be a welcome addition.

Yes, @yzgyyang is an intern working for the FreeBSD Foundation for the summer and this is one of the projects he's working on. Right now we are working to understand what artifacts and system changes we need to produce to plug into NOOBS.

[XECDesign]

Brilliant. Off the top of my head, I think we'd need to add ufstools and enable ufs write support in the kernel.

[procount]

@emaste & @yzgyyang - I have converted around 20 different OS distributions into NOOBS format for installation by my fork called PINN, so I'll try to help out if you have any questions. I even had a look at NetBSD, but postponed it when I saw it used UFS, so any help in this area is welcome.

I came up with a standard set of templates that I use for the NOOBS metadata files that are required for each OS. You can use these as a starting point (attached). These assume a "standard" installation of an EXT4 partition with an accompanying .tar.xz file that is expanded into it, so if you are going to use a "raw" image, you will need to make the necessary modifications.

Typically, there are 2 versions of each OS: 1 for the RPi 0/1 and one for the RPi 2/3, hence why there are 2 versions of some of the files.

template.tar.gz

The hardest part for me is often creating a set of installation png files. You need to create some (2-4?) in the slides_vga folder and then tar the folder into marketing.tar. Each slide needs to be approx 384x288 pixels.

[emaste]

It just needs to be mountable from linux so that the fstab entry can be corrected.

I suspect we can just set up the image so that it does not require fstab to be changed -- e.g. using something like /dev/ufs/rootfs / ufs rw,noatime 1 1 to mount based on a filesystem label.

[XECDesign]

Although the files still need to be extracted there in the first place. UFS write support is labelled as experimental in linux, so I'd start by seeing how well that works. If that falls through and we have to use the 'raw' partition approach, growfs.ufs seems to be available in linux, which gives some options. For example, we could dd the partition and then use growfs to resize it appropriately. The third alternative could be to investigate other filesystems supported by FreeBSD (zfs and ext2?).

[procount]

I suspect we can just set up the image so that it does not require fstab to be changed

Nevertheless, if @XECDesign can add ufs support, it will bring a lot more flexibility because users can choose the size of the freeBSD partition (See http://www.matthuisman.nz/2017/05/noobs-pinn-set-sizes.html as an example).

( @XECDesign, BTW - if you add btrfs support at the same time, you could support for e.g. XBian as well. I already added it to PINN, so I can create a PR if you like?)

[XECDesign]

Thanks, @procount. I don't think we can add XBian, given that we already have LibreELEC and OSMC. At this time, btrfs would only take up space.

[emaste]

If the Linux UFS write support is usable (if experimental just means not widely used/tested) then that seems like a good path. Otherwise I don't see a lot of downside from the raw partition -- NOOBS can create a partition of the desired size, dd the raw image at the beginning, and FreeBSD can grow the filesystem to match the partition size on boot.

[procount]

btrfs would only take up space.

Yeah, my image is growing noticeably bigger these days! :)

[maxnet]

I suspect we can just set up the image so that it does not require fstab to be changed -- e.g. using something like /dev/ufs/rootfs / ufs rw,noatime 1 1 to mount based on a filesystem label.

Do note that NOOBS allows the user to install multiple operating systems at the same time. And there might be more than one OS interested in using not so unique labels like rootfs/root... So if you go that way, using slightly more unique identifiers would be preferred.

While FreeBSD creates links like /dev/ufs/rootfs, note that other OSes like Linux create them in the form /dev/mapper/rootfs not mentioning the file system. So it can even be a problem to others if you are the only one using UFS, but there do is another ext4 file system using the same name. When NOOBS formats a partition for an OS it will check if the requested label is available. But you will have to ensure uniqueness yourself if using raw partition images.

Also, ideally it should be possible to install your OS to any logical MBR partition on SD card or USB storage. In case it only supports booting from a primary partition it does is possible to indicate that, but then it cannot be installed at the same time as other OSes that claim primary partitions, like Windows.

[emaste]

So if you go that way, using slightly more unique identifiers would be preferred.

Certainly we could use something like /dev/ufs/FreeBSD_root instead.

Also, ideally it should be possible to install your OS to any logical MBR partition on SD card or USB storage.

Ah, indeed.

Looking at my FreeBSD R-Pi3 image from rasbpsd.org I see we have a FAT boot partition and a UFS partition:

/dev/da0s1:  DOS/MBR boot sector, code offset 0x3c+2, OEM-ID "BSD4.4  ", 
  sectors/cluster 8, root entries 512, sectors/FAT 50, sectors/track 63,
  heads 255, sectors 102375 (volumes > 32 MB) , serial number 0xd11d17f2,
  label: "BOOT       ", FAT (16 bit)
/dev/da0s2a: Unix Fast File system [v2] (little-endian) last mounted on /,
  last written at Tue Feb  7 18:30:09 2017, clean flag 1, readonly flag 0,
  number of blocks 7778688, number of data blocks 7531143,
  number of cylinder groups 68, block size 32768, fragment size 4096,
  average file size 16384, average number of files in dir 64,
  pending blocks to free 0, pending inodes to free 0, system-wide uuid 0,
  minimum percentage of free blocks 8, TIME optimization

The contents of the FAT partition:

EFI                     config.txt              start_cd.elf
LICENCE.broadcom        fixup.dat               start_x.elf
README                  fixup_cd.dat            startup.nsh
armstub8.bin            fixup_x.dat             u-boot.bin
bcm2710-rpi-3-b.dtb     overlays
bootcode.bin            start.elf

So we boot via R-Pi firmware -> ? -> u-boot EFI -> bootaa64.efi which is FreeBSD's boot1.efi. The contents of this FAT partition could be shared with other operating systems but that seems error-prone and cumbersome.

It seems to me we could have NOOBS set up FreeBSD by creating the small FAT boot partition and a larger partition (size of the user's choosing) into which a raw UFS filesystem is written. We can then store loader.conf (boot config) in the FAT partition, which NOOBS can configure (instead of /etc/fstab). Then on first boot FreeBSD will resize the UFS filesystem to occupy the partition it was written into. How does that approach sound?

I've asked @yzgyyang to try this on an existing NOOBS SD card (manually creating partitions and copying things over), to test out the end result; assuming that looks good we'll then figure out how to have NOOBS put it there in te first place.

[maxnet]

It seems to me we could have NOOBS set up FreeBSD by creating the small FAT boot partition and a larger partition (size of the user's choosing) into which a raw UFS filesystem is written. We can then store loader.conf (boot config) in the FAT partition, which NOOBS can configure (instead of /etc/fstab).

Yes, with NOOBS each OS must always have a FAT boot partition of its own.

Do take into account that the partition number you get from NOOBS varies though. And unlike the Raspberry boot firmware (that is typically used to boot Linux kernels directly), I have my doubts other software like u-boot really has a clue which FAT partition it was booted from. And it likely does need to figure that out, to load your bootaa64.efi.

So in addition to patching any FreeBSD configuration, you may also need to patch the partition number in u-boot if it's hard coded. Or in case it scans all available partitions to look for its own files: give your loader a more unique name than bootaa64.efi, and modify u-boot to look for that...

[procount]

It would be nice to have a standard solution for U-BOOT that we can promote since many of the 64-bit OSes use it with a hard coded partition number (2) to load the rootfs from. It is often part of a boot script that is extracted, so whilst NOOBS could extract it, modify it and compress it again, I don't think it is something we want to support, so if we can avoid NOOBS having to patch the U-BOOT image to account for the differing partitions (by using e.g. unique labels or Partuuid etc) then all the better.

[maxnet]

It would be nice to have a standard solution for U-BOOT that we can promote since many of the 64-bit OSes use it with a hard coded partition number (2) to load the rootfs from. It is often part of a boot script that is extracted, so whilst NOOBS could extract it, modify it and compress it again,

Updating the root partition is not so hard, given that a lot of u-boot builds also support plain text uEnv.txt instead of boot.scr scripts. Patching u-boot.bin to look for uEnv.txt/boot.scr on the right boot partition and not just in the first is harder...

But that's with plain u-boot. Not sure what the EFI version does and does not.

[ykla]

How is the progress going?

[lurch]

One of the earlier comments said "Yes, @yzgyyang is an intern working for the FreeBSD Foundation for the summer", and indeed there's been no updates to this issue since summer 2017, i.e. 3 years ago... draw your own conclusions :wink:

[emaste]

Still interested in having this happen; with ongoing work to support EFI on RPi4 it might be an easier path.

@yzgyyang's work term is indeed long over, although the FreeBSD Foundation has had other interns since and will have more, so one of them may take this up.