tractordev / wanix

Experimental, local-first, web-native, Unix-like development environment
https://wanix.sh
MIT License
225 stars 11 forks source link

rework initfs for smaller kernel size #138

Closed progrium closed 5 months ago

progrium commented 6 months ago

We want to minimize the kernel binary size as much as possible so that it can be cached by browsers, and if it can't, it adds as little as possible to pageload.

We've experimented with having all bootstrap files within wanix-bootloader.js, splitting the kernel out, and also embedding some files within the kernel. We've also played with embedding the source to the shell and building it on first boot instead of embedding the binary. These all have different tradeoffs. We've talked about fetching binaries from dl.wanix.sh as needed, but this introduces release and versioning complexity on top of moving away from a self-contained ideal. We've proven we can make a single self-contained bootstrapping bootloader, but it comes with the price of large load times. Even if we don't do that by default, I'd like that to be possible and not create a dependency on hosted files.

Here I'd like to propose, not the final/ideal configuration, but one that is I think in the right direction and achieves goals for this release:

Rename current initdata/initfs references to bootdata and bootfs. This will specifically refer to files embedded in wanix-bootloader.js and should be limited to just the JavaScript files needed to start the kernel task.

Embed binaries, source, and other files in a new, separate wanix-initfs.gz bundle artifact. Stop embedding files in the kernel, allowing it to be much smaller (about 1/3 size). Instead, the kernel on fs initialization will check for a /sys/init directory. If it does not exist, it will fetch wanix-initfs.gz, unzip the tar file and write out the files to /sys/init. Eventually a script can move files into place from /sys/init if they don't exist, but for now the kernel can do this like it currently is with the existing initfs and embedded files.

This should result in a large wanix-initfs.gz file to only be downloaded on first boot (init). It introduces a 3rd bundle artifact, but I'm ok with this for now. It allows the kernel and bootloader to remain small.

Roughly what I'd expect to be in initfs, which is hardcoded for now, but later can be configured:

This would just be a gzipped tarball, which can be done in-process of the wanix bundle command.