netbootxyz / netboot.xyz

Your favorite operating systems in one place. A network-based bootable operating system installer based on iPXE.
https://netboot.xyz
Apache License 2.0
9.04k stars 674 forks source link

Local assets #1334

Open tristangrimaux opened 10 months ago

tristangrimaux commented 10 months ago

Netboot is a superb tool to manage together all the assets and menus needed to perform a netboot in the network. As an old sysadmin, y recon netbooting is a great tool that saved my life in many occasions. Right now I'm using another http server to serve all files and I'm thinking on moving the tftp server to prevent the problems arising from running netboot from docker. So... the main thing that keeps me here are the gorgeous menus (I'm trying to find a way to put a background image, but they are ok).

Is your feature request related to a problem? Please describe. This is a feature request.

Describe the solution you'd like 1) The program should let you download the complete distro, and should know what files are needed. Right now multipart file systems are failing because the asset browser is not showing filesystem.squashfs.part2 files. At least the files could be grouped by the asset name <-- I moved this thing to the browser app

2) Netboot should build a "only local assets" menu when booting

3) A way to proxy download the files if not found. Right now I added this to the configuration of an apache2 serving as the http server to emulate this behaviour

<location "/netboot.xyz/">
   #ProxyPass "http://github.com/netbootxyz/"
   RewriteEngine On
   RewriteCond "%{REQUEST_FILENAME}"       !-f
   RewriteCond "%{REQUEST_FILENAME}"       !-d
   RewriteRule "/var/www/html/netboot.xyz/(.*)" "http://github.com/netbootxyz/$1" [P]
   ProxyPassReverse "http://github.com/netbootxyz/"
</location>
ckdarby commented 10 months ago

It looks like Nginx is being used as a part of netboot.

I thought about adding an endpoint that would try local assets and, if not found, would fall back to a default param for the github one.

This probably could be expanded via nginx to cache locally anything pulled remotely. This way, it becomes a local asset afterwards.

Does this sound like an ideal scenario @tristangrimaux ?

Would like some feedback from @antonym if there is an appetite for me to go off and attempt the PR.

qu1ck commented 9 months ago

There is this related issue that's been open for a while https://github.com/netbootxyz/netboot.xyz/issues/1220

I would definitely be interested in having a fallback or auto caching functionality.

dpieski commented 8 months ago

That number 2 would be amazing. Alternatively, if there was a way to add a symbol next to the menu entry if there is a local asset for it.

tristangrimaux commented 7 months ago

It looks like Nginx is being used as a part of netboot.

I thought about adding an endpoint that would try local assets and, if not found, would fall back to a default param for the github one.

This probably could be expanded via nginx to cache locally anything pulled remotely. This way, it becomes a local asset afterwards.

Does this sound like an ideal scenario @tristangrimaux ?

Would like some feedback from @antonym if there is an appetite for me to go off and attempt the PR.

Lovely!

niklask52t commented 4 months ago

Can you help me set it up? if i enter the link with the apache proxy in my browser it downloads the correct file from github. But netboot cant find the file and gives an error

callistajosette commented 1 month ago

Can you help me set it up? if i enter the link with the apache proxy in my browser it downloads the correct file from github. But netboot cant find the file and gives an error

Running into the same issue did you ever figure this out?

65816 commented 1 month ago

I made a couple of things to point to a local ISO repository instead of downloading from the internet.

I use portainer to manage my docker. So I mounted an NFS volume to point to the container /assets/nfs volume.

My end goal is to have a boot to answer file to install ubuntu automatically. I tweak some code in ubuntu.ipxe to do this. I add the following after the :ubuntu

set live_endpoint http://{url}:8080 # where to find the vmlinuz initrd
set local_iso_dir yes # used to determine if local ISO repo instead of internet repo
set local_iso_url http://{url}:8080/nfs # where to find the ISO

/assets/nfs is accessible by the httpd daemon nothing special needed.

in :sub_boot of ubuntu.ipxe is where the path is generated before passing to the boot kernel

original code:

:sub_boot
imgfree
iseq ${os_arch} amd64 && set ubuntu_iso_url http://releases.ubuntu.com/${codename}/ubuntu-${version_number}-live-server-${os_arch}.iso ||
iseq ${os_arch} arm64 && set ubuntu_iso_url http://cdimage.ubuntu.com/releases/${version_number}/release/ubuntu-${version_number}-live-server-${os_arch}.iso ||
isset ${dhcp-server} && set netboot_params ip=dhcp url=${ubuntu_iso_url} || set netboot_params
echo Loading Ubuntu Subiquity Network Installer...
kernel ${kernel_url}vmlinuz root=/dev/ram0 ramdisk_size=3500000 cloud-config-url=/dev/null ${netboot_params} ${install_params} initrd=initrd.magic ${cmdline}
initrd ${kernel_url}initrd
echo
echo MD5sums:
md5sum vmlinuz initrd
boot

my modified code:

:sub_boot
imgfree
iseq ${local_iso_dir} yes && goto seturl ||
iseq ${os_arch} amd64 && set ubuntu_iso_url http://releases.ubuntu.com/${codename}/ubuntu-${version_number}-live-server-${os_arch}.iso ||
iseq ${os_arch} arm64 && set ubuntu_iso_url http://cdimage.ubuntu.com/releases/${version_number}/release/ubuntu-${version_number}-live-server-${os_arch}.iso ||
goto sub_boot2

:seturl
iseq ${os_arch} amd64 && set ubuntu_iso_url ${local_iso_url}/ubuntu-${version_number}-live-server-${os_arch}.iso ||

:sub_boot2
isset ${dhcp-server} && set netboot_params ip=dhcp url=${ubuntu_iso_url} || set netboot_params
echo Loading Ubuntu Subiquity Network Installer...
kernel ${kernel_url}vmlinuz root=/dev/ram0 ramdisk_size=3500000 cloud-config-url=/dev/null ${netboot_params} ${install_params} initrd=initrd.magic ${cmdline}
initrd ${kernel_url}initrd
echo
echo MD5sums:
md5sum vmlinuz initrd
boot