search

raspberrypi / firmware

This repository contains pre-compiled binaries of the current Raspberry Pi kernel and modules, userspace libraries, and bootloader/GPU firmware.
5.15k stars 1.68k forks source link

R.Pi PXE Bootloader doesn't work on IPv6 only network #1808

Closed owendelong closed 8 months ago

owendelong commented 1 year ago

Describe the bug The PXE Loader on the Pi appears to be IPv4 only. Please add IPv6 support.

To reproduce

  1. Create an IPv6 only network with properly configured PXE Boot server
  2. Attempt to boot a Pi via PXE on said network.

Expected behaviour I expect the Pi to attempt to get an IPv6 address via appropriate RS/NS messages and given the A and/or M flags in the RA, to follow those instructions and request appropriate data from the DHCPv6 server.

Actual behaviour No IPv6 packets are emitted by the R.Pi and IPv6 Router Advertisements are ignored by the Pi during the PXE boot.

System

Logs This problem occurs before the kernel is loaded, there is no logging available during PXE Boot.

Additional context I believe the above is a sufficient description of the situation. I will endeavor to monitor comments or requests for additional information on this issue and respond timely. While I think this is a bug in this day and age, I understand that some may think it more of a feature request.

peterharperuk commented 1 year ago

Are you expecting it to follow some "standard" for IPv6 network boot? If you are - do you have any details? RPi 4 has some unproven IPv6 boot functionality if that's any use https://www.raspberrypi.com/documentation/computers/remote-access.html#network-booting-using-ipv6

owendelong commented 1 year ago

Yes. I’m not sure why nfsroot is protocol specific to IPv4, but it sounds like the newer firmware solves the issues I felt necessary to report here. Hopefully that will soon move from experimental to standard.

In terms of DHCP server support, we are already using KEA which is a vastly superior alternative to dnsmasq (which I consider fairly awful, tbh) and has several advantages over the old isc dhcpd. (KEA is from ISC also and is a ground-up fresh implementation incorporating decades of operational experience and lessons learned in a well designed and more consistent framework).

peterharperuk commented 1 year ago

Interesting - I might do some testing with KEA

owendelong commented 1 year ago

FWIW, our entire environment is here: https://github.com/socallinuxexpo/scale-network

Our KEA environment is built in NIX from here: https://github.com/socallinuxexpo/scale-network/blob/master/nix/machines/core/common.nix

Most of the important KEA configuration fragments are here: https://github.com/socallinuxexpo/scale-network/tree/master/facts/servers/examples/dhcp

In case that helps.

This is the automation and configuration for the Southern California Linux Expo. Next year will be our 21st annual conference. It generally runs from a Thursday through a Sunday in March. Setup of the network usually starts the Monday before and Teardown usually starts about 5PM on Sunday and we usually have the truck fully unloaded with everything back in storage before 9PM on Sunday.

The network consists of 150+ WAPs (Open WRT), 50+ 48 Port 1U PoE switches (Juniper EX4200-48P), 3 Juniper SRX-300 routers, and about 4 miles of deployed Ethernet cables (Plus some fiber jumpers).

All VLANs are either Dual Stack or IPv6 Only. (Signs is the primary IPv6-only VLAN where this is an issue for us).

I know this is a bit of a digression from the actual issue here, but I figure sometimes understanding the deployment environment helps gain insight into solutions.

owendelong commented 1 year ago

You may run into additional troubles since the decision was made to close https://github.com/raspberrypi/rpi-eeprom/issues/182 without properly obtaining an IANA number for Option93. However, according to https://www.iana.org/assignments/dhcpv6-parameters/dhcpv6-parameters.xhtml#processor-architecture there is Value 0x0029 for Raspberry Pi.

I couldn't find a corresponding map for DHCPv4. It looks like that's usually handled as a substring of Option 60 containing "PXEClient:Arch:xxxxx" where xxxxx is the 5 digit hex version of the assigned architecture number.

The most recent document I could find lists the following:

# 00:06, 00:07, 00:09 EFI (x86, x64, ByteCode) respectively
# 00:00, 00:02 BIOS (x86, Itanium)
# 00:03 thru 00:05, 00:08 -- Deprecated
# 00:0a thru 00:0b, 00:15 thru 00:18 -- ARM
# 00:0c thru 00:0e -- PowerPC / Power OPAL
# 00:0f thru 00:14 -- Boot from HTTP
# 00:19 thru 00:1e -- RISC-V
# 00:1f thru 00:20 -- S390
# 00:21 thru ff:ff -- Unassigned (13 Jan 2020)

So if the Pi firmware is updated to request netbook over IPv6, I suggest properly setting option 93 to 0x0029 for DHCP6 requests. I would argue that Any of the ARM choices (000a, 000b, 0015-0018) would be a better choice than the current report of x86 BIOS, but I believe a better solution would be to seek a proper IANA registration.

I'm honestly not sure where I found that table and I haven't been able to find it searching this evening. A greatly abbreviated version is present in RFC4758 which defines that particular syntax for Option 60. The RFC for Option 60 in DHCP4, however, is basically a free-for-all allowing any vendor to put whatever they want there.

peterharperuk commented 1 year ago

However, according to https://www.iana.org/assignments/dhcpv6-parameters/dhcpv6-parameters.xhtml#processor-architecture there is Value 0x0029 for Raspberry Pi

I had that added and it IS used if you boot using IPV6 for CLIENT_ARCH_TYPE (option 61). I don't believe option 93 is relevant for DHCPv6.

owendelong commented 1 year ago

OK, so if we upgrade the firmware, with the PXE on the R.Pi now try to get an IPv6 address? The firmware we were running never even asked for an IPv6 address (No RS, no DHCPv6 Request, and no apparent processing of RA packets).

If that's the case, I'll try to test this weekend.

peterharperuk commented 1 year ago

Currently it's IPV4 or IPV6 not both. You have to modify the bootloader configuration for IPV6 (see https://www.raspberrypi.com/documentation/computers/remote-access.html#network-booting-using-ipv6)

Bootloader Modify the configuration to tell it to attempt network boot via IPv6 rather than IPv4.

BOOT_ORDER=0xf21 # 2=Network boot USE_IPV6=1 # Enable IPv6 network boot BOOT_UART=1 # Debug

To revert to IPv4 network boot just remove the USE_IPV6 line from boot.conf.

timg236 commented 8 months ago

Closing as resolved because ipv6 is support is available if really desired. No plans to extend it further right now

MatthewCroughan commented 6 months ago

https://github.com/raspberrypi/documentation/blob/develop/documentation/asciidoc/computers/raspberry-pi/boot-net.adoc

You must set option 43 to "Raspberry Pi Boot "

owendelong commented 5 months ago

I think this should be reopened...

  1. It would be nice if there were additional information reported to the user on what failed on the boot screen.
    • We had systems trying to boot over IPv6 and apparently never getting an address.
    • We tried this both with O and with M+O bits set in the RA.
    • We definitely saw (we believe proper) DHCPv6 responses being sent back to the client.
    • The client merely timed out as though those responses were never received. Some detail as to why responses were discarded would be useful.
  2. If there is DHCP magic required, having that documented would be worth while.
  3. Correct, Option 93 does not apply to DHCPv6
  4. Option 43 also is not the Vendor-Specific option for DHCPv6, it uses 17 instead.
  5. We aren't certain, but we believe this may be related to the fact that our DHCPv6 environment is using a relay agent to reach the DHCPv6 server (Juniper SRX300).
  6. I'm happy to help document this once I can get it working.
owendelong commented 5 months ago

If it would be useful, I'm willing to post our KEA configuration and any other requested data here. (it's all in a public GitHub repository anyway -- socallinuxexpo/scale-network).

One difficulty for us is that while we have dev and test environments, the full-blown network only exists for 7 days per year. We start setting it up on Monday before the conference, the conference runs from Thursday to Sunday, then it all gets torn down and put back into storage for a year.

Nonetheless, I will replicate whatever is needed to solve this. We are really hoping to have a solution before next year's conference.

peterharperuk commented 5 months ago

So you're asking for IPV6 boot to work with the KEA DHCP server? Are there any guides for getting this running on a Raspberry PI? I vaguely remember looking at this years a go but wasn't easily able to get anything running so gave up as . I will have a look if you have proper support for IPV6 boot.

sarcasticadmin commented 1 week ago

So you're asking for IPV6 boot to work with the KEA DHCP server? Are there any guides for getting this running on a Raspberry PI? I vaguely remember looking at this years a go but wasn't easily able to get anything running so gave up as . I will have a look if you have proper support for IPV6 boot.

@peterharperuk The Kea DHCP server just happens to be what we've been running for the conference network that @owendelong mentioned. The issue is that there are outstanding unknowns for what is needed to make the rpi boot via an ipv6 only network @owendelong has called these out above: https://github.com/raspberrypi/firmware/issues/1808#issuecomment-2018705766

Attached is a slimmed down version of our kea config (excuse the .txt prefix, github upload make me add it): slim-dhcp6-server.conf.txt You will need to modify this config for your particular kea server ip, tftp server, and ipv6 subnet(s) to run the Kea DHCPv6 service.