next-server (Server-IP) external PXE server functionality is broken

[zapotah]

iPXE versions: 1.20.1, 1.21.1 BIOS & UEFI

When you define next-server in dhcp to point to another server, in BIOS mode the client starts to send the proxydhcp requests to the original dhcp server on port 4011 instead of the server pointed out with next-server. With UEFI the requests are sent to the correct server, however the client does nothing. The parameters sent are correct. With BIOS, if i setup two instances of dhcpd on the same box and on the server listening on port 67 omit filenames and instead leave next-server only, the client tries again on port 4011 and gets the correct information and boots.

BIOS: ipxe -> bcast request server1 -> ipxe (next-server server2) ipxe -> server1 port 4011 (should go to server2, if another instance is setup on port 4011 on server1, ipxe boots)

UEFI:

ipxe -> bcast request server1 -> ipxe (next-server server2) ipxe -> server2 port 4011 server2 ack -> ipxe ipxe -> server2 port 4011 (this goes on until the client gives up even when a pcap shows the options and parameters are correct)

if you setup server1 to point directly the correct bootloaders etc, everything works as expected.

[stappersg]

02:42 -!- zapotah [~zapotah@user/zapotah] has joined #ipxe
02:45 < zapotah> hi, im not quite sure if the binaries ive here are the newest
                 since theyre the ones shipped with qemu (pve to be exact)
         but theres some strange behavior with the pxe roms,
         the bios version does not continue to send the proxydhcp
         request to the Server-IP that is in the initial response from
         the server but instead tries to go to port 4011
         on the initial server
02:45 < zapotah> and the uefi rom sends the proxydhcp requests
                 to the correct server but essentially does nothing
03:01 < zapotah> same behavior on 1.20 and 1.21
03:20 < zapotah> anyway, the whole next-server functionality
                 is quite broken on both bios and uefi roms
03:59 < zapotah> made a ticket on github
03:59 < zapotah> https://github.com/ipxe/ipxe/issues/482

Regarding the quite broken

You are allowed to keep the pieces

[NiKiZe]

Do you have any documentation on what it should do? (As in RFC) I would not expect this to have anything to do with next-server at all.

[zapotah]

Every other rom on the planet functions as described, so does the UEFI rom as i described above. You are free to test with any device that has a pxe rom in it (intel, realtek etc.) even if they have other bugs.

Here are example configs for the two dhcp servers, in this case isc dhcpd (there are filters in place but those are irrelevant for reproducing): server1 (IP 10.45.64.1): allow booting; option arch code 93 = unsigned integer 16; update-conflict-detection false; authoritative; subnet 10.45.64.0 netmask 255.255.255.0 { pool { ignore-client-uids true; range 10.45.64.10 10.45.64.40; } option routers 10.45.64.1; option domain-name "domain.tld"; option domain-search "domain.tld"; option domain-name-servers 10.43.47.100,10.43.48.100; option vendor-class-identifier "PXEClient"; next-server 10.45.64.100; }

server2 (IP 10.45.64.100): allow booting; option arch code 93 = unsigned integer 16; update-conflict-detection false; authoritative; local-port 4011; class "pxeclients" { match if substring (option vendor-class-identifier, 0, 9) = "PXEClient"; server-name "bootstrap.domain.tld"; option vendor-class-identifier "PXEClient"; if option architecture-type = 00:00 { filename "grubneti386.0"; } elsif option architecture-type = 00:06 { filename "grubnetx64.efi.signed"; } elsif option architecture-type = 00:07 { filename "grubnetx64.efi.signed"; } elsif option architecture-type = 00:09 { filename "grubnetx64.efi.signed"; } } subnet 10.45.64.0 netmask 255.255.255.0 { pool { ignore-client-uids true; allow members of "pxeclients"; range 10.45.64.10 10.45.64.40; } }

If there is a syntactical error in there (shouldnt), its due to copy-paste, not an error in the config, the actual config is working fine.

[NiKiZe]

Config quote for formatting and readability.

server1 (IP 10.45.64.1):

#allow booting;
option arch code 93 = unsigned integer 16;
update-conflict-detection false;
authoritative;
subnet 10.45.64.0 netmask 255.255.255.0 {
pool {
ignore-client-uids true;
range 10.45.64.10 10.45.64.40;
}
option routers 10.45.64.1;
option domain-name-servers 10.43.47.100,10.43.48.100;
option vendor-class-identifier "PXEClient";
next-server 10.45.64.100;
}

server2 (IP 10.45.64.100):

#allow booting;
option arch code 93 = unsigned integer 16;
update-conflict-detection false;
authoritative;
local-port 4011;
class "pxeclients" {
match if substring (option vendor-class-identifier, 0, 9) = "PXEClient";
server-name "bootstrap.domain.tld";
option vendor-class-identifier "PXEClient";
if option architecture-type = 00:00 {
filename "grubneti386.0";
} elsif option architecture-type = 00:06 {
filename "grubnetx64.efi.signed";
} elsif option architecture-type = 00:07 {
filename "grubnetx64.efi.signed";
} elsif option architecture-type = 00:09 {
filename "grubnetx64.efi.signed";
}
}
subnet 10.45.64.0 netmask 255.255.255.0 {
pool {
ignore-client-uids true;
allow members of "pxeclients";
range 10.45.64.10 10.45.64.40;
}
}

Some information gathering: http://www.pix.net/software/pxeboot/archive/pxespec.pdf https://lists.ipxe.org/pipermail/ipxe-devel/2014-December/003912.html

[zapotah]

After some research the bug is different than first thought.

https://web.archive.org/web/20131102003141/http://download.intel.com/design/archives/wfm/downloads/pxespec.pdf Page 17 and 18 describe how the client should behave.

In my test case ive set PXEClient on both the original DHCP server and the relay, the BIOS rom then, according to spec above, sends the dhcprelay request to the original server (server2 does not listen on dhcp port 67 but instead only on the relay port, thus the client does not receive a PXEClient option 60 in the initial discover from the pxe server). The UEFI rom however takes the next-server (which sets Server-IP field) and sends the dhcprelay request to server2, in violation of the spec above.

A very corner case im sure but bug nonetheless.

The correct way to implement this, as documented absolutely nowhere on the internet besides that intel spec document, is to run an additional dhcp relay service on the dhcp server (that sets option 60 PXEClient) that can optionally send that request to the boot server that runs a dhcp server for this explicit purpose to give the required parameters, or to run a second server that is relayed the initial request that sends an address of 0.0.0.0 but sets option 60 PXEClient and then the proxydhcp service on port 4011 that gives out the required parameters.

[stappersg]

From my point of view is this github issue about an "It does NOT work as I expect it should work"

The iPXE project is, those who care about iPXE are, better of by closing this issue.

The irc log has / had some further advice that hasn't got any follow-up yet.

Please prevent that this issue is another one on the pile no one cares.

[NiKiZe]

@zapotah did you see https://lists.ipxe.org/pipermail/ipxe-devel/2014-December/003912.html ? From what I understand that message covers this case, iPXE follows spec, and that is not your wanted behavior?

[stappersg]

@NiKiZe please use your closing power on this issue.