search

raspberrypi / rpi-eeprom

Installation scripts and binaries for the Raspberry Pi 4 and Raspberry Pi 5 bootloader EEPROMs
https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#raspberry-pi-boot-eeprom
Other
1.27k stars 203 forks source link

No DHCPACK with DHCP relay agent #58

Closed marcolefo closed 4 years ago

marcolefo commented 4 years ago

Hi

I am trying to netboot my pi4 with no success.

After plug the pi, the green LED is fix. I saw DHCP requests on my server. The Pi keep requesting DHCP server and the server answer with the IP. I only see DHCPDISCOVER and DHCPOFFER. No DHCPACK.

I have tried with pieeprom-2019-11-18.bin and pieeprom-2019-10-16.bin.

My dhcp server is a isc-dhcp on a Debian Jessie. Note that it works for a pi3. The definition for the subnet is :

subnet 10.10.10.0 netmask 255.255.255.0 {
        interface eth0;
        allow unknown-clients;
        range 10.10.10.230 10.10.10.234;
        option subnet-mask 255.255.255.0;
        option routers 10.10.10.254;

        next-server 10.10.10.1;
        option tftp-server-name "10.10.10.1";
}
Muxxxi commented 4 years ago

Hi

I don't know if it helps, but the following configuration works for me so far. I'm running the Docker Image networkboot/dhcpd.

subnet 192.168.1.0 netmask 255.255.255.0 {
        range 192.168.1.20 192.168.1.100;
        next-server 192.168.1.1;
        option tftp-server-name "192.168.1.1";
        always-broadcast false;
        option vendor-class-identifier "PXEClient";
        option vendor-encapsulated-options "Raspberry Pi Boot";
        option routers 192.168.1.1;
        option domain-name-servers 8.8.8.8;
}
timg236 commented 4 years ago

Please can you post wireshark capture. Might be expecting SI_ADDR but won’t be able to look at this for a few days.

marcolefo commented 4 years ago

Hi

I don't know if it helps, but the following configuration works for me so far. I'm running the Docker Image networkboot/dhcpd.

subnet 192.168.1.0 netmask 255.255.255.0 {
        range 192.168.1.20 192.168.1.100;
          next-server 192.168.1.1;
        option tftp-server-name "192.168.1.1";
      always-broadcast false;
      option vendor-class-identifier "PXEClient";
        option vendor-encapsulated-options "Raspberry Pi Boot";
        option routers 192.168.1.1;
        option domain-name-servers 8.8.8.8;
}

No change with adding these options

marcolefo commented 4 years ago

Please can you post wireshark capture. Might be expecting SI_ADDR but won’t be able to look at this for a few days.

With fiIter udp.port==68 see only DHCP Discover

No.     Time           Source                Destination           Protocol Length Info
     40 17.643001      0.0.0.0               255.255.255.255       DHCP     364    DHCP Discover - Transaction ID 0x5ad50db4

Frame 40: 364 bytes on wire (2912 bits), 364 bytes captured (2912 bits) on interface 0
Ethernet II, Src: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Destination: Broadcast (ff:ff:ff:ff:ff:ff)
    Source: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc)
    Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 350
    Identification: 0x506a (20586)
    Flags: 0x0000
    Time to live: 64
    Protocol: UDP (17)
    Header checksum: 0x2926 [validation disabled]
    [Header checksum status: Unverified]
    Source: 0.0.0.0
    Destination: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
    Source Port: 68
    Destination Port: 67
    Length: 330
    Checksum: 0xfad3 [unverified]
    [Checksum Status: Unverified]
    [Stream index: 0]
Bootstrap Protocol (Discover)
    Message type: Boot Request (1)
    Hardware type: Ethernet (0x01)
    Hardware address length: 6
    Hops: 0
    Transaction ID: 0x5ad50db4
    Seconds elapsed: 0
    Bootp flags: 0x0000 (Unicast)
        0... .... .... .... = Broadcast flag: Unicast
        .000 0000 0000 0000 = Reserved flags: 0x0000
    Client IP address: 0.0.0.0
    Your (client) IP address: 0.0.0.0
    Next server IP address: 0.0.0.0
    Relay agent IP address: 0.0.0.0
    Client MAC address: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc)
    Client hardware address padding: 00000000000000000000
    Server host name not given
    Boot file name not given
    Magic cookie: DHCP
    Option: (53) DHCP Message Type (Discover)
        Length: 1
        DHCP: Discover (1)
    Option: (55) Parameter Request List
        Length: 14
        Parameter Request List Item: (1) Subnet Mask
        Parameter Request List Item: (3) Router
        Parameter Request List Item: (43) Vendor-Specific Information
        Parameter Request List Item: (60) Vendor class identifier
        Parameter Request List Item: (66) TFTP Server Name
        Parameter Request List Item: (67) Bootfile name
        Parameter Request List Item: (128) DOCSIS full security server IP [TODO]
        Parameter Request List Item: (129) PXE - undefined (vendor specific)
        Parameter Request List Item: (130) PXE - undefined (vendor specific)
        Parameter Request List Item: (131) PXE - undefined (vendor specific)
        Parameter Request List Item: (132) PXE - undefined (vendor specific)
        Parameter Request List Item: (133) PXE - undefined (vendor specific)
        Parameter Request List Item: (134) PXE - undefined (vendor specific)
        Parameter Request List Item: (135) PXE - undefined (vendor specific)
    Option: (93) Client System Architecture
        Length: 2
        Client System Architecture: IA x86 PC (0)
    Option: (94) Client Network Device Interface
        Length: 3
        Major Version: 2
        Minor Version: 1
    Option: (97) UUID/GUID-based Client Identifier
        Length: 17
        Client Identifier (UUID): 6aee7243-7243-6aee-4372-ee6a4372ee6a
    Option: (60) Vendor class identifier
        Length: 32
        Vendor class identifier: PXEClient:Arch:00000:UNDI:002001
    Option: (255) End
        Option End: 255

No.     Time           Source                Destination           Protocol Length Info
     84 33.643465      0.0.0.0               255.255.255.255       DHCP     364    DHCP Discover - Transaction ID 0x5ad50db4

Frame 84: 364 bytes on wire (2912 bits), 364 bytes captured (2912 bits) on interface 0
Ethernet II, Src: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Destination: Broadcast (ff:ff:ff:ff:ff:ff)
    Source: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc)
    Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 350
    Identification: 0x506b (20587)
    Flags: 0x0000
    Time to live: 64
    Protocol: UDP (17)
    Header checksum: 0x2925 [validation disabled]
    [Header checksum status: Unverified]
    Source: 0.0.0.0
    Destination: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
    Source Port: 68
    Destination Port: 67
    Length: 330
    Checksum: 0xfad3 [unverified]
    [Checksum Status: Unverified]
    [Stream index: 0]
Bootstrap Protocol (Discover)
    Message type: Boot Request (1)
    Hardware type: Ethernet (0x01)
    Hardware address length: 6
    Hops: 0
    Transaction ID: 0x5ad50db4
    Seconds elapsed: 0
    Bootp flags: 0x0000 (Unicast)
        0... .... .... .... = Broadcast flag: Unicast
        .000 0000 0000 0000 = Reserved flags: 0x0000
    Client IP address: 0.0.0.0
    Your (client) IP address: 0.0.0.0
    Next server IP address: 0.0.0.0
    Relay agent IP address: 0.0.0.0
    Client MAC address: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc)
    Client hardware address padding: 00000000000000000000
    Server host name not given
    Boot file name not given
    Magic cookie: DHCP
    Option: (53) DHCP Message Type (Discover)
        Length: 1
        DHCP: Discover (1)
    Option: (55) Parameter Request List
        Length: 14
        Parameter Request List Item: (1) Subnet Mask
        Parameter Request List Item: (3) Router
        Parameter Request List Item: (43) Vendor-Specific Information
        Parameter Request List Item: (60) Vendor class identifier
        Parameter Request List Item: (66) TFTP Server Name
        Parameter Request List Item: (67) Bootfile name
        Parameter Request List Item: (128) DOCSIS full security server IP [TODO]
        Parameter Request List Item: (129) PXE - undefined (vendor specific)
        Parameter Request List Item: (130) PXE - undefined (vendor specific)
        Parameter Request List Item: (131) PXE - undefined (vendor specific)
        Parameter Request List Item: (132) PXE - undefined (vendor specific)
        Parameter Request List Item: (133) PXE - undefined (vendor specific)
        Parameter Request List Item: (134) PXE - undefined (vendor specific)
        Parameter Request List Item: (135) PXE - undefined (vendor specific)
    Option: (93) Client System Architecture
        Length: 2
        Client System Architecture: IA x86 PC (0)
    Option: (94) Client Network Device Interface
        Length: 3
        Major Version: 2
        Minor Version: 1
    Option: (97) UUID/GUID-based Client Identifier
        Length: 17
        Client Identifier (UUID): 6aee7243-7243-6aee-4372-ee6a4372ee6a
    Option: (60) Vendor class identifier
        Length: 32
        Vendor class identifier: PXEClient:Arch:00000:UNDI:002001
    Option: (255) End
        Option End: 255

No.     Time           Source                Destination           Protocol Length Info
    135 51.853365      0.0.0.0               255.255.255.255       DHCP     364    DHCP Discover - Transaction ID 0x5864f63c

Frame 135: 364 bytes on wire (2912 bits), 364 bytes captured (2912 bits) on interface 0
Ethernet II, Src: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Destination: Broadcast (ff:ff:ff:ff:ff:ff)
    Source: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc)
    Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 350
    Identification: 0x506c (20588)
    Flags: 0x0000
    Time to live: 64
    Protocol: UDP (17)
    Header checksum: 0x2924 [validation disabled]
    [Header checksum status: Unverified]
    Source: 0.0.0.0
    Destination: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
    Source Port: 68
    Destination Port: 67
    Length: 330
    Checksum: 0x14bc [unverified]
    [Checksum Status: Unverified]
    [Stream index: 0]
Bootstrap Protocol (Discover)
    Message type: Boot Request (1)
    Hardware type: Ethernet (0x01)
    Hardware address length: 6
    Hops: 0
    Transaction ID: 0x5864f63c
    Seconds elapsed: 0
    Bootp flags: 0x0000 (Unicast)
        0... .... .... .... = Broadcast flag: Unicast
        .000 0000 0000 0000 = Reserved flags: 0x0000
    Client IP address: 0.0.0.0
    Your (client) IP address: 0.0.0.0
    Next server IP address: 0.0.0.0
    Relay agent IP address: 0.0.0.0
    Client MAC address: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc)
    Client hardware address padding: 00000000000000000000
    Server host name not given
    Boot file name not given
    Magic cookie: DHCP
    Option: (53) DHCP Message Type (Discover)
        Length: 1
        DHCP: Discover (1)
    Option: (55) Parameter Request List
        Length: 14
        Parameter Request List Item: (1) Subnet Mask
        Parameter Request List Item: (3) Router
        Parameter Request List Item: (43) Vendor-Specific Information
        Parameter Request List Item: (60) Vendor class identifier
        Parameter Request List Item: (66) TFTP Server Name
        Parameter Request List Item: (67) Bootfile name
        Parameter Request List Item: (128) DOCSIS full security server IP [TODO]
        Parameter Request List Item: (129) PXE - undefined (vendor specific)
        Parameter Request List Item: (130) PXE - undefined (vendor specific)
        Parameter Request List Item: (131) PXE - undefined (vendor specific)
        Parameter Request List Item: (132) PXE - undefined (vendor specific)
        Parameter Request List Item: (133) PXE - undefined (vendor specific)
        Parameter Request List Item: (134) PXE - undefined (vendor specific)
        Parameter Request List Item: (135) PXE - undefined (vendor specific)
    Option: (93) Client System Architecture
        Length: 2
        Client System Architecture: IA x86 PC (0)
    Option: (94) Client Network Device Interface
        Length: 3
        Major Version: 2
        Minor Version: 1
    Option: (97) UUID/GUID-based Client Identifier
        Length: 17
        Client Identifier (UUID): 6aee7243-7243-6aee-4372-ee6a4372ee6a
    Option: (60) Vendor class identifier
        Length: 32
        Vendor class identifier: PXEClient:Arch:00000:UNDI:002001
    Option: (255) End
        Option End: 255

No.     Time           Source                Destination           Protocol Length Info
    156 59.845873      0.0.0.0               255.255.255.255       DHCP     364    DHCP Discover - Transaction ID 0x5864f63c

Frame 156: 364 bytes on wire (2912 bits), 364 bytes captured (2912 bits) on interface 0
Ethernet II, Src: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Destination: Broadcast (ff:ff:ff:ff:ff:ff)
    Source: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc)
    Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 350
    Identification: 0x506d (20589)
    Flags: 0x0000
    Time to live: 64
    Protocol: UDP (17)
    Header checksum: 0x2923 [validation disabled]
    [Header checksum status: Unverified]
    Source: 0.0.0.0
    Destination: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
    Source Port: 68
    Destination Port: 67
    Length: 330
    Checksum: 0x14bc [unverified]
    [Checksum Status: Unverified]
    [Stream index: 0]
Bootstrap Protocol (Discover)
    Message type: Boot Request (1)
    Hardware type: Ethernet (0x01)
    Hardware address length: 6
    Hops: 0
    Transaction ID: 0x5864f63c
    Seconds elapsed: 0
    Bootp flags: 0x0000 (Unicast)
        0... .... .... .... = Broadcast flag: Unicast
        .000 0000 0000 0000 = Reserved flags: 0x0000
    Client IP address: 0.0.0.0
    Your (client) IP address: 0.0.0.0
    Next server IP address: 0.0.0.0
    Relay agent IP address: 0.0.0.0
    Client MAC address: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc)
    Client hardware address padding: 00000000000000000000
    Server host name not given
    Boot file name not given
    Magic cookie: DHCP
    Option: (53) DHCP Message Type (Discover)
        Length: 1
        DHCP: Discover (1)
    Option: (55) Parameter Request List
        Length: 14
        Parameter Request List Item: (1) Subnet Mask
        Parameter Request List Item: (3) Router
        Parameter Request List Item: (43) Vendor-Specific Information
        Parameter Request List Item: (60) Vendor class identifier
        Parameter Request List Item: (66) TFTP Server Name
        Parameter Request List Item: (67) Bootfile name
        Parameter Request List Item: (128) DOCSIS full security server IP [TODO]
        Parameter Request List Item: (129) PXE - undefined (vendor specific)
        Parameter Request List Item: (130) PXE - undefined (vendor specific)
        Parameter Request List Item: (131) PXE - undefined (vendor specific)
        Parameter Request List Item: (132) PXE - undefined (vendor specific)
        Parameter Request List Item: (133) PXE - undefined (vendor specific)
        Parameter Request List Item: (134) PXE - undefined (vendor specific)
        Parameter Request List Item: (135) PXE - undefined (vendor specific)
    Option: (93) Client System Architecture
        Length: 2
        Client System Architecture: IA x86 PC (0)
    Option: (94) Client Network Device Interface
        Length: 3
        Major Version: 2
        Minor Version: 1
    Option: (97) UUID/GUID-based Client Identifier
        Length: 17
        Client Identifier (UUID): 6aee7243-7243-6aee-4372-ee6a4372ee6a
    Option: (60) Vendor class identifier
        Length: 32
        Vendor class identifier: PXEClient:Arch:00000:UNDI:002001
    Option: (255) End
        Option End: 255

No.     Time           Source                Destination           Protocol Length Info
    198 75.845397      0.0.0.0               255.255.255.255       DHCP     364    DHCP Discover - Transaction ID 0x5864f63c

Frame 198: 364 bytes on wire (2912 bits), 364 bytes captured (2912 bits) on interface 0
Ethernet II, Src: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Destination: Broadcast (ff:ff:ff:ff:ff:ff)
    Source: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc)
    Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 350
    Identification: 0x506e (20590)
    Flags: 0x0000
    Time to live: 64
    Protocol: UDP (17)
    Header checksum: 0x2922 [validation disabled]
    [Header checksum status: Unverified]
    Source: 0.0.0.0
    Destination: 255.255.255.255
User Datagram Protocol, Src Port: 68, Dst Port: 67
    Source Port: 68
    Destination Port: 67
    Length: 330
    Checksum: 0x14bc [unverified]
    [Checksum Status: Unverified]
    [Stream index: 0]
Bootstrap Protocol (Discover)
    Message type: Boot Request (1)
    Hardware type: Ethernet (0x01)
    Hardware address length: 6
    Hops: 0
    Transaction ID: 0x5864f63c
    Seconds elapsed: 0
    Bootp flags: 0x0000 (Unicast)
        0... .... .... .... = Broadcast flag: Unicast
        .000 0000 0000 0000 = Reserved flags: 0x0000
    Client IP address: 0.0.0.0
    Your (client) IP address: 0.0.0.0
    Next server IP address: 0.0.0.0
    Relay agent IP address: 0.0.0.0
    Client MAC address: dc:a6:32:27:76:bc (dc:a6:32:27:76:bc)
    Client hardware address padding: 00000000000000000000
    Server host name not given
    Boot file name not given
    Magic cookie: DHCP
    Option: (53) DHCP Message Type (Discover)
        Length: 1
        DHCP: Discover (1)
    Option: (55) Parameter Request List
        Length: 14
        Parameter Request List Item: (1) Subnet Mask
        Parameter Request List Item: (3) Router
        Parameter Request List Item: (43) Vendor-Specific Information
        Parameter Request List Item: (60) Vendor class identifier
        Parameter Request List Item: (66) TFTP Server Name
        Parameter Request List Item: (67) Bootfile name
        Parameter Request List Item: (128) DOCSIS full security server IP [TODO]
        Parameter Request List Item: (129) PXE - undefined (vendor specific)
        Parameter Request List Item: (130) PXE - undefined (vendor specific)
        Parameter Request List Item: (131) PXE - undefined (vendor specific)
        Parameter Request List Item: (132) PXE - undefined (vendor specific)
        Parameter Request List Item: (133) PXE - undefined (vendor specific)
        Parameter Request List Item: (134) PXE - undefined (vendor specific)
        Parameter Request List Item: (135) PXE - undefined (vendor specific)
    Option: (93) Client System Architecture
        Length: 2
        Client System Architecture: IA x86 PC (0)
    Option: (94) Client Network Device Interface
        Length: 3
        Major Version: 2
        Minor Version: 1
    Option: (97) UUID/GUID-based Client Identifier
        Length: 17
        Client Identifier (UUID): 6aee7243-7243-6aee-4372-ee6a4372ee6a
    Option: (60) Vendor class identifier
        Length: 32
        Vendor class identifier: PXEClient:Arch:00000:UNDI:002001
    Option: (255) End
        Option End: 255
timg236 commented 4 years ago

I think we'd need to see the DHCPOFFER packets the DHCPDISCOVER is constant and looks reasonable

marcolefo commented 4 years ago

I know it's strange but I have captured no DHCPOFFER packet... but in dhcp log I can see : 

Nov 25 10:47:14  dhcpd: DHCPDISCOVER from dc:a6:32:27:76:bc via 10.10.10.254
Nov 25 10:47:14  dhcpd: DHCPOFFER on 10.10.10.230 to dc:a6:32:27:76:bc via 10.10.10.254
Nov 25 10:47:22  dhcpd: DHCPDISCOVER from dc:a6:32:27:76:bc via 10.10.10.254
Nov 25 10:47:22  dhcpd: DHCPOFFER on 10.10.10.230 to dc:a6:32:27:76:bc via 10.10.10.254
Nov 25 10:47:38  dhcpd: DHCPDISCOVER from dc:a6:32:27:76:bc via 10.10.10.254
Nov 25 10:47:38  dhcpd: DHCPOFFER on 10.10.10.230 to dc:a6:32:27:76:bc via 10.10.10.254
Nov 25 10:47:59  dhcpd: DHCPDISCOVER from dc:a6:32:27:76:bc via 10.10.10.254
Nov 25 10:47:59  dhcpd: DHCPOFFER on 10.10.10.230 to dc:a6:32:27:76:bc via 10.10.10.254
Nov 25 10:48:07  dhcpd: DHCPDISCOVER from dc:a6:32:27:76:bc via 10.10.10.254
Nov 25 10:48:07  dhcpd: DHCPOFFER on 10.10.10.230 to dc:a6:32:27:76:bc via 10.10.10.254
Nov 25 10:48:23  dhcpd: DHCPDISCOVER from dc:a6:32:27:76:bc via 10.10.10.254
Nov 25 10:48:23  dhcpd: DHCPOFFER on 10.10.10.230 to dc:a6:32:27:76:bc via 10.10.10.254
ghollingworth commented 4 years ago

Where are you trying to capture those packets from? Are you doing that on the DHCP server itself?

timg236 commented 4 years ago

I think you might need to allow UDP port 67 as well as 68, or filter by MAC address instead

timg236 commented 4 years ago

Please could you retry with the latest beta ?

Mavyre commented 4 years ago

Hi,

I'm answering to you @timg236 as I'm running into the exact same issue with a Raspberry Pi 4 and a Synology's DHCP server. Note that it works perfectly on a Raspberry pi 3B+. I'm this the following log using full tcpdump. The boot works fine if I flash the EEPROM with the TFTP server's IP address directly though. But I'd like it not to be hardcoded into the EEPROM...

21:39:02.787538 IP (tos 0x0, ttl 64, id 20627, offset 0, flags [none], proto UDP (17), length 350)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from dc:a6:32:2a:ad:db, length 322, xid 0xd7910b3d, Flags [none]
      Client-Ethernet-Address dc:a6:32:2a:ad:db
      Vendor-rfc1048 Extensions
        Magic Cookie 0x63825363
        DHCP-Message Option 53, length 1: Discover
        Parameter-Request Option 55, length 14: 
          Subnet-Mask, Default-Gateway, Vendor-Option, Vendor-Class
          TFTP, BF, Option 128, Option 129
          Option 130, Option 131, Option 132, Option 133
          Option 134, Option 135
        ARCH Option 93, length 2: 0
        NDI Option 94, length 3: 1.2.1
        GUID Option 97, length 17: 0.195.120.127.194.195.120.127.194.195.120.127.194.195.120.127.194
        Vendor-Class Option 60, length 32: "PXEClient:Arch:00000:UNDI:002001"
21:39:10.787356 IP (tos 0x0, ttl 64, id 20628, offset 0, flags [none], proto UDP (17), length 350)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from dc:a6:32:2a:ad:db, length 322, xid 0xd7910b3d, Flags [none]
      Client-Ethernet-Address dc:a6:32:2a:ad:db
      Vendor-rfc1048 Extensions
        Magic Cookie 0x63825363
        DHCP-Message Option 53, length 1: Discover
        Parameter-Request Option 55, length 14: 
          Subnet-Mask, Default-Gateway, Vendor-Option, Vendor-Class
          TFTP, BF, Option 128, Option 129
          Option 130, Option 131, Option 132, Option 133
          Option 134, Option 135
        ARCH Option 93, length 2: 0
        NDI Option 94, length 3: 1.2.1
        GUID Option 97, length 17: 0.195.120.127.194.195.120.127.194.195.120.127.194.195.120.127.194
        Vendor-Class Option 60, length 32: "PXEClient:Arch:00000:UNDI:002001"
ibot3 commented 4 years ago

From raspberrypi.org:

Hello everybody,

I successfully netbooted a RPi4 Model B with a dnsmasq PXE Server.

However, the same setup does not seem to work when trying to boot via a DHCP relay. The Pi sends out DHCP Discovers and always receives the related Offer. But nothing more, no DHCP request, no TFTP requests. After some retries, the Pi gives up.

Does someone have an idea why the Pi does nothing when receiving the DHCP offer?

DHCP DISCOVER packet: https://gist.github.com/ibot3/f9ffda7b8dd95e799f0e0fd76b476af2 DHCP OFFER packet: https://gist.github.com/ibot3/be19ec69914210cd225413c91290c91c

I have the problem described above when using a DHCP relay. I also attached the wireshark captures of the packets.

timg236 commented 4 years ago

There's no explicit support for relay agents so this would have to be transparent to the client Pi. I'd recommend updating to https://github.com/raspberrypi/rpi-eeprom/blob/master/firmware/beta/pieeprom-2020-03-19.bin and enabling BOOT_UART and capturing the log either via a USB serial or UART on another Pi. That should tell you whether client Pi sees the DHCP OFFER packet

ibot3 commented 4 years ago

I tried it with the newest version (2020-04-09) an got this output after three DISCOVER/OFFER iterations:

PM_RSTS: 0x00001000
RPi: BOOTLOADER release VERSION:fffd67bc DATE: Apr  9 2020 TIME: 17:55:43 BOOTMODE: 0x00000006 part: 0 BUILD_TIMESTAMP=1586451341 0xfdda8717 0x00b03111
uSD voltage 3.3V
...
Failed to open device: 'sdcard'
SD card not detected
SD boot failed
Initialising SDRAM 'Micron' 16Gb x1 total-size: 16 Gbit 3200
GENET: RESET_PHY
NET_BOOT: dc:a6:32:23:28:46 wait for link TFTP: 0.0.0.0
LINK STATUS: speed: 1000 full duplex
GENET START: 64 16 32
GENET: UMAC_START 0xdca63223 0x28460000
RX: 0 IP: 0 IPV4: 0 MAC: 0 UDP: 0 UDP RECV: 0 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0
RX: 30 IP: 0 IPV4: 4 MAC: 2 UDP: 2 UDP RECV: 2 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0
RX: 30 IP: 0 IPV4: 4 MAC: 2 UDP: 2 UDP RECV: 2 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0
RX_DISCARD: 0
  GR64: 0x00000001   GR64: 0x00000015  GR127: 0x00000000  GR255: 0x00000008 
 GR511: 0x00000000 GR1023: 0x00000000 GR1518: 0x00000000  GRMGV: 0x00000000 
GR2047: 0x00000000 GR4095: 0x00000000 GR9216: 0x0000001e  GRPKT: 0x0000143f 
 GRBYT: 0x0000001b  GRMCA: 0x00000001  GRBCA: 0x00000000  GRFCS: 0x00000000 
 GRXCF: 0x00000000  GRXPF: 0x00000000  GRXUO: 0x00000000  GRALN: 0x00000000 
 GRFLR: 0x00000000  GRCDE: 0x00000000  GRFCR: 0x00000000  GROVR: 0x00000000 
 GRJBR: 0x00000000 GRMTUE: 0x0000001e  GRPOK: 0x00000002   GRUC: 0x00000000 
 GRPPP: 0x00000000  GRCRC: 0x00000000 

  TR64: 0x00000000  TR127: 0x00000000  TR255: 0x00000000  TR511: 0x00000003 
TR1023: 0x00000000 TR1518: 0x00000000  TRMGV: 0x00000000 TR2047: 0x00000000 
TR4095: 0x00000000 TR9216: 0x00000000  GTPKT: 0x00000003  GTMCA: 0x00000000 
 GTBCA: 0x00000003  GTXPF: 0x00000000  GTXCF: 0x00000000  GTFCS: 0x00000000 
 GTOVR: 0x00000000  GTDRF: 0x00000000  GTEDF: 0x00000000  GTSCL: 0x00000000 
 GTMCL: 0x00000000  GTLCL: 0x00000000  GTXCL: 0x00000000  GTFRG: 0x00000000 
 GTNCL: 0x00000000  GTJBR: 0x00000000  GTBYT: 0x00000450  GTPOK: 0x00000003 
  GTUC: 0x00000000 
GENET STOP: 0
NETBOOT CANCEL
NETBOOT init failed
NET_BOOT: dc:a6:32:23:28:46 wait for link TFTP: 0.0.0.0
LINK STATUS: speed: 1000 full duplex
GENET START: 64 16 32
GENET: UMAC_START 0xdca63223 0x28460000
RX: 30 IP: 0 IPV4: 4 MAC: 2 UDP: 2 UDP RECV: 2 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0
RX: 62 IP: 0 IPV4: 10 MAC: 5 UDP: 5 UDP RECV: 5 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0
RX: 62 IP: 0 IPV4: 10 MAC: 5 UDP: 5 UDP RECV: 5 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0
RX_DISCARD: 0
  GR64: 0x00000003   GR64: 0x00000017  GR127: 0x00000000  GR255: 0x00000006 
 GR511: 0x00000000 GR1023: 0x00000000 GR1518: 0x00000000  GRMGV: 0x00000000 
GR2047: 0x00000000 GR4095: 0x00000000 GR9216: 0x00000020  GRPKT: 0x000013ad 
 GRBYT: 0x0000001b  GRMCA: 0x00000002  GRBCA: 0x00000000  GRFCS: 0x00000000 
 GRXCF: 0x00000000  GRXPF: 0x00000000  GRXUO: 0x00000000  GRALN: 0x00000000 
 GRFLR: 0x00000000  GRCDE: 0x00000000  GRFCR: 0x00000000  GROVR: 0x00000000 
 GRJBR: 0x00000000 GRMTUE: 0x00000020  GRPOK: 0x00000003   GRUC: 0x00000000 
 GRPPP: 0x00000000  GRCRC: 0x00000000 

  TR64: 0x00000000  TR127: 0x00000000  TR255: 0x00000000  TR511: 0x00000003 
TR1023: 0x00000000 TR1518: 0x00000000  TRMGV: 0x00000000 TR2047: 0x00000000 
TR4095: 0x00000000 TR9216: 0x00000000  GTPKT: 0x00000003  GTMCA: 0x00000000 
 GTBCA: 0x00000003  GTXPF: 0x00000000  GTXCF: 0x00000000  GTFCS: 0x00000000 
 GTOVR: 0x00000000  GTDRF: 0x00000000  GTEDF: 0x00000000  GTSCL: 0x00000000 
 GTMCL: 0x00000000  GTLCL: 0x00000000  GTXCL: 0x00000000  GTFRG: 0x00000000 
 GTNCL: 0x00000000  GTJBR: 0x00000000  GTBYT: 0x00000450  GTPOK: 0x00000003 
  GTUC: 0x00000000 
GENET STOP: 0
NETBOOT CANCEL
NETBOOT init failed
NET_BOOT: dc:a6:32:23:28:46 wait for link TFTP: 0.0.0.0
LINK STATUS: speed: 1000 full duplex
GENET START: 64 16 32
GENET: UMAC_START 0xdca63223 0x28460000
RX: 62 IP: 0 IPV4: 10 MAC: 5 UDP: 5 UDP RECV: 5 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0

However, I don't really understand what's going on there :)

timg236 commented 4 years ago

The copy from the serial output is a little mangled but these are statistics from the software about the number of packets received for each layer

IPV4: 10 MAC: 5 UDP: 5 UDP RECV: 5 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0 e.g. 10 IPV4 packets with 5 UDP packets no checksum errors

The GR values are dumps from the ethernet registers.

It looks as though the link is up.

The absence of any DHCP logs suggests that either it didn't receive the packets, wrong address / port.

I'd suggest using PiServer to validate with a known DHCP/TFTP server and comparing Wireshark logs https://hackaday.com/2019/11/11/network-booting-the-pi-4/

ibot3 commented 4 years ago

I corrected the output in my previous post to look better and more complete.

https://gist.github.com/ibot3/a4f168820bb956da1819770262f20b1a This is the DHCP OFFER that the Pi is receiving on its link. AFAICS it is looking good (except that it is over a DHCP relay). If I connect the Pi to my home router which does simple DHCP without any PXE, DHCP logs start to appear on the serial console.

The same setup is working when the Pi and the dnsmasq server are in the same L2 network. Therefore the dnsmasq config can't be too wrong:

dhcp-range=set:local-mgmt-zw41,10.160.41.5,10.160.41.62,255.255.255.192,10.160.41.63,12h
dhcp-option=tag:local-mgmt-zw41,option:router,10.160.41.1
dhcp-host=10.160.41.1

dhcp-option=tag:user-pi,66,10.128.1.5

dhcp-host=dc:a6:32:23:28:46,set:user-pi,user-pi-zw41,10.160.41.20,12h

pxe-service=tag:user-pi,0,"Raspberry Pi Boot"
timg236 commented 4 years ago

Thanks, I can't see anything obviously wrong with that although it's not the raw binary and I can't tell if the checksums are correct, packets with incorrect checksums are discarded. The ports and mac-address look correct which would be things which could cause the packet to be discarded very early on.

It's not practical for me to duplicate network setups right not (that's also very time consuming) but if I get some spare time I could look at adding an EEPROM config to control the verbosity of network debug. This can be in a separate GPIO enabled section to allow very verbose debug e.g. dump all packets back from the netboot state machine.

I'm doing something similar for USB-MSD interop tests.

ibot3 commented 4 years ago

Thanks. I updated the gist and enabled checksum validation and appended the raw hex at the end.

I also created a bash script for you which sets an complete test environment up using network namespaces. dnsmasq must be installed. For wireshark to diplay correct checksums, the optional commands with ethtool must be executed.

Just change the physical interface at the top to the interface where you want to connect the pi. The only difference to my setup that I encountered is, that the dnsmasq dhcp relay evaluates the dhcp broadcast flag and is therefore sending the OFFER as a unicast. But this doesn't change anything on the current behavior.

https://gist.github.com/ibot3/2615bd5ae1164b32a0f7610b60755f3d

timg236 commented 4 years ago

Changed the title to indicate this bug is specifically about adding support for DHCP relay agents

timg236 commented 4 years ago

Thanks for the script, I'll add support for relay agents to the todo list after the MSD BOOT beta is released (hopefully quite soon)

ibot3 commented 4 years ago

@timg236 You added the "awaiting information" label. Do you need any further information or is this related to an internal process?

timg236 commented 4 years ago

As I understand it a DHCP relay should be completely invisible to the client device so the question is what's different about the packets. The difference may well be something which is valid but not handled by the Pi but nobody has had a chance to do the A-B comparison.

ibot3 commented 4 years ago

The only real difference is that the "Relay agent IP address" is set. Can't you just use the script I wrote and check why the Pi doesn't see the offer packets?

And is there some option already to increase the verbosity of the network debug log?

timg236 commented 4 years ago

It's on the todo list but not going to get chance to look at this any time soon

ibot3 commented 4 years ago

Is there any chance that I can help or something? This is a blocker for our project, where we want to deploy around 30-50 Pis. All locations only get dhcp via dhcp relay. And dhcp relay isn't a totally rare use case..

timg236 commented 4 years ago

If you have a binary capture containing the offer packets then that's quick to confirm in the unit tests. Snowed under with USB stuff at the moment

ibot3 commented 4 years ago

The hex capture was appended to the gist. I created a new gist with the hex and binary stream: https://gist.github.com/ibot3/f8b77b8ad930c1ca1dfb8d43614802d5

ghollingworth commented 4 years ago

I'm guessing Tim requires a pcap file?

timg236 commented 4 years ago

I missed the hex earlier, the pcap is always the best but I normally export that to hex for the unit test. Will take a look once I escape from the lab :)

timg236 commented 4 years ago

Got it, there was some overzealous code that required the HOPS field to be zero. A DHCP relay agent may change this. I've attached an EEPROM image with the quick fix which I plan to roll into the next beta release.

If you could give this a quick test in the next couple of days that would be great.

pieeprom-c35a7d498.zip

ibot3 commented 4 years ago

Yes, this image is working fine.

Thank you!

timg236 commented 4 years ago

Thanks @ibot3 I'll close this bug when the beta with this fix is released. We are currently trying to close out any blockers which would prevent the current USB MSD beta from going to stable and then production so this change should make it to critical/production reasonably soon.

timg236 commented 4 years ago

Resolved in pieeprom-2020-06-15