zhangyumingsh
commented
4 months ago Due to high packet loss on BTT Pi v1.2 (exactly the same behavior as follows), I switched to BTT Pi2.
I am running the latest version of system on the BTT Pi2, namely the CB2_Debian11_minimal_kernel4.19_20240619.img.xz
I used Netplan to assign a static IP: Netplan config files:
network: version: 2 renderer: NetworkManager ethernets: eth0: dhcp4: no addresses: [192.168.1.199/24] gateway4: 192.168.1.1 nameservers: addresses: [192.168.1.1, 223.5.5.5]And I am getting huge packet drops on eth0, the onboard Ethernet. Here's some ping tests over the course of roughly 10 mins:
Pinging the BTT Pi2 (192.168.1.199) from my computer (192.168.1.5)
Pinging the main router (192.168.1.1, running OpenWRT, x86 Intel system) from BTT Pi2 (192.168.1.199)
Network topology: Main Router -> 2.5GbE Switch -> 2.5GbE Switch ->BTT Pi2
Here's some outputs of command lines:
root@Hachiroku:~# dmesg | grep -i eth [ 0.000000] psci: probing for conduit method from DT. [ 2.284113] rk_gmac-dwmac fe010000.ethernet: Looking up phy-supply from device tree [ 2.284143] rk_gmac-dwmac fe010000.ethernet: phy regulator is not available yet, deferred probing [ 2.286201] usbcore: registered new interface driver cdc_ether [ 3.275477] rk_gmac-dwmac fe010000.ethernet: Looking up phy-supply from device tree [ 3.275547] rk_gmac-dwmac fe010000.ethernet: Linked as a consumer to regulator.27 [ 3.275553] rk_gmac-dwmac fe010000.ethernet: clock input or output? (input). [ 3.275558] rk_gmac-dwmac fe010000.ethernet: TX delay(0x19). [ 3.275562] rk_gmac-dwmac fe010000.ethernet: RX delay(0x5). [ 3.275569] rk_gmac-dwmac fe010000.ethernet: integrated PHY? (no). [ 3.275603] rk_gmac-dwmac fe010000.ethernet: clock input from PHY [ 3.280614] rk_gmac-dwmac fe010000.ethernet: init for RGMII [ 3.280805] rk_gmac-dwmac fe010000.ethernet: User ID: 0x30, Synopsys ID: 0x51 [ 3.280812] rk_gmac-dwmac fe010000.ethernet: DWMAC4/5 [ 3.280817] rk_gmac-dwmac fe010000.ethernet: DMA HW capability register supported [ 3.280821] rk_gmac-dwmac fe010000.ethernet: RX Checksum Offload Engine supported [ 3.280823] rk_gmac-dwmac fe010000.ethernet: TX Checksum insertion supported [ 3.280826] rk_gmac-dwmac fe010000.ethernet: Wake-Up On Lan supported [ 3.280854] rk_gmac-dwmac fe010000.ethernet: TSO supported [ 3.280857] rk_gmac-dwmac fe010000.ethernet: Enable RX Mitigation via HW Watchdog Timer [ 3.280862] rk_gmac-dwmac fe010000.ethernet: TSO feature enabled [ 3.420669] YT8531 Gigabit Ethernet stmmac-1:00: [phy_probe-1989] led_status_value normal [ 12.854724] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready [ 13.096759] YT8531 Gigabit Ethernet stmmac-1:00: attached PHY driver [YT8531 Gigabit Ethernet] (mii_bus:phy_addr=stmmac-1:00, irq=POLL) [ 13.114158] rk_gmac-dwmac fe010000.ethernet eth0: No Safety Features support found [ 13.114181] rk_gmac-dwmac fe010000.ethernet eth0: IEEE 1588-2008 Advanced Timestamp supported [ 13.120794] rk_gmac-dwmac fe010000.ethernet eth0: registered PTP clock [ 13.121532] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready [ 16.173233] rk_gmac-dwmac fe010000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx [ 16.173311] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [ 132.705917] rk_gmac-dwmac fe010000.ethernet eth0: Link is Down [ 133.719396] rk_gmac-dwmac fe010000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/txroot@Hachiroku:~# netstat -s Ip: Forwarding: 2 102780 total packets received 3 with invalid addresses 0 forwarded 0 incoming packets discarded 102403 incoming packets delivered 89739 requests sent out 42 dropped because of missing route Icmp: 2241 ICMP messages received 0 input ICMP message failed ICMP input histogram: destination unreachable: 65 echo requests: 1500 echo replies: 676 2373 ICMP messages sent 0 ICMP messages failed ICMP output histogram: destination unreachable: 67 echo requests: 806 echo replies: 1500 IcmpMsg: InType0: 676 InType3: 65 InType8: 1500 OutType0: 1500 OutType3: 67 OutType8: 806 Tcp: 85 active connection openings 74 passive connection openings 0 failed connection attempts 106 connection resets received 1 connections established 98993 segments received 174744 segments sent out 5887 segments retransmitted 0 bad segments received 88 resets sent Udp: 1137 packets received 67 packets to unknown port received 0 packet receive errors 216 packets sent 0 receive buffer errors 0 send buffer errors IgnoredMulti: 283 UdpLite: TcpExt: 18 TCP sockets finished time wait in fast timer 323 delayed acks sent 20 delayed acks further delayed because of locked socket Quick ack mode was activated 85 times 21899 packet headers predicted 22631 acknowledgments not containing data payload received 52355 predicted acknowledgments TCPSackRecovery: 3850 1 congestion windows recovered without slow start after partial ack TCPLostRetransmit: 795 3 timeouts in loss state 4767 fast retransmits 64 retransmits in slow start TCPTimeouts: 607 TCPLossProbes: 467 TCPLossProbeRecovery: 238 TCPSackRecoveryFail: 414 TCPDSACKOldSent: 85 TCPDSACKRecv: 137 71 connections reset due to unexpected data 1 connections reset due to early user close TCPDSACKIgnoredNoUndo: 42 TCPSpuriousRTOs: 1 TCPSackShifted: 1247 TCPSackMerged: 1873 TCPSackShiftFallback: 6872 TCPRcvCoalesce: 3628 TCPOFOQueue: 9 TCPSpuriousRtxHostQueues: 90 TCPAutoCorking: 3925 TCPFromZeroWindowAdv: 13 TCPToZeroWindowAdv: 13 TCPWantZeroWindowAdv: 81 TCPOrigDataSent: 158565 TCPACKSkippedSeq: 5 TCPWinProbe: 19 TCPDelivered: 158153 IpExt: InMcastPkts: 790 OutMcastPkts: 126 InBcastPkts: 283 InOctets: 150789572 OutOctets: 299309607 InMcastOctets: 146236 OutMcastOctets: 34243 InBcastOctets: 114600 InNoECTPkts: 102846root@Hachiroku:~# ethtool -S eth0 NIC statistics: mmc_tx_octetcount_gb: 340174588 mmc_tx_framecount_gb: 331037 mmc_tx_broadcastframe_g: 0 mmc_tx_multicastframe_g: 0 mmc_tx_64_octets_gb: 0 mmc_tx_65_to_127_octets_gb: 0 mmc_tx_128_to_255_octets_gb: 0 mmc_tx_256_to_511_octets_gb: 0 mmc_tx_512_to_1023_octets_gb: 0 mmc_tx_1024_to_max_octets_gb: 0 mmc_tx_unicast_gb: 0 mmc_tx_multicast_gb: 0 mmc_tx_broadcast_gb: 0 mmc_tx_underflow_error: 0 mmc_tx_singlecol_g: 0 mmc_tx_multicol_g: 0 mmc_tx_deferred: 0 mmc_tx_latecol: 0 mmc_tx_exesscol: 0 mmc_tx_carrier_error: 0 mmc_tx_octetcount_g: 340174588 mmc_tx_framecount_g: 331037 mmc_tx_excessdef: 0 mmc_tx_pause_frame: 0 mmc_tx_vlan_frame_g: 0 mmc_rx_framecount_gb: 162558 mmc_rx_octetcount_gb: 13586448 mmc_rx_octetcount_g: 13586448 mmc_rx_broadcastframe_g: 0 mmc_rx_multicastframe_g: 3456 mmc_rx_crc_error: 0 mmc_rx_align_error: 0 mmc_rx_run_error: 0 mmc_rx_jabber_error: 0 mmc_rx_undersize_g: 0 mmc_rx_oversize_g: 0 mmc_rx_64_octets_gb: 0 mmc_rx_65_to_127_octets_gb: 0 mmc_rx_128_to_255_octets_gb: 0 mmc_rx_256_to_511_octets_gb: 0 mmc_rx_512_to_1023_octets_gb: 0 mmc_rx_1024_to_max_octets_gb: 0 mmc_rx_unicast_g: 0 mmc_rx_length_error: 0 mmc_rx_autofrangetype: 0 mmc_rx_pause_frames: 0 mmc_rx_fifo_overflow: 0 mmc_rx_vlan_frames_gb: 0 mmc_rx_watchdog_error: 0 mmc_rx_ipc_intr_mask: 1417958598 mmc_rx_ipc_intr: 0 mmc_rx_ipv4_gd: 158088 mmc_rx_ipv4_hderr: 800 mmc_rx_ipv4_nopay: 0 mmc_rx_ipv4_frag: 0 mmc_rx_ipv4_udsbl: 0 mmc_rx_ipv4_gd_octets: 0 mmc_rx_ipv4_hderr_octets: 267214 mmc_rx_ipv4_nopay_octets: 0 mmc_rx_ipv4_frag_octets: 0 mmc_rx_ipv4_udsbl_octets: 0 mmc_rx_ipv6_gd_octets: 0 mmc_rx_ipv6_hderr_octets: 0 mmc_rx_ipv6_nopay_octets: 0 mmc_rx_ipv6_gd: 870 mmc_rx_ipv6_hderr: 0 mmc_rx_ipv6_nopay: 0 mmc_rx_udp_gd: 0 mmc_rx_udp_err: 0 mmc_rx_tcp_gd: 0 mmc_rx_tcp_err: 0 mmc_rx_icmp_gd: 0 mmc_rx_icmp_err: 0 mmc_rx_udp_gd_octets: 0 mmc_rx_udp_err_octets: 0 mmc_rx_tcp_gd_octets: 0 mmc_rx_tcp_err_octets: 0 mmc_rx_icmp_gd_octets: 0 mmc_rx_icmp_err_octets: 0 tx_underflow: 0 tx_carrier: 0 tx_losscarrier: 0 vlan_tag: 0 tx_deferred: 0 tx_vlan: 0 tx_jabber: 0 tx_frame_flushed: 0 tx_payload_error: 0 tx_ip_header_error: 0 rx_desc: 0 sa_filter_fail: 0 overflow_error: 0 ipc_csum_error: 0 rx_collision: 0 rx_crc_errors: 0 dribbling_bit: 0 rx_length: 0 rx_mii: 0 rx_multicast: 0 rx_gmac_overflow: 0 rx_watchdog: 0 da_rx_filter_fail: 0 sa_rx_filter_fail: 0 rx_missed_cntr: 0 rx_overflow_cntr: 0 rx_vlan: 0 tx_undeflow_irq: 0 tx_process_stopped_irq: 0 tx_jabber_irq: 0 rx_overflow_irq: 0 rx_buf_unav_irq: 0 rx_process_stopped_irq: 0 rx_watchdog_irq: 0 tx_early_irq: 0 fatal_bus_error_irq: 0 rx_early_irq: 10105 threshold: 1 tx_pkt_n: 66908 rx_pkt_n: 81373 normal_irq_n: 103547 rx_normal_irq_n: 38111 napi_poll: 111900 tx_normal_irq_n: 65011 tx_clean: 111900 tx_set_ic_bit: 66908 irq_receive_pmt_irq_n: 0 mmc_tx_irq_n: 0 mmc_rx_irq_n: 0 mmc_rx_csum_offload_irq_n: 0 irq_tx_path_in_lpi_mode_n: 0 irq_tx_path_exit_lpi_mode_n: 0 irq_rx_path_in_lpi_mode_n: 3 irq_rx_path_exit_lpi_mode_n: 3 phy_eee_wakeup_error_n: 0 ip_hdr_err: 0 ip_payload_err: 0 ip_csum_bypassed: 1415 ipv4_pkt_rcvd: 79515 ipv6_pkt_rcvd: 443 no_ptp_rx_msg_type_ext: 81373 ptp_rx_msg_type_sync: 0 ptp_rx_msg_type_follow_up: 0 ptp_rx_msg_type_delay_req: 0 ptp_rx_msg_type_delay_resp: 0 ptp_rx_msg_type_pdelay_req: 0 ptp_rx_msg_type_pdelay_resp: 0 ptp_rx_msg_type_pdelay_follow_up: 0 ptp_rx_msg_type_announce: 0 ptp_rx_msg_type_management: 0 ptp_rx_msg_pkt_reserved_type: 0 ptp_frame_type: 0 ptp_ver: 0 timestamp_dropped: 0 av_pkt_rcvd: 0 av_tagged_pkt_rcvd: 0 vlan_tag_priority_val: 0 l3_filter_match: 0 l4_filter_match: 0 l3_l4_filter_no_match: 0 irq_pcs_ane_n: 0 irq_pcs_link_n: 0 irq_rgmii_n: 0 mtl_tx_status_fifo_full: 0 mtl_tx_fifo_not_empty: 0 mmtl_fifo_ctrl: 0 mtl_tx_fifo_read_ctrl_write: 0 mtl_tx_fifo_read_ctrl_wait: 0 mtl_tx_fifo_read_ctrl_read: 0 mtl_tx_fifo_read_ctrl_idle: 0 mac_tx_in_pause: 0 mac_tx_frame_ctrl_xfer: 0 mac_tx_frame_ctrl_idle: 0 mac_tx_frame_ctrl_wait: 0 mac_tx_frame_ctrl_pause: 0 mac_gmii_tx_proto_engine: 0 mtl_rx_fifo_fill_level_full: 0 mtl_rx_fifo_fill_above_thresh: 0 mtl_rx_fifo_fill_below_thresh: 0 mtl_rx_fifo_fill_level_empty: 0 mtl_rx_fifo_read_ctrl_flush: 0 mtl_rx_fifo_read_ctrl_read_data: 0 mtl_rx_fifo_read_ctrl_status: 0 mtl_rx_fifo_read_ctrl_idle: 0 mtl_rx_fifo_ctrl_active: 0 mac_rx_frame_ctrl_fifo: 0 mac_gmii_rx_proto_engine: 0 tx_tso_frames: 9225 tx_tso_nfrags: 11905I ran the iperf test, 192.168.1.234 is a Windows machine running the iperf3 server.
root@Hachiroku:~# iperf3 -c 192.168.1.234 -p 5201 Connecting to host 192.168.1.234, port 5201 [ 5] local 192.168.1.199 port 40630 connected to 192.168.1.234 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.42 MBytes 11.9 Mbits/sec 10 58.5 KBytes [ 5] 1.00-2.00 sec 1.10 MBytes 9.25 Mbits/sec 4 1.43 KBytes [ 5] 2.00-3.00 sec 2.08 MBytes 17.5 Mbits/sec 6 58.5 KBytes [ 5] 3.00-4.00 sec 941 KBytes 7.71 Mbits/sec 4 58.5 KBytes [ 5] 4.00-5.00 sec 627 KBytes 5.14 Mbits/sec 6 58.5 KBytes [ 5] 5.00-6.00 sec 1.76 MBytes 14.8 Mbits/sec 8 58.5 KBytes [ 5] 6.00-7.00 sec 376 KBytes 3.08 Mbits/sec 4 1.43 KBytes [ 5] 7.00-8.00 sec 68.4 KBytes 560 Kbits/sec 5 5.70 KBytes [ 5] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec 5 8.55 KBytes [ 5] 9.00-10.00 sec 55.6 KBytes 456 Kbits/sec 2 11.4 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 8.39 MBytes 7.04 Mbits/sec 54 sender [ 5] 0.00-10.00 sec 8.25 MBytes 6.92 Mbits/sec receiver iperf Done.Here's a screenshot of the wireshark on my computer while SSHing.
Also, here's some good amount of outputs of the
tcpdump -i eth0 -n -vtcpdump.txtI also tested like this: BTT Pi2 -> My computer, which still gives me around 20% packet loss.
THANKFULLY, the USB ethernet dongle (Realtek 8153) works and is able to provide a somewhat decent experience with no packet loss.
I have no idea what happened to the onboard ethernet on both the PI2 and PI v1.2.
Check if the network cable is properly plugged in
ChesterHachirokuJamie
frutabruta
bigtreetech
Sotiocsi
smwoodward
kunday
Due to high packet loss on BTT Pi v1.2 (exactly the same behavior as follows), I switched to BTT Pi2.
I am running the latest version of system on the BTT Pi2, namely the CB2_Debian11_minimal_kernel4.19_20240619.img.xz
I used Netplan to assign a static IP: Netplan config files:
And I am getting huge packet drops on eth0, the onboard Ethernet. Here's some ping tests over the course of roughly 10 mins:
Pinging the BTT Pi2 (192.168.1.199) from my computer (192.168.1.5)
Pinging the main router (192.168.1.1, running OpenWRT, x86 Intel system) from BTT Pi2 (192.168.1.199)
Network topology: Main Router -> 2.5GbE Switch -> 2.5GbE Switch ->BTT Pi2
Here's some outputs of command lines:
I ran the iperf test, 192.168.1.234 is a Windows machine running the iperf3 server.
Here's a screenshot of the wireshark on my computer while SSHing.
Also, here's some good amount of outputs of the
tcpdump -i eth0 -n -vtcpdump.txtI also tested like this: BTT Pi2 -> My computer, which still gives me around 20% packet loss.
THANKFULLY, the USB ethernet dongle (Realtek 8153) works and is able to provide a somewhat decent experience with no packet loss.
I have no idea what happened to the onboard ethernet on both the PI2 and PI v1.2.