[chassis] packet forwarding issue on Clearwater2ms

[arlakshm]

On some ports the packet forwarding on the clearwater2ms card doesn't work. When we do ping from the peer device, on some port the ping fails. On these ports the packets are received or transmitted on the phy. The link is up on the ASIC and peer-device.

Some debugs we collected during debug session lua(slice 0) > /serdes info +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | SerDes Params | Index | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | Param | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | rx_adapt | 2 | 2 | 5 | 5 | 2 | 2 | 4 | 4 | 2 | 2 | 2 | 2 | 1 | 1 | 1 | 1 | 3 | 3 | 3 | 3 | 1 | 1 | 1 | 1 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | rx_agcgain | 16 | 23 | 9 | 18 | 20 | 25 | 24 | 28 | 34 | 30 | 27 | 46 | 49 | 61 | 62 | 36 | 29 | 29 | 36 | 38 | 57 | 40 | 40 | 27 | | | 31 | 31 | 31 | 23 | 31 | 31 | 31 | 31 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | rx_channel_est | 1.64 | 1.61 | 1.66 | 1.5 | 1.74 | 1.7 | 1.7 | 1.64 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | rx_channel_hf | 28 | 28 | 29 | 30 | 27 | 27 | 27 | 28 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | rx_channel_of | 46 | 45 | 48 | 45 | 47 | 46 | 46 | 46 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | rx_ctle | 6 | 6 | 6 | 4 | 6 | 6 | 3 | 6 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | | | 3 | 3 | 3 | 6 | 3 | 3 | 5 | 3 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | rx_dac | 15 | 15 | 8 | 9 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 14 | 13 | 14 | 14 | 15 | 15 | 15 | 15 | | rx_delta | -4 | -2 | 3 | 3 | -6 | -4 | 0 | -4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | rx_dfe | -0 | -0 | 0.766 | 0.728 | -0 | -0 | -0 | -0 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 5 | 4 | 4 | 5 | 4 | 4 | 4 | 4 | | | -0 | -0 | 0.22 | 0.241 | -0 | -0 | -0 | -0 | -4 | -4 | -4 | -4 | -4 | -4 | -4 | -4 | -6 | -8 | -6 | -7 | -4 | -4 | -4 | -4 | | | 0 | 0 | 0.288 | 0.331 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 3 | 1 | 3 | 0 | 0 | 0 | 0 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | rx_edge | 39321 | 39321 | 39321 | 39321 | 43690 | 43690 | 43690 | 39321 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | rx_eye_height | 215 | 206 | 211 | 206 | 198 | 198 | 215 | 206 | 735 | 654 | 639 | 682 | 723 | 777 | 715 | 678 | 689 | 678 | 693 | 688 | 798 | 681 | 713 | 737 | | | 258 | 241 | 234 | 241 | 232 | 232 | 232 | 241 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | | 223 | 206 | 211 | 208 | 206 | 198 | 206 | 215 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | rx_f1over3 | 5 | 5 | 5 | 4 | 5 | 5 | 5 | 5 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | rx_ffe_taps | 115 | 151 | 115 | 234 | 122 | 120 | 120 | 135 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | | | 94 | 63 | 55 | -6 | 19 | 77 | 75 | 122 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | | | 1 | 1 | 1 | 122 | 95 | 1 | 1 | 1 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | | | 3 | 1 | 5 | -165 | -4 | 2 | -26 | -24 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | | | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | | | 1 | 1 | 1 | 4 | 4 | 1 | 1 | 1 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | | | 136 | 136 | 136 | 139 | 136 | 136 | 136 | 136 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | rx_graycode | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | rx_input_coupling | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | | rx_linklost | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | | rx_msblsb | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | rx_pll_cap | 27 | 27 | 27 | 27 | 27 | 27 | 27 | 27 | 31 | 31 | 31 | 31 | 31 | 32 | 31 | 31 | 31 | 31 | 31 | 31 | 31 | 31 | 31 | 32 | | rx_pol | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | rx_ppm | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -35 | -35 | -35 | -35 | -35 | -35 | -35 | -35 | -40 | -40 | -35 | -40 | -35 | -35 | -35 | -35 | | rx_precoder | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | rx_readapt | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | rx_ready | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | | rx_signal_detect | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | | rx_skef_addcap | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | | rx_skef_degen | 4 | 2 | 4 | 2 | 4 | 4 | 1 | 3 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | | rx_skef_en | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | rx_skef_gain | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | | tx_graycode | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | tx_msblsb | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | tx_pll_cap | 27 | 27 | 27 | 27 | 27 | 27 | 27 | 27 | 31 | 31 | 31 | 32 | 32 | 32 | 31 | 31 | 31 | 31 | 31 | 32 | 32 | 32 | 32 | 32 | | tx_pol | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | tx_precoder | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+ | tx_taps | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | | -5 | -5 | -5 | -5 | -5 | -5 | -5 | -5 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | | | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | | | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | | | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+

crlua(slice 0) crlua(slice 0) > /lane status +---------+------+---------+----------+---------------+------------+ | Lane | Mode | Speed | LB Mode | Tx State | Rx State | | | | | | | SD RDY ADP | +---------+------+---------+----------+---------------+------------+ | A0(0) | PAM4 | 53.125G | DISABLED | FORCE_DISABLE | 0 0 0 | | A1(1) | PAM4 | 53.125G | DISABLED | FORCE_DISABLE | 0 0 0 | | A2(2) | PAM4 | 53.125G | DISABLED | TRAFFIC | 1 1 1 | | A3(3) | PAM4 | 53.125G | DISABLED | TRAFFIC | 1 1 1 | | A4(4) | PAM4 | 53.125G | DISABLED | FORCE_DISABLE | 0 0 0 | | A5(5) | PAM4 | 53.125G | DISABLED | FORCE_DISABLE | 0 0 0 | | A6(6) | PAM4 | 53.125G | DISABLED | FORCE_DISABLE | 0 0 0 | | A7(7) | PAM4 | 53.125G | DISABLED | FORCE_DISABLE | 0 0 0 | | A8(8) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 | | A9(9) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 | | A10(10) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 | | A11(11) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 | | B0(12) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 | | B1(13) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 | | B2(14) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 | | B3(15) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 | | B4(16) | NRZ | 25.781G | DISABLED | TRAFFIC | 1 1 1 | | B5(17) | NRZ | 25.781G | DISABLED | TRAFFIC | 1 1 1 | | B6(18) | NRZ | 25.781G | DISABLED | TRAFFIC | 1 1 1 | | B7(19) | NRZ | 25.781G | DISABLED | TRAFFIC | 1 1 1 | | B8(20) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 | | B9(21) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 | | B10(22) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 | | B11(23) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 | +---------+------+---------+----------+---------------+------------+

crlua(slice 0) >

[arlakshm]

@kenneth-arista created this issue to track the clearwater2ms issue

[kenneth-arista]

Quick update. A new Credo SAI release is forthcoming to address this issue.

[byu343]

A new version v0.9.3 has been released. It can fix this issue from my testing.

[kenneth-arista]

Closing this issue as v0.9.3 Credo SAI should address this issue.