search

firedancer-io / firedancer

Firedancer is Jump Crypto's Solana consensus node implementation.
https://firedancer.io
Other
803 stars 140 forks source link

Tracking: Jumbo Ethernet frames (>2K packet size) #2045

Open ripatel-fd opened 1 month ago

ripatel-fd commented 1 month ago

Firedancer support for Ethernet frames exceeding 2048 bytes requires some more work.

The maximum frame size in AF_XDP is 4096 bytes. (Probably due to some unfortunate kernel page size and DMA restrictions). Larger frames require fragmentation.

In AF_XDP this feature is called multi-buffer support. In Firedancer, this feature is already built into fd_tango via the fd_frag_meta_t's control bits (SOM/EOM).

One could trivially translate XSK XDP_PKT_CONTD flags into fd_tango SOM/EOM in the net tile. Downstream tiles like netmux and QUIC server need further polishing in order to handle large packets correctly.

ripatel-fd commented 1 month ago

Should probably also mention: Why? We don't know whether jumbo frames work reliably across any peer-to-peer connection. The Firedancer team generally opposes coalescing of user packets into larger IP packets. We would need path MTU discovery (so far also supported in QUIC). The only use case so far seems to be larger transaction sizes.