who is supposed to set interface MTU?

[elenril]

I am trying to set up jumbo frames on (some subnets of) my home LAN (in order to reduce the router load for large transfers) and I'm struggling to understand how to set the interface MTU without manually hardcoding it on every machine. Apparently dhcpcd used to have a hook that did this, but it was removed in ca6cdf5847cda720b65793ea6827b1b373c62382, with the replacement code only setting route MTU, which apparently has no effect if the interface MTU is smaller.

Any hints on how the interface MTU is supposed to be set up properly? Hardcoding it is annoying and error-prone, ideally something would just pick it up from RAs. Or are there any major disadvantages to that?

[rsmarples]

The problem is that a lot of interfaces reset their PHY on MTU change whether through driver error or a hardware limitation. This in turn triggers carrier down/up which isn't exactly desirable. The other side effect is when different protocols want different MTU values (strikes me as weird, but it's how some setups work). As such dhcpcd just changes the MTU at the route level as you now see.

The downside is that this won't allow jumbo frames. There is also the issue where you change network from jumbo frames to non jumbo frames and there's nothing on the new network to say what the MTU size should be, expecting the interface default to just work.

I suppose we could add an option to increase the interface MTU if the procotol MTU is bigger and then reset it back to what it was when the carrier goes down OR dhcpcd exits. This would likely default to off though due the resetting PHY problem.

[elenril]

First of all, thank you for the reply. It's so hard to find any reliable information on this - it seems very few people use jumbo frames, even though in my experience they are supported by most current consumer NICs.

Quoting Roy Marples (2021-10-29 13:03:47)

The problem is that a lot of interfaces reset their PHY on MTU change whether through driver error or a hardware limitation. This in turn triggers carrier down/up which isn't exactly desirable.

Right, the Realtek NIC on my desktop machine does that. It's annoying, but as long as it only happens once per boot it should not be a big deal.

The other side effect is when different protocols want different MTU values (strikes me as weird, but it's how some setups work). As such dhcpcd just changes the MTU at the route level as you now see.

The downside is that this won't allow jumbo frames. There is also the issue where you change network from jumbo frames to non jumbo frames and there's nothing on the new network to say what the MTU size should be, expecting the interface default to just work.

I suppose we could add an option to increase the interface MTU if the procotol MTU is bigger and then reset it back to what it was when the carrier goes down OR dhcpcd exits. This would likely default to off though due the resetting PHY problem.

That sounds reasonable to me. I don't expect this to work automagically, toggling an option is still a big improvement over manually setting the exact MTU value everywhere.

-- Anton Khirnov

[carsten-AEI]

As we are currently having issues with isc-dhcp-client, we looked at dhcpcd as a drop-in replacement for our largish cluster (Debian 11/Bullseye). We run several relatively static networks per machine and the "fast" 1/10/40Gbit/s network has been using L2 jumbo frames with MTU 9000 for 15 years now.

The DHCP server returns the MTU size for each client/interface (option 26 IIRC) and thus it would be nice to have a way for the local DHCP client to react to that "properly". Even if this was just an opt-in option, we would be very grateful for the addition :)

Anything from our side, we can help to get this feature?

[rsmarples]

Sorry for the late reply.

You can create a hook script to set MTU based on the value returned by new_mtu and compare it to interface_mtu and optionally old_mtu. It setting the MTU resets the PHY and causes a carrier reset you will need to add this to /etc/dhcpcd.conf

interface eth0
    nolink