microsoft / SDN

This repo includes PowerShell scripts and VMM service templates for setting up the Microsoft Software Defined Networking (SDN) Stack using Windows Server 2016
Other
486 stars 541 forks source link

New-HNSNetwork changes the MTU size from 1500 to 1450 in AKS Windows VMs #517

Open AbelHu opened 2 years ago

AbelHu commented 2 years ago

It seems like that someone has observed it in https://raw.githubusercontent.com/pjh/SDN/mtu-with-new-hnsnetwork/mtu-with-new-hnsnetwork.txt.

#####################################################################
## The just-created L2Bridge type HNS network subsumes the previous
## Ethernet interface with the "vEthernet (Ethernet) 2" interface
## which for some reason has an MTU of 1450 rather than 1432 or 1500.
## Also, the MTU of the existing "vEthernet (nat)" interface has been
## decreased by 50 bytes!
#####################################################################

Reference: TCP/IP performance tuning for Azure VMs | Microsoft Docs

Azure and VM MTU
The default MTU for Azure VMs is 1,500 bytes. The Azure Virtual Network stack will attempt to fragment a packet at 1,400 bytes.
Note that the Virtual Network stack isn't inherently inefficient because it fragments packets at 1,400 bytes even though VMs have an MTU of 1,500. A large percentage of network packets are much smaller than 1,400 or 1,500 bytes.

Below is the command in provisioning AKS Windows nodes.

    $externalNetwork = "ext"
    $nas = @(Get-NetAdapter -Physical)

    foreach($na in $nas)
    {
        $netIP = Get-NetIPAddress -ifIndex $na.ifIndex -AddressFamily IPv4 -ErrorAction SilentlyContinue -ErrorVariable netIPErr
        if ($netIP)
        {
            $managementIP = $netIP.IPAddress
            $adapterName = $na.Name
            break
        }
        else {
            Write-Error "No IPv4 found on the network adapter $($na.Name); trying the next adapter ..."
            if ($netIPErr) {
                Write-Host "error when retrieving IPAddress: $netIPErr"
                $netIPErr.Clear()
            }
        }
    }
    if(-Not $managementIP)
    {
        throw "None of the physical network adapters has an IP address"
    }
    Write-Host "Using adapter $adapterName with IP address $managementIP"    
    New-HNSNetwork -Type "L2Bridge" -AddressPrefix "192.168.255.0/30" -Gateway "192.168.255.1" -AdapterName $adapterName -Name $externalNetwork -Verbose

Logs:

Getting MTU size

   MTU  MediaSenseState   Bytes In  Bytes Out  Interface
------  ---------------  ---------  ---------  -------------
4294967295                1          0          0  Loopback Pseudo-Interface 1
  1500                1          0       1988  vEthernet (nat)
  1500                1   35399353    1419013  Ethernet 2

2022-03-24T04:46:56.1914010+00:00: Creating new HNS network "ext"
2022-03-24T04:46:59.6000519+00:00: No IPv4 found on the network adapter Ethernet 3; trying the next adapter ...
2022-03-24T04:46:59.6000519+00:00: error when retrieving IPAddress: No matching MSFT_NetIPAddress objects found by CIM query for instances of the ROOT/StandardCimv2/MSFT_NetIPAddress class on the  CIM server: SELECT * FROM MSFT_NetIPAddress  WHERE ((InterfaceIndex = 17)) AND ((AddressFamily = 2)). Verify query parameters and retry.
2022-03-24T04:46:59.6156744+00:00: Using adapter Ethernet 2 with IP address 10.240.1.151
VERBOSE: Invoke-HNSRequest Method[POST] Path[/networks] Data[{

    "Subnets":  [

                    {

                        "GatewayAddress":  "192.168.255.1",

                        "AddressPrefix":  "192.168.255.0/30"

                    }

                ],

    "NetworkAdapterName":  "Ethernet 2",

    "Name":  "ext",

    "Type":  "L2Bridge"

}]
VERBOSE: Result :
{"Output":{"ActivityId":"F8905258-1CFD-4B88-95F8-072C9B663F39","AdditionalParams":{},"CurrentEndpointCount":0,"Extensio
ns":[{"Id":"E7C3B2F0-F3C5-48DF-AF2B-10FED6D72E7A","IsEnabled":false,"Name":"Microsoft Windows Filtering
Platform"},{"Id":"E9B59CFA-2BE1-4B21-828F-B6FBDBDDC017","IsEnabled":true,"Name":"Microsoft Azure VFP Switch
Extension"},{"Id":"EA24CD6C-D17A-4348-9190-09F0D5BE83DD","IsEnabled":true,"Name":"Microsoft NDIS
Capture"}],"Flags":0,"Health":{"LastErrorCode":0,"LastUpdateTime":132925708223917418},"ID":"7F4003A8-5E19-4414-990F-72B
19796E78E","IPv6":false,"LayeredOn":"798783E7-1A28-4CDD-9CDC-F438A4E30B46","MacPools":[{"EndMacAddress":"00-15-5D-9D-DF
-FF","StartMacAddress":"00-15-5D-9D-D0-00"}],"ManagementIP":"10.240.1.151","MaxConcurrentEndpoints":0,"Name":"ext","Net
workAdapterName":"Ethernet
2","Policies":[],"Resources":{"AdditionalParams":{},"AllocationOrder":0,"Health":{"LastErrorCode":0,"LastUpdateTime":13
2925708267108672},"ID":"F8905258-1CFD-4B88-95F8-072C9B663F39","PortOperationTime":0,"State":1,"SwitchOperationTime":0,"
VfpOperationTime":0,"parentId":"6C5D500F-C4C8-45CA-8BBB-730C5E13D338"},"State":1,"Subnets":[{"AdditionalParams":{},"Add
ressPrefix":"192.168.255.0/30","GatewayAddress":"192.168.255.1","Health":{"LastErrorCode":0,"LastUpdateTime":1329257082
23917418},"ID":"76E15ADC-3AE9-4DF7-90C4-5FF342E88901","ObjectType":5,"Policies":[],"State":0}],"TotalEndpoints":0,"Type
":"L2Bridge","Version":38654705669},"Success":true}

ActivityId             : F8905258-1CFD-4B88-95F8-072C9B663F39
AdditionalParams       :
CurrentEndpointCount   : 0
Extensions             : {@{Id=E7C3B2F0-F3C5-48DF-AF2B-10FED6D72E7A; IsEnabled=False; Name=Microsoft Windows Filtering
                         Platform}, @{Id=E9B59CFA-2BE1-4B21-828F-B6FBDBDDC017; IsEnabled=True; Name=Microsoft Azure
                         VFP Switch Extension}, @{Id=EA24CD6C-D17A-4348-9190-09F0D5BE83DD; IsEnabled=True;
                         Name=Microsoft NDIS Capture}}
Flags                  : 0
Health                 : @{LastErrorCode=0; LastUpdateTime=132925708223917418}
ID                     : 7F4003A8-5E19-4414-990F-72B19796E78E
IPv6                   : False
LayeredOn              : 798783E7-1A28-4CDD-9CDC-F438A4E30B46
MacPools               : {@{EndMacAddress=00-15-5D-9D-DF-FF; StartMacAddress=00-15-5D-9D-D0-00}}
ManagementIP           : 10.240.1.151
MaxConcurrentEndpoints : 0
Name                   : ext
NetworkAdapterName     : Ethernet 2
Policies               : {}
Resources              : @{AdditionalParams=; AllocationOrder=0; Health=; ID=F8905258-1CFD-4B88-95F8-072C9B663F39;
                         PortOperationTime=0; State=1; SwitchOperationTime=0; VfpOperationTime=0;
                         parentId=6C5D500F-C4C8-45CA-8BBB-730C5E13D338}
State                  : 1
Subnets                : {@{AdditionalParams=; AddressPrefix=192.168.255.0/30; GatewayAddress=192.168.255.1; Health=;
                         ID=76E15ADC-3AE9-4DF7-90C4-5FF342E88901; ObjectType=5; Policies=System.Object[]; State=0}}
TotalEndpoints         : 0
Type                   : L2Bridge
Version                : 38654705669

2022-03-24T04:47:06.7421215+00:00: It took 7 seconds to create the ext network.
Getting MTU size

   MTU  MediaSenseState   Bytes In  Bytes Out  Interface
------  ---------------  ---------  ---------  -------------
4294967295                1          0          0  Loopback Pseudo-Interface 1
  1450                1          0       1988  vEthernet (nat)
  1450                1       2676       3066  vEthernet (Ethernet 2)
AbelHu commented 2 years ago

We have closed the issue in AKS with "It is by design that HNS sets the MTU from 1500 to 1450 for process-isolated containers". It should be better if HNS can share more details for this design. Thanks.