Does the UDP client connection need to be closed are re-initialised for each new IP address?

swinster commented 2 years ago

Hey,

I was scanning the Internat for a native PS way to run an ARP scan and came across your blog and hence GitHub repo. I thought "great, this is exactly what I need". I even thought "awesome" when I noted one of your examples was to pipe in the interface that has been assigned the default gateway, so ignoring VNets and the like. However, on running Find-LANHosts with a set of IP addresses, I noted something odd.

The returned list from the arp cache was a lot smaller than I anticipated. Running a Wireshark scan after flushing the ARP cache, I noted that relatively few ARP requests were being sent. Slow the function down by running it through the VS Code debugger, it seems that seven though you are looping through the IPs in the relevant array, the Connect method only connects to the first IP that was used after the Udpclient was initiated. Hence, I only see an ARP request for the first IP in the array.

The function then only returns items that are already in the arp cache or makes it to the catch before it completes.

Based on https://docs.microsoft.com/en-us/dotnet/api/system.net.sockets.udpclient?view=net-6.0, I think that you need to call the Close method of the Udpclient, then initialise a new Udpclient for each IP address.

As an example, I think this does the trick (but I'm no expert here, so creating and closing Udpclients like this might be inefficient :) )

<#
    .SYNOPSIS
    Quickly finds host on a local network using ARP for discovery

    .DESCRIPTION
    Uses ARP requests to determine whether a host is present on a network segment.

    As APR is a Layer 2 mechanism, the list of IP addesses need to be on the same network as the device running the script.

    .PARAMETER IP
    Optional. Specifies one or more IP addresses to scan for. Typically this will be a list of all usable hosts on a network.

    .PARAMETER NetAdapter
    Optional. Specifies one or more NetAdaper (CimInstance) objects from Get-NetAdapter. These interfaces will have attached subnets detected and used for the scan.

    If both the IP and NetAdapter parameters are omitted, all network adapters will be enumerated and local subnets automatically determined. This may require elevated priviliges.
    Please note that this can include adapters with very high host counts (/16, etc) which will take considerable time to enumerate.

    .PARAMETER DelayMS
    Optional. Specifies the interpacket delay, default is 2ms. Can be increased if scanning unreliable or high latency networks.

    .PARAMETER ClearARPCache
    Optional. Clears the ARP cache before starting a scan. This is recommended, but may require elevated priviliges.

    .EXAMPLE
    Find-LANHosts

    .EXAMPLE
    Find-LANHosts -ClearARPCache -DelayMS 5

    .EXAMPLE
    Get-NetAdapter -Name Ethernet | Find-LANHosts

    .EXAMPLE
    Get-NetAdapter | ? {($_ | Get-NetIPAddress -AddressFamily IPv4 -ErrorAction SilentlyContinue) -ne $null} | Find-LANHosts

    .EXAMPLE
    Get-NetRoute -DestinationPrefix 0.0.0.0/0 | Get-NetAdapter | Find-LANHosts

    .EXAMPLE
    $IPs = 1..254 | % {"10.250.1.$_"}
    Find-LANHosts -IP $IPs

    .EXAMPLE
    1..254 | % {"192.168.1.$_"} | Find-LANHosts -ClearARPCache

    .EXAMPLE
    1..254 | % {"10.1.1.$_"} | Find-LANHosts -DelayMS 5

    .LINK
    https://github.com/mdjx/PSLANScan
#>
function Find-LANHosts {
    [Cmdletbinding(DefaultParameterSetName = "IPBlock")]

    Param (
        [Parameter(Mandatory = $false, ValueFromPipeline, ParameterSetName = "IPBlock")]
        [string[]]$IP = $null,

        [Parameter(Mandatory = $false, ValueFromPipeline, ParameterSetName = "Interface")]
        [CimInstance[]]$NetAdapter = $null,

        [Parameter(Mandatory = $false, Position = 2)]
        [ValidateRange(0, 15000)]
        [int]$DelayMS = 2,

        [switch]$ClearARPCache
    )

    Begin {

        $ASCIIEncoding = New-Object System.Text.ASCIIEncoding
        $Bytes = $ASCIIEncoding.GetBytes("!")
        #$UDP = New-Object System.Net.Sockets.Udpclient

        if ($ClearARPCache) {
            $ARPClear = arp -d 2>&1
            if (($ARPClear.count -gt 0) -and ($ARPClear[0] -is [System.Management.Automation.ErrorRecord]) -and ($ARPClear[0].Exception -notmatch "The parameter is incorrect")) {
                Throw $ARPClear[0].Exception
            }
        }

        $IPList = [System.Collections.ArrayList]@()
        $Timer = [System.Diagnostics.Stopwatch]::StartNew()
        Write-Verbose "Beginning scan"
    }

    Process {

        if (($null -eq $IP) -and ($null -eq $NetAdapter)) {
            if ($VerbosePreference -eq "SilentlyContinue") { [array]$IP = Get-IPs -ReturnIntRange }
            else {[array]$IP = Get-IPs -ReturnIntRange -Verbose }
        }

        if ($PsCmdlet.ParameterSetName -eq "Interface") {
            if ($VerbosePreference -eq "SilentlyContinue") {[array]$IP = Get-IPs -NetAdapter $NetAdapter -ReturnIntRange }
            else { [array]$IP = Get-IPs -NetAdapter $NetAdapter -ReturnIntRange -Verbose }
        }

        if ($IP.Count -lt 1) {
            Write-Error "IP Count is less than 1, please check provided IPs or Adapter for valid address space"
        }

        if ($null -ne $IP.FirstIPInt) {
            $IP | ForEach-Object {
                $CurrentIPInt = $_.FirstIPInt
                Do {
                    $UDP = New-Object System.Net.Sockets.Udpclient
                    $CurrIP = [IPAddress]$CurrentIPInt
                    $CurrIP = ($CurrIP).GetAddressBytes()
                    [Array]::Reverse($CurrIP)
                    $CurrIP = ([IPAddress]$CurrIP).IPAddressToString
                    #$UDP.Connect($CurrIP, 1)
                    #[void]$UDP.Send($Bytes, $Bytes.length)
                    [void]$UDP.Send($Bytes, $Bytes.length, $CurrIP, 1)
                    [void]$IPList.Add($CurrIP)
                    if ($DelayMS) {
                        [System.Threading.Thread]::Sleep($DelayMS)
                    }

                    $CurrentIPInt++
                    $UDP.Close()
                } While ($CurrentIPInt -le $_.LastIPInt)
            }
        }
        else {
            $IP | ForEach-Object {
                $UDP = New-Object System.Net.Sockets.Udpclient
                #$UDP.Connect($_, 1)
                #[void]$UDP.Send($Bytes, $Bytes.length)
                [void]$UDP.Send($Bytes, $Bytes.length, $_, 1)
                [void]$IPList.Add($_)
                if ($DelayMS) {
                    [System.Threading.Thread]::Sleep($DelayMS)
                }
                $UDP.Close()
            }
        }
    }

    End {
        $Hosts = arp -a
        $Timer.Stop()

        if ($Timer.Elapsed.TotalSeconds -gt 15) {
            Write-Warning "Scan took longer than 15 seconds, ARP entries may have been flushed. Recommend lowering DelayMS parameter"
        }

        $Hosts = $Hosts | Where-Object { $_ -match "dynamic" } | % { ($_.trim() -replace " {1,}", ",") | ConvertFrom-Csv -Header "IP", "MACAddress" }
        $Hosts = $Hosts | Where-Object { $_.IP -in $IPList }

        Write-Output $Hosts
    }
}

Find-LANHosts -IP $IPs

swinster commented 2 years ago

Just in case there is anything weird going on here, I tested this on

Name                           Value
----                           -----
PSVersion                      7.2.5
PSEdition                      Core
GitCommitId                    7.2.5
OS                             Microsoft Windows 10.0.22000
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

and

Name                           Value
----                           -----
PSVersion                      5.1.22000.653
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.22000.653
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

swinster commented 2 years ago

Ah ha, better still, you don't need to create a new Udpclient object each time, you just use the Send method. Looks like if you call the Connect method, you are stuck using the same remote host (see https://docs.microsoft.com/en-us/dotnet/api/system.net.sockets.udpclient.connect?view=net-6.0#system-net-sockets-udpclient-connect(system-net-ipaddress-system-int32) ).

So, this is back to initialising the client object once, closing it out in the end, and only using the send method (no default connect).


<#
    .SYNOPSIS
    Quickly finds host on a local network using ARP for discovery

    .DESCRIPTION
    Uses ARP requests to determine whether a host is present on a network segment.

    As APR is a Layer 2 mechanism, the list of IP addesses need to be on the same network as the device running the script.

    .PARAMETER IP
    Optional. Specifies one or more IP addresses to scan for. Typically this will be a list of all usable hosts on a network.

    .PARAMETER NetAdapter
    Optional. Specifies one or more NetAdaper (CimInstance) objects from Get-NetAdapter. These interfaces will have attached subnets detected and used for the scan.

    If both the IP and NetAdapter parameters are omitted, all network adapters will be enumerated and local subnets automatically determined. This may require elevated priviliges.
    Please note that this can include adapters with very high host counts (/16, etc) which will take considerable time to enumerate.

    .PARAMETER DelayMS
    Optional. Specifies the interpacket delay, default is 2ms. Can be increased if scanning unreliable or high latency networks.

    .PARAMETER ClearARPCache
    Optional. Clears the ARP cache before starting a scan. This is recommended, but may require elevated priviliges.

    .EXAMPLE
    Find-LANHosts

    .EXAMPLE
    Find-LANHosts -ClearARPCache -DelayMS 5

    .EXAMPLE
    Get-NetAdapter -Name Ethernet | Find-LANHosts

    .EXAMPLE
    Get-NetAdapter | ? {($_ | Get-NetIPAddress -AddressFamily IPv4 -ErrorAction SilentlyContinue) -ne $null} | Find-LANHosts

    .EXAMPLE
    Get-NetRoute -DestinationPrefix 0.0.0.0/0 | Get-NetAdapter | Find-LANHosts

    .EXAMPLE
    $IPs = 1..254 | % {"10.250.1.$_"}
    Find-LANHosts -IP $IPs

    .EXAMPLE
    1..254 | % {"192.168.1.$_"} | Find-LANHosts -ClearARPCache

    .EXAMPLE
    1..254 | % {"10.1.1.$_"} | Find-LANHosts -DelayMS 5

    .LINK
    https://github.com/mdjx/PSLANScan
#>
function Find-LANHosts {
    [Cmdletbinding(DefaultParameterSetName = "IPBlock")]

    Param (
        [Parameter(Mandatory = $false, ValueFromPipeline, ParameterSetName = "IPBlock")]
        [string[]]$IP = $null,

        [Parameter(Mandatory = $false, ValueFromPipeline, ParameterSetName = "Interface")]
        [CimInstance[]]$NetAdapter = $null,

        [Parameter(Mandatory = $false, Position = 2)]
        [ValidateRange(0, 15000)]
        [int]$DelayMS = 2,

        [switch]$ClearARPCache
    )

    Begin {

        $ASCIIEncoding = New-Object System.Text.ASCIIEncoding
        $Bytes = $ASCIIEncoding.GetBytes("!")
        $UDP = New-Object System.Net.Sockets.Udpclient

        if ($ClearARPCache) {
            $ARPClear = arp -d 2>&1
            if (($ARPClear.count -gt 0) -and ($ARPClear[0] -is [System.Management.Automation.ErrorRecord]) -and ($ARPClear[0].Exception -notmatch "The parameter is incorrect")) {
                Throw $ARPClear[0].Exception
            }
        }

        $IPList = [System.Collections.ArrayList]@()
        $Timer = [System.Diagnostics.Stopwatch]::StartNew()
        Write-Verbose "Beginning scan"
    }

    Process {

        if (($null -eq $IP) -and ($null -eq $NetAdapter)) {
            if ($VerbosePreference -eq "SilentlyContinue") { [array]$IP = Get-IPs -ReturnIntRange }
            else {[array]$IP = Get-IPs -ReturnIntRange -Verbose }
        }

        if ($PsCmdlet.ParameterSetName -eq "Interface") {
            if ($VerbosePreference -eq "SilentlyContinue") {[array]$IP = Get-IPs -NetAdapter $NetAdapter -ReturnIntRange }
            else { [array]$IP = Get-IPs -NetAdapter $NetAdapter -ReturnIntRange -Verbose }
        }

        if ($IP.Count -lt 1) {
            Write-Error "IP Count is less than 1, please check provided IPs or Adapter for valid address space"
        }

        if ($null -ne $IP.FirstIPInt) {
            $IP | ForEach-Object {
                $CurrentIPInt = $_.FirstIPInt
                Do {
                    #$UDP = New-Object System.Net.Sockets.Udpclient
                    $CurrIP = [IPAddress]$CurrentIPInt
                    $CurrIP = ($CurrIP).GetAddressBytes()
                    [Array]::Reverse($CurrIP)
                    $CurrIP = ([IPAddress]$CurrIP).IPAddressToString
                    #$UDP.Connect($CurrIP, 1)
                    #[void]$UDP.Send($Bytes, $Bytes.length)
                    [void]$UDP.Send($Bytes, $Bytes.length, $CurrIP, 1)
                    [void]$IPList.Add($CurrIP)
                    if ($DelayMS) {
                        [System.Threading.Thread]::Sleep($DelayMS)
                    }

                    $CurrentIPInt++
                    #$UDP.Close()
                } While ($CurrentIPInt -le $_.LastIPInt)
            }
        }
        else {
            $IP | ForEach-Object {
                #$UDP = New-Object System.Net.Sockets.Udpclient
                #$UDP.Connect($_, 1)
                #[void]$UDP.Send($Bytes, $Bytes.length)
                [void]$UDP.Send($Bytes, $Bytes.length, $_, 1)
                [void]$IPList.Add($_)
                if ($DelayMS) {
                    [System.Threading.Thread]::Sleep($DelayMS)
                }
                #$UDP.Close()
            }
        }
    }

    End {
        $UDP.Close()
        $Hosts = arp -a
        $Timer.Stop()

        if ($Timer.Elapsed.TotalSeconds -gt 15) {
            Write-Warning "Scan took longer than 15 seconds, ARP entries may have been flushed. Recommend lowering DelayMS parameter"
        }

        $Hosts = $Hosts | Where-Object { $_ -match "dynamic" } | % { ($_.trim() -replace " {1,}", ",") | ConvertFrom-Csv -Header "IP", "MACAddress" }
        $Hosts = $Hosts | Where-Object { $_.IP -in $IPList }

        Write-Output $Hosts
    }
}

mdjx commented 2 years ago

Hi

So I think there's a few things going on. Firstly, with the latest v7.2 build I can confirm the lack of ARP packets you're seeing and it is due to the Connect method. Bypassing this is cleaner and fixes the issue.

However, I can't reproduce any of it in v5.1 - that still works perfectly fine for me with the current 1.2.0 code.

Obviously truncated but it goes all the way to 254 as expected.

You mentioned VNets so I tested in an Azure VM as well and saw the same thing, all expected ARP packets were sent.

Also confirmed that updating the connect method does work and it's not stuck on the first IP, but again, only in v5,1.

Happy to release an updated version with the removal of Connect - does that work as expected with v5.1 on your systems?

swinster commented 2 years ago

WRT VNets, I actually meant virtual network adapters installed locally via HyperV, VMware or the like :)

I will test more across different PS versions shortly.

mdjx / PSLANScan

Does the UDP client connection need to be closed are re-initialised for each new IP address? #1