Format-JsonOrder -SortAlphabetically -Recursive => "You must specify an object for the Get-Member cmdlet." & "You cannot call a method on a null-valued expression."

[kaiaschulz]

Hey, I would love to use your tool to get a sorted JSON output. My hashtable $htResult is huge, if I am trying to sort it with your module, I got tons of error logs which keep repeating:

Without Recursive-parameter, it seems to work. With that, only the main key of the hashtable got sorted correctly, but I would like to have the other sub-keys sorted as well.

$htResult | ConvertTo-Json -Depth 99 | Format-JsonOrder -SortAlphabetically -Recursive |  Out-File "C:\temp\results.json"

Get-Member: C:\Users\<userName>\OneDrive\Documents\PowerShell\Modules\SortJson\1.0.4\Functions\Set-ObjectPropertyOrder.ps1:52:54
Line |
  52 |  …               $PropertyType = ($object.$property | Get-Member) | Sele …
     |                                                       ~~~~~~~~~~
     | You must specify an object for the Get-Member cmdlet.
InvalidOperation: C:\Users\<userName>\OneDrive\Documents\PowerShell\Modules\SortJson\1.0.4\Functions\ConvertTo-OrderObject.ps1:145:25
Line |
 145 |  …             Write-Verbose ("{0} :: Adding property value: {1}" -f ${C …
     |                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | You cannot call a method on a null-valued expression.

[PowerShellCrack]

you would happen to have a sample of this large json or a script I can use to generate a large one? could you share a section of it that is not working so I can look into what is going on?

[kaiaschulz]

getVirtualNetwork_29-04-2024_16-32-20.json Hey @PowerShellCrack , to recude the complexety I have a static array for the supported APIversions. The script is iterating all APIversions and locations in parallel for this specific resource provider and trying to list all virtual networks within the subscriuption XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXX.

here is the script:

#Requires -Modules @{ ModuleName="SortJson"; ModuleVersion="1.0.4" }, @{ ModuleName="Az.Resources"; ModuleVersion="6.16.2" }, @{ ModuleName="Az.Accounts"; ModuleVersion="2.19.0" }

[CmdletBinding()]
param (
    [Parameter()]
    [string]$ResourceId = '/subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXX/providers/Microsoft.Network/virtualNetworks',
    [int]$ThrottleLimit = 20,
    [array]$APIVersions = @('2014-12-01-preview', '2015-05-01-preview', '2015-06-15', '2016-03-30', '2016-06-01', '2016-07-01', '2016-08-01', '2016-09-01', '2016-10-01', '2016-11-01', '2016-12-01', '2017-03-01', '2017-04-01', '2017-06-01', '2017-08-01', '2017-09-01', '2017-10-01', '2017-11-01', '2018-01-01', '2018-02-01', '2018-03-01', '2018-04-01', '2018-05-01', '2018-06-01', '2018-07-01', '2018-08-01', '2018-10-01', '2018-11-01', '2018-12-01', '2019-02-01', '2019-04-01', '2019-06-01', '2019-07-01', '2019-08-01', '2019-09-01', '2019-11-01', '2019-12-01', '2020-01-01', '2020-03-01', '2020-04-01', '2020-05-01', '2020-06-01', '2020-07-01', '2020-08-01', '2020-11-01', '2021-01-01', '2021-02-01', '2021-03-01', '2021-04-01', '2021-05-01', '2021-06-01', '2021-08-01', '2021-12-01', '2022-01-01', '2022-05-01', '2022-07-01', '2022-09-01', '2022-11-01', '2023-02-01', '2023-04-01', '2023-05-01', '2023-06-01', '2023-09-01', '2023-11-01')
)

try {
    $null = Connect-AzAccount -ErrorAction Stop -Verbose
}
catch {
    Write-Warning '0f2cdd51-355f-4934-ae98-24592e94c165'
    Throw $_
}

try {
    $getAzLocation = Get-AzLocation -ErrorAction Stop -Verbose | Sort-Object Location -Unique
}
catch {
    Write-Warning '557d7ce1-1940-4229-81ed-ed494371294e'
    Throw $_
}

try {
    $bearerToken = $(Get-AzAccessToken -ResourceUrl 'https://management.azure.com' -ErrorAction Stop).Token
}
catch {
    Write-Warning '1fb70d99-2758-48b8-986e-f1c8d0d2fbb1'
    Throw $_
}

$header = @{
    'Content-Type'  = 'application/json'
    'Authorization' = "Bearer $bearerToken"
}

$htResult = [System.Collections.Hashtable]::Synchronized((New-Object System.Collections.Hashtable))

$htCounterAPIVersions = [System.Collections.Hashtable]::Synchronized((New-Object System.Collections.Hashtable))
$htCounterAPIVersions['Value'] = 0

try {
    $startTime = Get-Date -ErrorAction Stop
}
catch {
    Write-Warning 'a9486449-0cb9-48e4-bda3-270d668c8639'
    Throw $_
}

# foreach ($apiVersion in $APIVersions) {
$APIVersions | Foreach-Object -ThrottleLimit $ThrottleLimit -Parallel {
    $apiVersion = $_

    $APIVersions = $using:APIVersions
    $ResourceId = $using:ResourceId
    $header = $using:header
    $getAzLocation = $using:getAzLocation
    $htResult = $using:htResult
    $htCounterAPIVersions = $using:htCounterAPIVersions
    $ThrottleLimit = $using:ThrottleLimit

    $htCounterAPIVersions['Value']++
    $counterAPIVersions = $htCounterAPIVersions['Value']

    Write-Host "APIVersions: $counterAPIVersions of $($APIVersions.Count) - $apiVersion" -BackgroundColor DarkCyan

    $htCounterLocations = [System.Collections.Hashtable]::Synchronized((New-Object System.Collections.Hashtable))
    $htCounterLocations['Value'] = 0

    $htResult.$apiVersion = @{}

    # foreach ($location in $getAzLocation.Location) {
    $getAzLocation.Location | Foreach-Object -ThrottleLimit $ThrottleLimit -Parallel {
        $location = $_

        $apiVersion = $using:apiVersion
        $APIVersions = $using:APIVersions
        $ResourceId = $using:ResourceId
        $header = $using:header
        $getAzLocation = $using:getAzLocation
        $htResult = $using:htResult
        $counterAPIVersions = $using:counterAPIVersions
        $htCounterAPIVersions = $using:htCounterAPIVersions
        $htCounterLocations = $using:htCounterLocations

        $htCounterLocations['Value']++
        $counterLocations = $htCounterLocations['Value']

        Write-Host "APIVersions: $counterAPIVersions of $($APIVersions.Count) - Locations: $counterLocations of $($getAzLocation.Location.Count) - $location" -BackgroundColor DarkCyan

        try {
            # ? https://learn.microsoft.com/en-us/rest/api/virtualnetwork/virtual-networks/get?view=rest-virtualnetwork-2023-09-01&tabs=HTTP
            # ? GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}?api-version=2023-09-01
            # ? GET https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Network/virtualNetworks?api-version=2023-09-01
            $uri = "https://$location.management.azure.com$($ResourceId)?api-version=$apiVersion"
            $invokeWebRequest = Invoke-WebRequest -Uri $uri -Headers $header -ErrorAction Stop
            $invokeWebRequest = $invokeWebRequest.Content | ConvertFrom-Json

            if ($invokeWebRequest.value.Count -eq 1) {
                Write-Host "APIVersions: $counterAPIVersions of $($APIVersions.Count) - Locations: $counterLocations of $($getAzLocation.Location.Count) - $location - $($invokeWebRequest.value.Count)" -ForegroundColor Green
            }
            else {
                Write-Host "APIVersions: $counterAPIVersions of $($APIVersions.Count) - Locations: $counterLocations of $($getAzLocation.Location.Count) - $location - $($invokeWebRequest.value.Count)" -ForegroundColor Red
            }

            $htResults = @{
                Location   = $location
                APIVersion = $apiVersion
                Uri        = $uri
                ResourceId = $ResourceId
                Result     = $invokeWebRequest.value
                Count      = $invokeWebRequest.value.Count
            }
        }
        catch {
            if ($_.Exception.Message -match 'No such host is known.') {
                Write-Warning '1fb70d99-2758-48b8-986e-f1c8d0d2fbb1'
                Write-Warning "APIVersions: $counterAPIVersions of $($APIVersions.Count) - Locations: $counterLocations of $($getAzLocation.Location.Count) - $location - No such host is known."

                $htResults = @{
                    Location     = $location
                    APIVersion   = $apiVersion
                    Uri          = $uri
                    ResourceId   = $ResourceId
                    Count        = $invokeWebRequest.value.Count
                    ErrorCode    = '1fb70d99-2758-48b8-986e-f1c8d0d2fbb1'
                    ErrorMessage = 'No such host is known.'
                }
            }
            else {
                Write-Warning '7e2262c8-416e-4a85-9a12-e6e419626c22'

                Write-Warning "Location: $location"
                Write-Warning "ApiVersion: $apiVersion"
                Write-Warning "Uri: $uri"
                Write-Warning "ResourceId: $ResourceId"

                $htResults = @{
                    Location     = $location
                    APIVersion   = $apiVersion
                    Uri          = $uri
                    ResourceId   = $ResourceId
                    Count        = $invokeWebRequest.value.Count
                    ErrorCode    = '7e2262c8-416e-4a85-9a12-e6e419626c22'
                    ErrorMessage = 'No such host is known.'
                }

                Write-Error $_ -ErrorAction Continue
            }
        }
        $htResult.$apiVersion.$location = $htResults
    }
}

try {
    $endTime = Get-Date -ErrorAction Stop
}
catch {
    Write-Warning 'adb4cfad-4f62-41ce-a753-0774fb283f58'
    Throw $_
}

Write-Host "The iteration took $(($endTime - $startTime).TotalSeconds) seconds."

try {
    # $htResult | ConvertTo-Json -Depth 99 | Out-File "C:\temp\getVirtualNetwork_$(Get-Date -Format 'dd-MM-yyy_HH-mm-ss').json"
    $htResult | ConvertTo-Json -Depth 99 | Format-JsonOrder -SortAlphabetically | Out-File "C:\temp\getVirtualNetwork_$(Get-Date -Format 'dd-MM-yyy_HH-mm-ss').json" # ? https://github.com/PowerShellCrack/PSSortJsonModule
}
catch {
    Write-Warning '936225e2-a624-490a-8c4e-029318a1b748'
    Throw $_
}

Hopefully this helps to reproduce the issue and find a way to fix it.

Cheers, Kai

[PowerShellCrack]

Ok so I did run this using the script you provided. Wow it created 61mb json! Anyway, what I did was output the $htResult as a Json with the sort options and without. I got no errors on output, but I am seeing that it's not sorting certain nodes...

Here is small snippet of what it looks like when not sorted.

If I run it with no -Recursive, it will only sort the first node...as you see here the api dates are ordered but not the results underneath

If you use the --Recursive, it orders the results and any other sub nodes. I sorted the first three rows but not the properties row. however it did for the subnet node sub items but again not it properties.

I will have to break this down to see why that is happening.

In your comment, your receiving errors when running -Recursive btu you are getting errors. Which version are you running? Are you running in Posh 7 or 5.1?

[kaiaschulz]

Hey @PowerShellCrack, thanks for having a look.

Quiet interesting that you don't encounter the error which I saw. Is this might related to the additional SortAlphabetically-parameter I used, or is this just a default param?

In your examples it seems like that the problems are starting with the Result-level. This is an JSON array and the array [] contains JSON object(s) {}. Is this might causing the problem or should it work as well?

I am using Virtual Studio Code (VSC) with the official powershell extention (ms-vscode.powershell).

get-command Format-JsonOrder

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Function        Format-JsonOrder                                   1.0.4      SortJson

$PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.4.2
PSEdition                      Core
GitCommitId                    7.4.2
OS                             Microsoft Windows 10.0.22621
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[kaiaschulz]

Hey @PowerShellCrack, have you already had the chance to check it out?

[PowerShellCrack]

@kaiaschulz I believe I fixed the issue; what I found was it was erroring at null arrays in the json and causing ordering to stop. I have tested it and it seems to work on my side. Please test it out for me and let me know

[kaiaschulz]

Superb @PowerShellCrack, this time it is working as expected. Will give it further tries on huge json-files. Thanks a lot 👍