Bug in Get-D365Module function as it only detects custom models if they were built at least once

[TrudAX]

A bug in Get-D365Module function as it only detects custom models if they were built at least once. For example, for a new VM, I still have to build all custom models from VS. Any idea how to fix that? This is related to #722

please note that Get-D365Model works fine in this case

[Splaxi]

We are depending on the CreateRuntimeProvider and the CreateDiskProvider classes.

https://github.com/d365collaborative/d365fo.tools/blob/d97b79f76efd44c3b318b1fc533441807ab2a258/d365fo.tools/functions/get-d365module.ps1#L172C13-L179

I think the issue is that we actually load the modules from the disk provider, but never appends them to the overall $modules variable. For now it only marks the module IsBinary (True/False) based on the below logic.

            $diskModules = $metadataProviderViaDisk.ModelManifest.ListModules()

            foreach($module in $modules) {
                if ($diskModules.Name -NotContains $module.Name) {
                    $module.IsBinary = $true
                }
            }
        }

Would it be possible for you to do some testing with the following code, directly on your machine?

function Import-AssemblyFileIntoMemory {
    [CmdletBinding()]
    [OutputType()]
    param (
        [Parameter(Mandatory = $true, Position = 1)]
        [string[]] $Path,

        [switch] $UseTempFolder
    )

    if (-not (Test-PathExists -Path $Path -Type Leaf)) {
        Stop-PSFFunction -Message "Stopping because unable to locate file." -StepsUpward 1
        return
    }

    foreach ($itemPath in $Path) {

        if ($UseTempFolder) {
            $filename = Split-Path -Path $itemPath -Leaf
            $shadowClonePath = Join-Path $env:TEMP "$filename`_shadow.dll"
        }
        else {
            $shadowClonePath = "$itemPath`_shadow.dll"
        }

        try {
            Write-PSFMessage -Level Debug -Message "Cloning $itemPath to $shadowClonePath"
            Copy-Item -Path $itemPath -Destination $shadowClonePath -Force

            Write-PSFMessage -Level Debug -Message "Loading $shadowClonePath into memory"
            $null = [AppDomain]::CurrentDomain.Load(([System.IO.File]::ReadAllBytes($shadowClonePath)))
        }
        catch {
            Write-PSFMessage -Level Host -Message "Something went wrong while working against the database" -Exception $PSItem.Exception
            Stop-PSFFunction -Message "Stopping because of errors"
            return
        }
        finally {
            Write-PSFMessage -Level Debug -Message "Removing $shadowClonePath"
            Remove-Item -Path $shadowClonePath -Force -ErrorAction SilentlyContinue
        }
    }
}

## Please set values in the below variables to match your machine ##
$BinDir = 'PATH-2-AOS-BIN-FOLDER'
$PackageDirectory = 'PATH-2-PACKAGE-DIRECTORY'

[System.Collections.ArrayList] $Files2Process = New-Object -TypeName "System.Collections.ArrayList"

$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Management.Delta.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Management.Diff.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Management.Merge.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Management.Core.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Core.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Storage.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.ApplicationPlatform.XppServices.Instrumentation.dll"))

Import-AssemblyFileIntoMemory -Path $($Files2Process.ToArray())

$diskProviderConfiguration = New-Object Microsoft.Dynamics.AX.Metadata.Storage.DiskProvider.DiskProviderConfiguration
$diskProviderConfiguration.AddMetadataPath($PackageDirectory)
$metadataProviderFactoryViaDisk = New-Object Microsoft.Dynamics.AX.Metadata.Storage.MetadataProviderFactory
$metadataProviderViaDisk = $metadataProviderFactoryViaDisk.CreateDiskProvider($diskProviderConfiguration)

$diskModules = $metadataProviderViaDisk.ModelManifest.ListModules()

foreach ($module in $diskModules ) {
    $module.Name
}

[TrudAX]

Thanks for the answer. The repro steps are quite simple - create a new model (I created AAATest), add a class to it(do not press save on the project to avoid compile). you get a folder in the package directory Then I run 2 commands 1. Get-D365Module -ExcludeBinaryModules -InDependencyOrder Actual result: - no AAATest in the list Expected result: AAATest should be in the list

2. Get-D365Model -ExcludeMicrosoftModels -CustomizableOnly Actual result: - no AAATest in the list Expected result: AAATest should be in the list

3. I run your code, and it generated an error, but then displayed all models with AAATest(that is correct). if I comment lines from 11 to 14, it just displayed all models with AAATest(that is correct). I am using 10.0.35

_Test-PathExists : The term 'Test-PathExists' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:11 char:15

• if (-not (Test-PathExists -Path $Path -Type Leaf)) {

• 
  + CategoryInfo          : ObjectNotFound: (Test-PathExists:String) [], CommandNotFoundException
  + FullyQualifiedErrorId : CommandNotFoundException_

[Splaxi]

So this confirms my suspicion - that we don't all the missing modules, while traversing the different providers.

I'll take a look in the near future - unless other people are up for the challenge.

[Splaxi]

Try this updated code:

function Import-AssemblyFileIntoMemory {
    [CmdletBinding()]
    [OutputType()]
    param (
        [Parameter(Mandatory = $true, Position = 1)]
        [string[]] $Path,

        [switch] $UseTempFolder
    )

    foreach ($itemPath in $Path) {

        if ($UseTempFolder) {
            $filename = Split-Path -Path $itemPath -Leaf
            $shadowClonePath = Join-Path $env:TEMP "$filename`_shadow.dll"
        }
        else {
            $shadowClonePath = "$itemPath`_shadow.dll"
        }

        try {
            Write-PSFMessage -Level Debug -Message "Cloning $itemPath to $shadowClonePath"
            Copy-Item -Path $itemPath -Destination $shadowClonePath -Force

            Write-PSFMessage -Level Debug -Message "Loading $shadowClonePath into memory"
            $null = [AppDomain]::CurrentDomain.Load(([System.IO.File]::ReadAllBytes($shadowClonePath)))
        }
        catch {
            Write-PSFMessage -Level Host -Message "Something went wrong while working against the database" -Exception $PSItem.Exception
            Stop-PSFFunction -Message "Stopping because of errors"
            return
        }
        finally {
            Write-PSFMessage -Level Debug -Message "Removing $shadowClonePath"
            Remove-Item -Path $shadowClonePath -Force -ErrorAction SilentlyContinue
        }
    }
}

## Please set values in the below variables to match your machine ##
$BinDir = Join-Path (Get-D365EnvironmentSettings).Common.BinDir "bin"
$PackageDirectory = (Get-D365EnvironmentSettings).Common.BinDir

[System.Collections.ArrayList] $Files2Process = New-Object -TypeName "System.Collections.ArrayList"

$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Management.Delta.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Management.Diff.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Management.Merge.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Management.Core.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Core.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Storage.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.ApplicationPlatform.XppServices.Instrumentation.dll"))

Import-AssemblyFileIntoMemory -Path $($Files2Process.ToArray())

$runtimeProviderConfiguration = New-Object Microsoft.Dynamics.AX.Metadata.Storage.Runtime.RuntimeProviderConfiguration -ArgumentList $PackageDirectory

$metadataProviderFactoryViaRuntime = New-Object Microsoft.Dynamics.AX.Metadata.Storage.MetadataProviderFactory
$metadataProviderViaRuntime = $metadataProviderFactoryViaRuntime.CreateRuntimeProvider($runtimeProviderConfiguration)

$modelManifest = $metadataProviderViaRuntime.ModelManifest

if ($InDependencyOrder -eq $true) {
    $modules = $modelManifest.ListModulesInDependencyOrder()
}
else {
    $modules = $modelManifest.ListModules() | Sort-Object -Property Name
}

$modules | ForEach-Object {
    $_ | Add-Member -MemberType NoteProperty -Name 'IsBinary' -Value $false
}

$diskProviderConfiguration = New-Object Microsoft.Dynamics.AX.Metadata.Storage.DiskProvider.DiskProviderConfiguration
$diskProviderConfiguration.AddMetadataPath($PackageDirectory)
$metadataProviderFactoryViaDisk = New-Object Microsoft.Dynamics.AX.Metadata.Storage.MetadataProviderFactory
$metadataProviderViaDisk = $metadataProviderFactoryViaDisk.CreateDiskProvider($diskProviderConfiguration)

$diskModules = $metadataProviderViaDisk.ModelManifest.ListModules() | Sort-Object -Property Name

foreach ($module in $modules) {
    if ($diskModules.Name -NotContains $module.Name) {
        $module.IsBinary = $true
    }
}

$uncompiledModules = @(
    foreach ($module in $diskModules) {
        if ($modules.Name -NotContains $module.Name) {
            $module | Add-Member -MemberType NoteProperty -Name 'IsBinary' -Value $false
            $module
        }
    }
)

# Combined both arrays
$modules = $modules + $uncompiledModules

$modules

[TrudAX]

This one looks good. The only problem is the "Readonly" column. Right now, it is TRUE for every model, even for the custom one, except AAATest Can you move this code to "Get-D365Module" and "Get-D365Model" ?

[Splaxi]

ReadOnly is not part of the output, when things are running from the correct cmdlets / functions in the PowerShell module. This was just raw prototyping...

[Splaxi]

Could you test this, for the models?

function Import-AssemblyFileIntoMemory {
    [CmdletBinding()]
    [OutputType()]
    param (
        [Parameter(Mandatory = $true, Position = 1)]
        [string[]] $Path,

        [switch] $UseTempFolder
    )

    foreach ($itemPath in $Path) {

        if ($UseTempFolder) {
            $filename = Split-Path -Path $itemPath -Leaf
            $shadowClonePath = Join-Path $env:TEMP "$filename`_shadow.dll"
        }
        else {
            $shadowClonePath = "$itemPath`_shadow.dll"
        }

        try {
            Write-PSFMessage -Level Debug -Message "Cloning $itemPath to $shadowClonePath"
            Copy-Item -Path $itemPath -Destination $shadowClonePath -Force

            Write-PSFMessage -Level Debug -Message "Loading $shadowClonePath into memory"
            $null = [AppDomain]::CurrentDomain.Load(([System.IO.File]::ReadAllBytes($shadowClonePath)))
        }
        catch {
            Write-PSFMessage -Level Host -Message "Something went wrong while working against the database" -Exception $PSItem.Exception
            Stop-PSFFunction -Message "Stopping because of errors"
            return
        }
        finally {
            Write-PSFMessage -Level Debug -Message "Removing $shadowClonePath"
            Remove-Item -Path $shadowClonePath -Force -ErrorAction SilentlyContinue
        }
    }
}

## Please set values in the below variables to match your machine ##
$BinDir = Join-Path (Get-D365EnvironmentSettings).Common.BinDir "bin"
$PackageDirectory = (Get-D365EnvironmentSettings).Common.BinDir

[System.Collections.ArrayList] $Files2Process = New-Object -TypeName "System.Collections.ArrayList"

$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Management.Delta.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Management.Diff.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Management.Merge.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Management.Core.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Core.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.AX.Metadata.Storage.dll"))
$null = $Files2Process.Add((Join-Path $BinDir "Microsoft.Dynamics.ApplicationPlatform.XppServices.Instrumentation.dll"))

Import-AssemblyFileIntoMemory -Path $($Files2Process.ToArray())

$runtimeProviderConfiguration = New-Object Microsoft.Dynamics.AX.Metadata.Storage.Runtime.RuntimeProviderConfiguration -ArgumentList $PackageDirectory
$metadataProviderFactoryViaRuntime = New-Object Microsoft.Dynamics.AX.Metadata.Storage.MetadataProviderFactory
$metadataProviderViaRuntime = $metadataProviderFactoryViaRuntime.CreateRuntimeProvider($runtimeProviderConfiguration)

$models = $metadataProviderViaRuntime.ModelManifest.ListModelInfos()
$models | ForEach-Object {
    $_ | Add-Member -MemberType NoteProperty -Name 'IsBinary' -Value $false
}

$diskProviderConfiguration = New-Object Microsoft.Dynamics.AX.Metadata.Storage.DiskProvider.DiskProviderConfiguration
$diskProviderConfiguration.AddMetadataPath($PackageDirectory)
$metadataProviderFactoryViaDisk = New-Object Microsoft.Dynamics.AX.Metadata.Storage.MetadataProviderFactory
$metadataProviderViaDisk = $metadataProviderFactoryViaDisk.CreateDiskProvider($diskProviderConfiguration)

$diskModels = $metadataProviderViaDisk.ModelManifest.ListModelInfos()

foreach ($model in $models) {
    if ($diskModels.Name -NotContains $model.Name) {
        $model.IsBinary = $true
    }
}

$uncompiledModels = @(
    foreach ($model in $diskModels) {
        if ($models.Name -NotContains $model.Name) {
            $model | Add-Member -MemberType NoteProperty -Name 'IsBinary' -Value $false
            $model
        }
    }
)

# Combined both arrays
$models = $models + $uncompiledModels

$models

[TrudAX]

Looks good, AAATest is in the output

[Splaxi]

Will release things tomorrow (Wednesday-Europe) during normal business hours.

[Splaxi]

0.7.1 has the changes. Take it for a spin.