Broken module installation due to older NuGet package provider

[ralish]

When installing the modular AWS PowerShell Tools under PowerShell 5.1 the modules installed by Install-AWSToolsModule may be in a broken state and unable to be imported into the session. No errors are reported during the installation process to indicate anything went wrong.

Expected Behavior

Installation of modules via Install-AWSToolsModule should result in a working installation.

Current Behavior

While the modules appear to be installed they are not visible in the output of Get-Module -ListAvailable -Name AWS.Tools.* and cannot be imported:

Import-Module -Name AWS.Tools.S3 -Verbose
VERBOSE: Skipping the Version folder 4.0.5 under Module C:\Users\<removed>\Documents\WindowsPowerShell\Modules\AWS.Tools.S3 as it does not have a valid module manifest file.
Import-Module : The specified module 'AWS.Tools.S3' was not loaded because no valid module file was found in any module directory.
At line:1 char:1
+ Import-Module -Name AWS.Tools.S3 -Verbose
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (AWS.Tools.S3:String) [Import-Module], FileNotFoundException
    + FullyQualifiedErrorId : Modules_ModuleNotFound,Microsoft.PowerShell.Commands.ImportModuleCommand

As the exception hints at the module itself is installed in a SxS configuration but with an incorrect version: 4.0.5. The revision field has been removed and so the directory name doesn't match the version in the module manifest.

The actual root cause of this was a pain to track-down:

1. The issue occurs with the NuGet v2.8.5.208 package provider which is installed on running Install-Module -Name AWS.Tools.PowerShell, assuming it's not already installed or the installed version is older than this release. The provider is needed to download packages from PSGallery.
2. The AWS.Tools.PowerShell module will download updated versions of the PowerShellGet and PackageManagement modules, the latter as a dependency of the former.
3. The updated PackageManagement module includes a newer version of the NuGet package provider. At the time of writing the v1.4.7 module includes v3.0.0.1 of the provider.

The v3.0.0.1 provider has several major changes to package installation behaviour. One of those is it sets the version number for the module installation directory correctly, as it will try and parse the module manifest and extract the version number. The v2.8.5.208 provider instead just directly uses the version number from the NuSpec. Except, NuSpec versions don't include a revision number.

Where this gets tricky is that once the v2 NuGet package provider is imported, the v3 NuGet package provider can't be imported into the same PowerShell session. It will be used for new PowerShell sessions, so the workaround is to simply start a new PowerShell session after installing AWS.Tools.Installer if the NuGet package provider was updated.

Possible Solution

A couple of options:

1. Throw an exception on module installation if the loaded NuGet package provider is less than v3 and instruct the user to start a new PowerShell session.
2. Only use version numbers with three fields (i.e. ditch the revision number) in PowerShell module manifests. This should be compatible with the v2 NuGet package provider.
3. More... ?

Steps to Reproduce (for bugs)

Below steps assume PowerShell 5.1. I'm guessing PowerShell Core bundles a newer version of the PackageManagement module, so may not be affected, but I haven't explicitly tested this.

# Ensure you've removed updated PackageManagement and PowerShellGet modules
#
# You should only see v1.0.0.1 versions for both modules on running this command:
Get-Module -ListAvailable -Name PackageManagement, PowerShellGet

# Install AWS PowerShell Tools & at least one tool module
Install-Module -Name AWS.Tools.Installer -Force
Install-AWSToolsModule -Name S3 -Force

# Try and list or import the module in the same session
Get-Module -ListAvailable -Name AWS.Tools.*
Import-Module -Name AWS.Tools.S3

Context

In vanilla installations of PowerShell 5.1 (e.g. on new servers) the modular AWS Tools for PowerShell is likely to be in a broken state on first install. The broken module installs must be either manually removed or the directory names fixed, as PowerShell itself won't be able to handle them.

Your Environment

Include as many relevant details about the environment where the bug was discovered.

• Affected module: AWS.Tools.Installer
• Module version as reported by Get-AWSPowerShellVersion: 4.0.5.0
• Operating System and version as reported by $PSVersionTable.OS: Property not present on PowerShell 5.1 (Windows Server 2016)
• PowerShell version as reported by $PSVersionTable.PSEdition: Desktop

[luckerby]

@ralish you're spot on. Just hit this today, using the modularized AWS.Tools.

I couldn't figure out why the AWS.Tools.IdentityManagement module was unusable, in the sense that it wouldn't show in the output of Get-Module -ListAvailable -Name AWS* nor could manually import it (Import-Module : The specified module 'AWS.Tools.IdentityManagement' was not loaded because no valid module file was found in any module directory). Picking up the ModuleVersion PS variable from within the .psd1 files - for both the AWS.Tools.IdentityManagement and its AWS.Tools.Common dependency - which is currently 4.1.2.0 - and renaming both containing directories (incorrectly named by the install cmdlet as 4.1.2) made everything work as expected.

[ashishdhingra]

This issue was observed while implementing fix for https://github.com/aws/aws-tools-for-powershell/issues/194 where revision (4th last part) was omitted during installation in certain instances in Windows PowerShell 5.1. It self-resolved after few attempts. The PowerShell session should use the latest version of modules automatically. Fix for https://github.com/aws/aws-tools-for-powershell/issues/194 would be included in next PowerShell release.

[ashishdhingra]

This should be fixed in AWS.Tools.Installer 1.0.2.1

[ThisIsArnab]

This issue is still present for AWS.Tools.Installer 1.0.2.1. For instance the folder name for EC2 module is missing the 4th revision (last part). It is 4.1.12 instead of 4.1.12.0