Question about extension vault module loading

[cansuerdogan]

I would like your advice on a problem I'm running into while using SecretManagement for CI environments. My use case includes using Az.KeyVault as the SecretVault and I see that it's not an official extension vault yet, so if that is somewhat the reason please let me know.

Let's say our intention is to provide credentials to our Artifactory repositories by using SecretManagement so developers can access these credentials and pass them into PowerShellGet cmdlets. Our CI nodes will be set up with something like this:

Install-Module Microsoft.PowerShell.SecretManagement
Install-Module Az.KeyVault                                       -> grabs the latest version as of CI node deployment
Register-SecretVault -ModuleName Az.KeyVault -Name artifactorykv -VaultParameters @{ AZKVaultName = "artifactorykv"; SubscriptionId = "<GUID>" }

and then we will advertise that developers can use it in their build scripts like this:

$username = Get-Secret -Vault artifactorykv -Name ArtifactoryUser -AsPlainText
$password = Get-Secret -Vault artifactorykv -Name ArtifactoryPassword
$artifactoryCreds = New-Object System.Management.Automation.PSCredential ($username, $password)
Save-Module -Name MyModule -Path C:\MyBuildDir\Modules -Repository Artifactory -Credential $artifactoryCreds
$env:PSModulePath = "C:\MyBuildDir\Modules;" + $env:PSModulePath
Import-Module MyModule

The problem is, once they invoke Get-Secret in their PowerShell session, SecretManagement will load the Az.KeyVault module we installed during CI node prep and therefore the version of Az.KeyVault that was grabbed at the time which is highly likely different than what the build scripts are pinning as their dependency. For example, in our case, we have the majority of our CI pipelines still using Az v3.0.0 which pulls Az.KeyVault v1.3.1 as opposed to the latest v3.4.3 and there are deprecations between those version jumps such as SecretValueText. SecretManagement loading the latest version of the module into the same session will start failing a lot of builds.

What are our options here? Do we need to work around this problem by never saving and importing modules in the same session? Could SecretManagement be updated to handle these operations without affecting the current PS session by unloading the extension vault module somehow?

I'm trying to imagine how it would work if PowerShellGet used SecretManagement behind the scenes for credential persistence. With this integration, users wouldn't need to explicitly call Get-Secret, but yet they would still suddenly be broken because some other Az.KeyVault version is being pulled down to their session implicitly as they call Save-Module which gets the creds from PSRepository's Authentication using SecretManagement. It would be very confusing.

[PaulHigin]

First, SecretManagement requires a module that meets the special requirements for a SecretManagement extension vault module. For more information about SecretManagement extension vaults see Readme and DesginDoc. The AzKeyVault team (or someone) would have to create a SecretManagement extension vault, that would presumably be based on and work with advancing versions of AzKeyVault module, or some REST API. I believe the AzKeyVault team is working on such an extension vault module and @SydneyhSmith may have more information on this.

Each SecretManagement extension vault has its own registration and configuration strategy, that may or may not include non-interactive script base use for CI.

For example, the SecretStore extension vault module, which I am the most familiar with, can be configured for both interactive and non-interactive use (for CI applications). However, it is a vault that stores secrets locally, and is arguably less secure than a cloud based vault such as an AzKeyVault based extension vault module. There is some discussion in the SecretStore vault repo about how to use it in CI scenarios, in the Issues section, and may be relevant to what you are doing.

[cansuerdogan]

Thanks for the information @PaulHigin .

My question wasn't so much related to non-interactive vault setup but the extension vault module hosting and I've noticed that you've actually addressed this exact problem in this PR. My description was maybe a bit complicated. For clarification, here is a comparison of what I was running into with the current released version (1.0.0) and how it's no longer an issue in the upcoming version (1.1.0).

I will close this issue and wait for the official release of 1.1.0. Thank you!

[PaulHigin]

I see. So there was a conflict with loaded Az modules with v1.0.0? I was always worried about this in the back of my mind, so it is good to know the change in v1.1.0 hosting has this added benefit.

[cansuerdogan]

Correct. It seems like

1. SecretManagement was attempting to use the module version last imported in the session (which can be an older version that does not have the vault extension nested module and that breaks SecretManagement functions) rather than loading the module from ModulePath saved in vaultinfo during Register-SecretVault
2. Depending on if and when SecretManagement cmdlets are invoked in a script, there could be multiple versions of Az loaded in the same session which may break certain Az use cases due to deprecations.

[PaulHigin]

Actually, these are general PowerShell module loading issues, but certainly affect SecretManagement extension vaults since they are based on PowerShell modules.

The module version problem becomes worse if the module uses .NET binaries. .NET won't allow different versions of the same binary to be loaded in the same process. So my separate runspace hosting solution won't help here, since all runspaces exist the same process and different (binary) module versions cannot be loaded.

I think the best solution is to try and ensure only one version of a module is installed on a machine. PowerShell will load the latest version of a module by default, but if you load specific versions then conflicts can arise.

[cansuerdogan]

Thanks for pointing out the .NET binary limitation, I was not aware of that, it's good to know.

In general, we are circumventing this situation by not installing any PowerShell modules on our CI nodes and telling teams to Save and Import their dependencies using the $env:PSModulePath trick. This way, the environment variable is only updated in their PS session/process pointing to their unique build folder, they can use whatever versions of modules they'd like and they don't impact other builds that may be running on the same node since those other builds and sessions do not know about the existence of these modules.

On a related note, we actually can't follow this same advice to set up SecretVaults if the module has extra dependencies, and Az.KeyVault does depend on Az.Accounts. So, for example,

1. Save-Module
2. update PSModulePath
3. Import-Module
4. Register-SecretVault, and then,
5. Get-Secret in a different PS window

works for SecretStore because its only dependency is SecretManagement. (For now we can probably safely assume CI builds won't have a use for SecretManagement outside of what we provide as administrators so we will be installing that on the machine and PSModulePath will include it.)

This same workflow doesn't work for Az.KeyVault though because in the second PS window it doesn't know where it should be loading Az.Accounts from.

(This is now an entirely different ask and may not belong in this issue) Would it be possible to get SecretManagement to maybe include the parent folder of ModulePath (in the above example that would be S:\AzModules) in PSModulePath to resolve any possible dependencies? Or is this very specific to Az.KeyVault (or a consequence of general PowerShell module loading) and SecretManagement shouldn't really care?