Problems with Microsoft.Powershell.SecretManagement

[Callidus2000]

Hi, I'm using your action for my validation and build workflows. My current module needs the following dependencies: RestartableSession,PSFramework,string,PSModuleDevelopment,Microsoft.Powershell.SecretManagement,ChangelogManagement

First run (building the cache) works, afterwards (relying on the cache) everythings works but Microsoft.Powershell.SecretManagement is missing.

In the first run the workflow

Run potatoqualitee/psmodulecache@main
Run Import-Module $env:GITHUB_ACTION_PATH/PSModuleCache.psd1
Getting variables for cacher
Action Parameters = @{Modules=RestartableSession,PSFramework,string,PSModuleDevelopment,Microsoft.Powershell.SecretManagement,ChangelogManagement; PrefixIdentifier=False; PrereleaseModules=; ContainerJob=False; Updatable=False; Shells=pwsh}
Run actions/cache@v3.0.11
Cache not found for input keys: Linux-5.2-Immutable-pwsh-RestartableSession-PSFramework-string-PSModuleDevelopment-Microsoft.Powershell.SecretManagement-ChangelogManagement
Run Write-Output "Saving all modules"
Saving all modules
Existing repositories 'PSGallery'
Saving module 'RestartableSession' version '' to '/usr/local/share/powershell/Modules/'. Search in the following repositories 'PSGallery'
    Module 'RestartableSession' version '0.4.0' found in 'PSGallery'.
Saving module 'PSFramework' version '' to '/usr/local/share/powershell/Modules/'. Search in the following repositories 'PSGallery'
    Module 'PSFramework' version '1.7.270' found in 'PSGallery'.
Saving module 'string' version '' to '/usr/local/share/powershell/Modules/'. Search in the following repositories 'PSGallery'
    Module 'string' version '1.1.3' found in 'PSGallery'.
Saving module 'PSModuleDevelopment' version '' to '/usr/local/share/powershell/Modules/'. Search in the following repositories 'PSGallery'
    Module 'PSModuleDevelopment' version '2.2.11.139' found in 'PSGallery'.
Saving module 'Microsoft.Powershell.SecretManagement' version '' to '/usr/local/share/powershell/Modules/'. Search in the following repositories 'PSGallery'
    Module 'Microsoft.PowerShell.SecretManagement' version '1.1.2' found in 'PSGallery'.
Saving module 'ChangelogManagement' version '' to '/usr/local/share/powershell/Modules/'. Search in the following repositories 'PSGallery'
    Module 'ChangelogManagement' version '3.0.1' found in 'PSGallery'.

second run

Run potatoqualitee/psmodulecache@main
Run Import-Module $env:GITHUB_ACTION_PATH/PSModuleCache.psd1
Getting variables for cacher
Action Parameters = @{Updatable=False; ContainerJob=False; Shells=pwsh; Modules=RestartableSession,PSFramework,string,PSModuleDevelopment,Microsoft.Powershell.SecretManagement,ChangelogManagement; PrereleaseModules=; PrefixIdentifier=False}
Run actions/cache@v3.0.11
Received 1505700 of 1505700 (100.0%), 13.1 MBs/sec
Cache Size: ~1 MB (1505700 B)
/usr/bin/tar -z -xf /home/runner/work/_temp/a65b75ea-18d9-4d13-8796-a24caf52dbc2/cache.tgz -P -C /home/runner/work/SecretManagement.ExtensionTemplate/SecretManagement.ExtensionTemplate
Cache restored successfully
Cache restored from key: Linux-5.2-Immutable-pwsh-RestartableSession-PSFramework-string-PSModuleDevelopment-Microsoft.Powershell.SecretManagement-ChangelogManagement

[LaurentDardenne]

Module dependencies are not managed today, I'm testing this part. I plan to deliver within a month, maybe less.

As PSModuleCache allows to address several repositories, I have to check different use cases. I encounter some bugs on PsGet, for example if there are redundant modules in several repositories, Find-module -IncludeDepencies returns the dependencies according to the order of declarations of PSRepositories and the external dependencies are not raised by Find-module -IncludeDepencies.

For the moment to correct this problem, it is necessary to explain the dependent modules in the parameters of the action.

[Callidus2000]

Sorry, chosen the wrong word... My module requires Microsoft.Powershell.SecretManagement

    RequiredModules   = @(@{ModuleName = 'RestartableSession'; ModuleVersion = '0.4.0'; },
        @{ModuleName = 'PSFramework'; ModuleVersion = '1.7.249'; },
        @{ModuleName = 'string'; ModuleVersion = '1.0.0'; },
        @{ModuleName = 'PSModuleDevelopment'; ModuleVersion = '2.2.11'; },
        @{ModuleName = 'Microsoft.Powershell.SecretManagement'; ModuleVersion = '1.1.2'; })

and uses in the workflow yaml

- name: Install and cache PowerShell modules
  id: psmodulecache
  uses: potatoqualitee/psmodulecache@v5.2
  with:
    modules-to-cache: RestartableSession,PSFramework,string,PSModuleDevelopment,Microsoft.Powershell.SecretManagement,ChangelogManagement

Found out the hardway that dependencies are (currently) not been dealt with, thats why I've added string as a direkt module.

Therefor dependency was wrong, the required module ist not preserved in the cache.

Taking a look at one of the debug logs 11_Install and cache PowerShell modules.txt

I've seen that in the tar call only the other paths are mentioned:

• usr/local/share/powershell/Modules/RestartableSession/
• usr/local/share/powershell/Modules/PSFramework/
• usr/local/share/powershell/Modules/string/
• usr/local/share/powershell/Modules/PSModuleDevelopment/
• usr/local/share/powershell/Modules/ChangelogManagement/
• usr/local/share/powershell/Modules/platyPS/

[LaurentDardenne]

I've seen that in the tar call only the other paths are mentioned

The management of '.' (dot char) is taken into account when generating the key and the path is inserted in the list of paths to be saved by Tar. I need to test this case on Linux and Windows.

[LaurentDardenne]

I have no problem with this syntax (Ubuntu):

      - name: Cache modules
        id: psmodulecache
        uses: ./
        with:
          modules-to-cache: RestartableSession,PSFramework,string,PSModuleDevelopment,Microsoft.Powershell.SecretManagement,ChangelogManagement
          shell: powershell,pwsh
          updatable: "false"

      - name: Get required modules on Windows PowerShell
        shell: pwsh
        run: |
          'RestartableSession','PSFramework','string','PSModuleDevelopment','Microsoft.Powershell.SecretManagement','ChangelogManagement'|
          foreach {
             Import-Module $_
             Write-Host "$_ '$(Get-Module $_ -ListAvailabLe|Select-Object Name,Version,@{Name='PreRelease';e={$_.PrivateData.PSData.Prerelease}},RepositorySourceLocation)'"
          }

But I haven't tested with this syntax (dynamic search for module dependencies), like you do in your workflow :

    - name: Query needed modules
      id: getReqModules
      shell: pwsh
      run: |
          "RequiredModules=$((Import-PowerShellDataFile "./$($env:MODULENAME)/$($env:MODULENAME).psd1").RequiredModules.modulename -join ',')" >> $env:GITHUB_OUTPUT
          # (Import-PowerShellDataFile ./SecretManagement.ExtensionTemplate/SecretManagement.ExtensionTemplate.psd1).RequiredModules.modulename -join ','
    - name: Install and cache PowerShell modules
      id: psmodulecache
      uses: potatoqualitee/psmodulecache@main
      with:
        modules-to-cache: ${{ steps.getReqModules.outputs.RequiredModules }},ChangelogManagement

[LaurentDardenne]

Can you test with this order of dependencies?

 RequiredModules   = @(@{ModuleName = 'RestartableSession'; ModuleVersion = '0.4.0'; },
        @{ModuleName = 'PSFramework'; ModuleVersion = '1.7.249'; },
        @{ModuleName = 'PSModuleDevelopment'; ModuleVersion = '2.2.11'; },
        @{ModuleName = 'Microsoft.Powershell.SecretManagement'; ModuleVersion = '1.1.2'; },
        @{ModuleName = 'string'; ModuleVersion = '1.0.0'; })

[Callidus2000]

Tested, same behavior, works once (as key changes), further iterations fail.

[Callidus2000]

Cut down my workflow to non fancyness:

name: CheckOnPR-Module-Debug
on: 
  workflow_dispatch:
    inputs:
      debug_enabled:
        type: boolean
        description: 'Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)'
        required: false
        default: false
jobs:
  CheckOnPR-Module-Debug:

    runs-on: ubuntu-latest
    defaults:
      run:
        shell: pwsh
    steps:
    - name: Install and cache PowerShell modules
      id: psmodulecache
      uses: potatoqualitee/psmodulecache@main
      with:
        modules-to-cache: RestartableSession,PSFramework,string,PSModuleDevelopment,Microsoft.Powershell.SecretManagement,ChangelogManagement
        updatable: "false"
        shell: powershell,pwsh
    - name: Make sure modules are installed
      shell: pwsh
      run: |
        'RestartableSession','PSFramework','string','PSModuleDevelopment','Microsoft.Powershell.SecretManagement','ChangelogManagement'|
        foreach {
            Write-Host "$_ '$(Get-Module $_ -ListAvailabLe|Select-Object Name,Version,@{Name='PreRelease';e={$_.PrivateData.PSData.Prerelease}},RepositorySourceLocation)'"
            Import-Module $_
        } 
    - name: Setup tmate session
      # if: ${{ failure() || (github.event_name == 'workflow_dispatch' && inputs.debug_enabled ) }}
      if: ${{ (github.event_name == 'workflow_dispatch' && inputs.debug_enabled ) }}
      uses: mxschmitt/action-tmate@v3

But the result from the second run on is the same:

Run 'RestartableSession','PSFramework','string','PSModuleDevelopment','Microsoft.Powershell.SecretManagement','ChangelogManagement'|
  'RestartableSession','PSFramework','string','PSModuleDevelopment','Microsoft.Powershell.SecretManagement','ChangelogManagement'|
  foreach {
      Write-Host "$_ '$(Get-Module $_ -ListAvailabLe|Select-Object Name,Version,@{Name='PreRelease';e={$_.PrivateData.PSData.Prerelease}},RepositorySourceLocation)'"
      Import-Module $_
  } 
  shell: /usr/bin/pwsh -command ". '{0}'"
RestartableSession '@{Name=RestartableSession; Version=0.4.0; PreRelease=; RepositorySourceLocation=https://www.powershellgallery.com/api/v2}'
PSFramework '@{Name=PSFramework; Version=[1](https://github.com/Callidus2000/SecretManagement.ExtensionTemplate/actions/runs/4438613010/jobs/7789907683#step:3:1).7.[2](https://github.com/Callidus2000/SecretManagement.ExtensionTemplate/actions/runs/4438613010/jobs/7789907683#step:3:2)70; PreRelease=; RepositorySourceLocation=https://www.powershellgallery.com/api/v2}'
string '@{Name=string; Version=1.1.[3](https://github.com/Callidus2000/SecretManagement.ExtensionTemplate/actions/runs/4438613010/jobs/7789907683#step:3:3); PreRelease=; RepositorySourceLocation=https://www.powershellgallery.com/api/v2}'
PSModuleDevelopment '@{Name=PSModuleDevelopment; Version=2.2.11.139; PreRelease=; RepositorySourceLocation=https://www.powershellgallery.com/api/v2}'
Microsoft.Powershell.SecretManagement ''
Import-Module: /home/runner/work/_temp/5bb[4](https://github.com/Callidus2000/SecretManagement.ExtensionTemplate/actions/runs/4438613010/jobs/7789907683#step:3:4)03bb-9281-4bb4-a[5](https://github.com/Callidus2000/SecretManagement.ExtensionTemplate/actions/runs/4438613010/jobs/7789907683#step:3:5)16-da6f5b4603b5.ps1:5
Line |
   5 |      Import-Module $_
     |      ~~~~~~~~~~~~~~~~
     | The specified module 'Microsoft.Powershell.SecretManagement' was not
     | loaded because no valid module file was found in any module directory.

[LaurentDardenne]

The latest code works on Windows and MacOS but not Ubuntu. I test the logs to check if it is not a problem in the runner. For the moment I do not see the list of files that Tar.exe must process. In any case, the module builds the list of paths to save in the cache.

[LaurentDardenne]

I have done many tests and modifications but I can't isolate the problem, or build a test case :

In the logs we can see that sometimes the module is inserted in the cache, but absent from the restore, other times the restore works:

For these last tests I used the menu and not the execution of the job via an event (on push) :

[edit] In fact I'm using a development version which partly manages the manifest name coming from nuget and not parameters... :-/

[LaurentDardenne]

I think I found the problem, same as this (https://github.com/potatoqualitee/psmodulecache/issues/45). Since there is no naming standard for module manifest names (e.g. PascalCase), this problem is encountered for Linux only :

Then Action GH (or Tar.exe?) triggers warnings that are not blocking/visible. And for this I don't know if there is a setting...

[Callidus2000]

Freaky problem.... That means if I use the wrong upper/lowercase in my parameter the action does fail. Using PowerShell instead of Powershell solves my build problem, but that does not cure the root cause.

I've dived a little deeper into your action code (thankfully Powershell 😄) and build a fix for it. Will create a PR for it. Basically it uses the information from Find-Module to correct the module name which is used further on. Find-Module does not care about the input case but returns the official case.

[LaurentDardenne]

Freaky problem...

In fact only in the case where we copy pathnames of module built as does PSmoduleCache. In the Powershell runtime (regarding module management) everything is case insensitive except when publishing a module which uses the name of the manifest file to name the Nuget package. It's very likely (I haven't checked) that when importing a module from a repository, the search for a package is case-insensitive. And this problem only occurs for FS which differentiates case.