viceice
commented
2 years ago a workaround, which should already work is using regex manager, use nugget datasource and assign PowerShell gallery as registry url .
Open SphenicPaul opened 2 years ago
viceice
commented
2 years ago a workaround, which should already work is using regex manager, use nugget datasource and assign PowerShell gallery as registry url .
github-actions[bot]
commented
2 years ago Hi there,
You're requesting support for a new package manager. We need to know some basic information about this package manager first. Please copy/paste the new package manager questionnaire, and fill it out in full.
Once the questionnaire is filled out we will evaluate if adding support for this manager is something we want to do.
Good luck,
The Renovate team
SphenicPaul
commented
2 years ago Thanks @viceice
I've managed to trawl through the documentation and (with more than a little trial and error) I've been able to get some suitable regexManagers configuration setup/defined that makes use of the public, PowerShell gallery (https://www.powershellgallery.com/) as a package/module source and caters for the 3 scenarios I'd highlighted in the original issue:
1) The #Requires -Module declaration/parameters in .ps1 (and .psm1) scripts
2) The RequiredModules section of the .psd1 (module manifest) file
3) Install-Module and/or Import-Module functions/cmdlet usages
Incase somebody might find this useful, the regexManagers config, with some examples of the patterns found are below (Note: The regular expressions could potentially be more concise and performant, but seems to functionality provide what I'd required).
Note that, in my case/scenario, I'm not going to be referencing the public, PowerShell gallery directly in my own config, but I've also been able to get this working within Azure DevOps, using a private, NuGet repository (referenced in the config) which is then using the pubic, PowerShell gallery as an "Upsteam Source". I've left in the commented-out lines below (replace the placeholders - yourorgnamehere and yournugetreponamehere, but you'd also need to setup the credentials for the internal feed elsewhere in your config.
Additionally, I've also not included MaximumVersion, PowerShell cmdlet/function parameters in my config/regex because my assumption is that if I'm making use of MaximumVersion then I don't want Renovate to auto-upgrade/amend these (although you should be able to amend the below regex relatively easily if you did want that functionality).
module.exports = {
...<snip>...
regexManagers: [
{
"fileMatch": [
"^.*\\.psd1", // PowerShell, 'Module Manifest' file(s)
],
"matchStrings": [
// 'RequiredModules' usages (Single and multi-line) - Examples:
// RequiredModules={@{ModuleName='SomeModuleName'; ModuleVersion="0.10.0"}}
// @{ModuleName='SomeOtherModuleName'; ModuleVersion="0.10.0"}
// @{ModuleName='YetAnother.ModuleName'; ModuleVersion='0.10.0'; Guid='a622dea5-2a7a-4616-a270-1f7abb777e71'}
"(?i)\\bModuleName\\b\\s*=\\s*[\\'\\\"](?<depName>.*?)[\\'\\\"]\\s*\\;?\\s*(?i)\\bModuleVersion\\b\\s*=\\s*[\\'\\\"](?<currentValue>.*?)[\\'\\\"]",
],
"datasourceTemplate": "nuget", // NuGet is used by PowerShell(Get)/PackageManagment
"registryUrlTemplate": "https://www.powershellgallery.com/api/v2/",
//"registryUrlTemplate": "https://yourorgnamehere.pkgs.visualstudio.com/_packaging/yournugetreponamehere/nuget/v3/index.json",
},
{
"fileMatch": [
"^.*\\.ps1", // PowerShell, 'Script' file(s)
"^.*\\.psm1", // PowerShell, 'Module Script' file(s)
],
"matchStrings": [
// '#Requires' usages (Note: Excludes 'MaximumVersion' parameter) - Examples:
// #Requires -Modules @{ ModuleName = "SomeModuleName" ; RequiredVersion = '0.1.0' }
// #Requires -Modules @{ ModuleName="SomeOtherModuleName"; ModuleVersion="1.2.3" }
"(?i)#\\bRequires\\b.*\\bModules\\b.*\\bModuleName\\b\\s*=\\s*[\\'\\\"](?<depName>.*?)[\\'\\\"]\\s*\\;?\\s*(\\bRequiredVersion\\b|\\bModuleVersion\\b)\\s*=\\s*[\\'\\\"](?<currentValue>.*?)[\\'\\\"]",
// "Install-Module" and "Import-Module" (named-parameter) usages (Note: Excludes 'MaximumVersion' parameter) - Examples:
// Install-Module -name 'SomeModuleName' -MinimumVersion '4.2.1'
// Import-Module -Name 'SomeOtherModuleName' -MinimumVersion '1.2.3'
// Install-Module -name 'YetAnother.ModuleName' -SomeOtherParameter 'ABC' -AnotherParameter "xyz" -RequiredVersion '2.3.1'
// Import-Module -name "One.Final.Module.Name" -SomeOtherParameter 'abc' -RequiredVersion "1.2.4"
"(?i)(\\bImport\\b|\\bInstall\\b)-Module\\s*\\-\\bName\\b\\s*[\\'\\\"](?<depName>.*?)[\\'\\\"].*\\-(\\bRequiredVersion\\b|\\bMinimumVersion\\b)\\s*[\\'\\\"](?<currentValue>.*?)[\\'\\\"]",
// "Install-Module" and "Import-Module" (parameter-position-zero) usages (Note: Excludes 'MaximumVersion' parameter) - Examples:
// Install-Module 'SomeModuleName' -SomeOtherParameter 'XYZ' -RequiredVersion '0.1.2'
// Import-Module "SomeOtherModuleName" -SomeOtherParameter 'xyz' -RequiredVersion "3.1.0"
// Install-Module 'YetAnother.ModuleName' -SomeOtherParameter 'ABC' -AnotherParameter "xyz" -MinimumVersion '0.1.2'
// Import-Module "One.Final.Module.Name" -SomeOtherParameter 'abc' -MinimumVersion "0.1.3"
"(?i)(\\bImport\\b|\\bInstall\\b)-Module\\s*[\\'\\\"](?<depName>.*?)[\\'\\\"].*\\-(\\bRequiredVersion\\b|\\bMinimumVersion\\b)\\s*[\\'\\\"](?<currentValue>.*?)[\\'\\\"]",
],
"datasourceTemplate": "nuget", // NuGet is used by PowerShell(Get)/PackageManagment
"registryUrlTemplate": "https://www.powershellgallery.com/api/v2/",
//"registryUrlTemplate": "https://yourorgnamehere.pkgs.visualstudio.com/_packaging/yournugetreponamehere/nuget/v3/index.json",
},
],
...<snip>...
};
Also, while this does fulfil my short-term requirements, I'll complete the `New package manager questionnaire' and add to this thread - Whether it wants/needs to be a new package manager, can be decided upon at a later point.
SphenicPaul
commented
2 years ago Did you read our documentation on adding a package manager?
PowerShellGet / PackageManagement
PowerShell (specifically for PowerShell 'Modules') including PowerShell Desired State Configuration (DSC) modules/components.
PowerShell Gallery metrics (as of 19/04/2022):
PowerShellGet / PackageManagement used within PowerShell is based upon NuGet package management. Installed packages (typically, PowerShell modules)
^.*\\.psd1$ - PowerShell, 'Module Manifest' file(s) - A document of metadata describing the module (typically, a PowerShell Data file)^.*\\.ps1$ // PowerShell, 'Script' file(s) - Could be any PowerShell script or function/cmdlet etc.^.*\\.psm1$ // PowerShell, 'Module Script' file(s) - Typically, executed as part of loading/initialising a module (typically treated a little like a .ps1 file)Feasibly, the above would recognise all .ps1, PowerShell scripts in a repository, for example, but any one, or all of those scripts may contain the functionality for finding, installing and importing modules (Find-Module, Install-Module and Import-Module, respectively). The .psm1 files would be similar to the .ps1 script, but there might be less likelyhood of similar functionality being included/needed in that file type.
In the case of the .psd1 file - There may be other .psd1 files used (it is a PowerShell data file, afterall), but in most cases, these will be much less common (compared to .ps1 files/scripts), and the structure of the content of the .psd1 file will likely highlight this as a PowerShell module manifest (which holds the module dependencies and versions, where specified).
No.
No.
RequiredModules section of this tableRequiredModules.ps1 - I've also seen modules in files like this, making use of PSDepend-ModuleVersion, -RequiredVersion, -MinimumVersion or -MaximumVersion parameters) as part of the functionality of those scripts. For example:
Import-Module 'SomeModuleName` -RequiredVersion '1.2.3'.psd1 file module examples and regex :
// 'RequiredModules' usages (Single and multi-line) - Examples:
// RequiredModules={@{ModuleName='SomeModuleName'; ModuleVersion="0.10.0"}}
// @{ModuleName='SomeOtherModuleName'; ModuleVersion="0.10.0"}
// @{ModuleName='YetAnother.ModuleName'; ModuleVersion='0.10.0'; Guid='a622dea5-2a7a-4616-a270-1f7abb777e71'}
"(?i)\\bModuleName\\b\\s*=\\s*[\\'\\\"](?<depName>.*?)[\\'\\\"]\\s*\\;?\\s*(?i)\\bModuleVersion\\b\\s*=\\s*[\\'\\\"](?<currentValue>.*?)[\\'\\\"]",.ps1 and .psm1 file module examples and regex :
// '#Requires' usages (Note: Excludes 'MaximumVersion' parameter) - Examples:
// #Requires -Modules @{ ModuleName = "SomeModuleName" ; RequiredVersion = '0.1.0' }
// #Requires -Modules @{ ModuleName="SomeOtherModuleName"; ModuleVersion="1.2.3" }
"(?i)#\\bRequires\\b.*\\bModules\\b.*\\bModuleName\\b\\s*=\\s*[\\'\\\"](?<depName>.*?)[\\'\\\"]\\s*\\;?\\s*(\\bRequiredVersion\\b|\\bModuleVersion\\b)\\s*=\\s*[\\'\\\"](?<currentValue>.*?)[\\'\\\"]",
// "Install-Module" and "Import-Module" (named-parameter) usages (Note: Excludes 'MaximumVersion' parameter) - Examples:
// Install-Module -name 'SomeModuleName' -MinimumVersion '4.2.1'
// Import-Module -Name 'SomeOtherModuleName' -MinimumVersion '1.2.3'
// Install-Module -name 'YetAnother.ModuleName' -SomeOtherParameter 'ABC' -AnotherParameter "xyz" -RequiredVersion '2.3.1'
// Import-Module -name "One.Final.Module.Name" -SomeOtherParameter 'abc' -RequiredVersion "1.2.4"
"(?i)(\\bImport\\b|\\bInstall\\b)-Module\\s*\\-\\bName\\b\\s*[\\'\\\"](?<depName>.*?)[\\'\\\"].*\\-(\\bRequiredVersion\\b|\\bMinimumVersion\\b)\\s*[\\'\\\"](?<currentValue>.*?)[\\'\\\"]",
// "Install-Module" and "Import-Module" (parameter-position-zero) usages (Note: Excludes 'MaximumVersion' parameter) - Examples:
// Install-Module 'SomeModuleName' -SomeOtherParameter 'XYZ' -RequiredVersion '0.1.2'
// Import-Module "SomeOtherModuleName" -SomeOtherParameter 'xyz' -RequiredVersion "3.1.0"
// Install-Module 'YetAnother.ModuleName' -SomeOtherParameter 'ABC' -AnotherParameter "xyz" -MinimumVersion '0.1.2'
// Import-Module "One.Final.Module.Name" -SomeOtherParameter 'abc' -MinimumVersion "0.1.3"
"(?i)(\\bImport\\b|\\bInstall\\b)-Module\\s*[\\'\\\"](?<depName>.*?)[\\'\\\"].*\\-(\\bRequiredVersion\\b|\\bMinimumVersion\\b)\\s*[\\'\\\"](?<currentValue>.*?)[\\'\\\"]",Also note, there are maybe options to pipe the Find-Module output into Install-Module, or similar like so
Find-Module 'SomeModuleName' -RequiredVersion '1.2.3' | Install-Module.. but Find-Module isn't catered for in the above examples (which are taken from my earlier post in this thread).
I don't believe any of the above dependencies are supported within Renovate as it stands, although I have been able to facilitate a similar requirement by making use of a RegexManager definition (example in earlier post).
Typically, it's 3 or 4 part, versioning (e.g. 1.2.3 or 1.2.3.4), athough, I believe 1 and two part versions are acceptable, and are treated like 3 part versions (with 0 for more granular, versioning components) - e.g. Version 1.0 would be effectively 1.0.0.
^1.0.0 or 1.x?^1.0.0 or 1.x)In the .ps1 scripts that call functions/cmdlets you are providing a single version with the parameter defining how the version provided is used. For example:
-ModuleVersion '1.22.3' - Any module version greater or equal to 1.22.3
-RequiredVersion '1.22.3' - Exact version of 1.22.3
-MinimumVersion '1.22.3' - Any module version greater or equal to 1.22.3
-MaximumVersion '1.22.3' - Any module version less than or equal to 1.22.3
In the .psd files (module manifests), typically a single version is being specified, which I believe still means that it requires any module version greater or equal to the version specified.
n/a
Technically, this is just using NuGet so might be simlar setup to that. There is a public [PowershellGallery]() which complies with NuGet v2 functionality. Package/API URI is:
Users can register their own package repositories onto a server/workstation using Register-PSRepository which will allow them to find/install packages from those locations (implicitly, because they are registered on their server/workstation, or by explicitly, providing the repository name to only search/use that repository). Note that users can provide their own names and URIs for repositories and it's unlikely they will be consistent from user to user, although note that the PSGallery repository (using the PowerShellGallery API URI above) is typically registered on Windows machines that have PackageManagement installed - I'd argue that would be a typically repository that most would have registered.
Any other user-specific/defined repositories are likely to be NuGet repositories setup within a third-party tool (e.g. Nexus Repository / Artifactory / etc.) or a file/network share. It's likely that users would have to configure Renovate to use their own custom registry URIs (as they do with NuGet, at present).
The PowerShell modules do define which version of PowerShell they are built for (primarily for preventing install of modules built for newer versions of PowerShell being able to be installed within older versions of PowerShell). I've not seen a scenario, to date, where this would impact dependency lookups/constraints - For example, you could have a PowerShell module that supports PowerShell v5.1 that references another module that supports back to PowerShell v3.
Not initially sure on this point.
These might be used behind the scenes but they don't seem to be visible in day-to-day usage. Note that most modules do have a GUID defined within the .psd1 that is meant to uniquely identify that module (although in practice, people can generate/use their own GUID so there may be clashes across some less-known used/modules, perhaps).
n/a
n/a
n/a
As previously mentioned, PowerShell PackageManagement/PowerShellGet seems to make use of NuGet packages for storing/versioning modules so I'd consider this similar to NuGet 'under the hood' but with scripts (.ps1 files) and module manifests/datafiles (.psd1 files) using and defining those modules and their dependencies.
As also mention and highlighted/exemplified in an earlier post, this can be implemented using the RegexManager - That might be suitable-enough and save an additional, package manager implementation.
Feel free to shout/tag me if you've any further questions etc.
viceice
commented
2 years ago i played with those modules already as part of building CI server / dev machines. so i can say we can reuse nuget versioning and datasource. we just need to use a different registry url for Powershell Gallery by default.
What would you like Renovate to be able to do?
Having found Renovate recently, and found it very useful for .NET, Docker, Terraform and Go dependency management, it doesn’t currently support the recognition and upgrades of PowerShell modules and scripts.
Example PowerShell scripts might include files with ‘.ps1’, ‘.psd1’ and ‘.psm1’ extensions, and typically would reference other PowerShell modules (and versions) by either: 1) The ‘#Requires -Module’ declaration/parameter as detailed here: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_requires?view=powershell-7.2#-modules-module-name--hashtable 2) A specific version of a required module defined within “RequiredModules” section of the ‘.psd1’ file (typically a PowerShell, module manifest), as defined under the ‘RequiredModules’ row in the following table: https://docs.microsoft.com/en-us/powershell/scripting/developer/module/how-to-write-a-powershell-module-manifest?view=powershell-7.2 3) References/Usages within scripts/files where the ‘Install-Module’ and/or ‘Import-Module’ functions/cmdlets are used with the ‘-RequiredVersion’/‘-MinimumVersion’/‘-MaximumVersion’ parameter passed/used (although I suspect the ‘-MaximumVersion’ has less relevance in this use case).
If you have any ideas on how this should be implemented, please tell us here.
PowerShell PackageManagement/PowerShellGet uses NuGet functionality behind-the-scenes (PowerShell repos are NuGet repos, and PowerShell module packages are NuGet packages) so there are potentially workarounds for this by setting up configurations to include additional files/patterns/repositories under the NuGet-specific configuration (I’ve not looked further into this yet, but if anyone has easy/simple config examples to point me in right direction, that would be helpful). The output of any work (interim or permanent) to meet this requirement in this way could just be some examples (for PowerShell dependencies by using NuGet) within the documentation.
In lieu of a NuGet-based workaround, another option might be to make PowerShell a first-class citizen of this (so it has it’s own configuration sections/objects/etc.) - If deemed valid/useful enough to be added, I’d also be intersect to know a suitable approach to take to getting this functionality added - I might try and find a little time to get something submitted.
Is this a feature you are interested in implementing yourself?
Maybe